I have just started studying an Open University course called H809 'Practice-based Research in Educational Technology' that forms part of the Institute of Educational Technology's Masters in Online and Distance Education. We've been encouraged to blog about the course so I'm going to try and do a weekly post, but to hopefully write in such a way that it makes sense and might be of interest to people not doing the course.
The topic for the first three weeks of the course is 'Contextualising the field'. This week was the introductory week and as well as various orientation activities, we were asked to answer the following question on the course forums: 'What kinds of evidence and inquiry methods were appropriate to the subject you studied for your undergraduate degree?'. This was my response about mathematics.
There is a remarkably universal level of consensus among mathematicians as to what constitutes evidence for a claim. Mathematics is structured very formally with theorems (claims) and proofs (evidence) clearly indicated. Obviously sometimes mistakes in proofs creep through but there is really no debate over what constitutes a valid proof. The nearest I can think of is the arguments about whether proof 'by computer' such as that of the Four-Colour Theorem should be regarded as proof, but these are so rare in practice that they don't really impinge on your life as a mathematician. Most mathematicians leave issues like that to the philosophers of mathematics. Nonetheless I think there are still different types of belief in the veracity of a theorem. Believing a theorem because I have scrutinised the proof myself and looked for holes and tried to come up with counterexamples is very different from believing a theorem because I know lots of mathematicians have done the same. I'm not sure which of them is a stronger belief!
It is rather hard to describe what constitutes a valid proof. Indeed it is something that lots of students struggle with. I guess a proof explains how you get from stated assumptions to a conclusion using the rules of logic, possibly referencing and using the results of other theorems along the way. Mathematics is a very rigorous subject and you are not allowed to take anything for granted. You are taught to be incredibly sceptical and to pick things apart. One of the first things I had to do as an undergraduate was prove from first principles that one multiplied by zero was zero. Fitting in with this, most branches of pure mathematics have axiomatic foundations e.g. you define a group to be an object satisfying a set of rules called axioms and then figure out what you can say about objects with those properties.
The neatness of mathematical proofs also makes it easy to forget that how mathematics is created in the first place is not necessarily at all neat, even if it is presented that way afterwards. Similarly, in seminars or lectures, mathematicians will sometimes try and explain slightly more heuristically why a result is true rather than give a completely formal proof. This may involve drawing pictures for instance to illustrate what is happening in the proof. An explanation or illustration would be regarded as a means to communicate a proof better rather than a replacement for a proof though. Formal mathematics papers would certainly rarely explain how the mathematician came to think of the results and proofs. Conjecture on the other hand is also a very legitimate mathematical activity and papers often included open problems and conjecture for which the author does not have a proof.
There can also be disagreement amongst mathematicians about the significance or interestingness of the claims made, although even there I would say there is more agreement that you might imagine. It's not easy to say what makes a theorem or proof interesting. If a result solves a long-standing unsolved problem, it will generally be regarded as important. Aesthetics is certainly up there too, but I guess that is a slightly unsatisfactory answer - after all, what makes a theorem or proof beautiful?
This is obviously all a world away from research in educational technology where there are clearly ambiguities over what constitutes reasonable evidence for a claim. One of the reasons that I am doing this course is that I haven't thought about this enough and to get involved in research, you need to make personal decisions about what you believe constitutes meaningful research and you need to be able to defend those beliefs. I hope this course helps me get a bit nearer that point!
I thought that it might be interesting too, to record my current thoughts on research in educational technology to see if they change during the course. I don't want to spend my whole life wrapped up in epistemology, but on the other hand, as I mentioned in my reply to the question, I do need to think about what you consider to be meaningful research both in terms of validity and what are interesting questions to try to be answering.
I think I am most interested in educational technology as a design science (Yishay Mor I think introduced me to this notion of Herbert Simon's). Though in the same way that engineering depends on physics or medicine on biology, educational technology can obviously draw on other types of research. Linked to this, I also want to understand what nature of research can most help the people trying to improve learning and teaching either directly for people actually teaching or for people setting policies that will have a pedagogical impact.
What am I not interested in? I have strong opinions on what things are worth learning and what are not, but I'm not sure this is something I want to engage in from a research perspective. Likewise, I may be curious about institutional and cross-institutional infrastructure and set-up for e-learning but don't think I want to dedicate too much of my life to their consideration. I think too that work is essentially only descriptive may be useful but isn't my sort of thing.
In terms of how one goes about doing research, there is the question of whether one makes interventions or sticks to trying to probe what is already happening (similar to the way that an astronomer studies the universe). Interventions obviously allow you test hypotheses and explore situations that you would otherwise not be able to, but introduce a whole host of issues as well as presenting practical challenges. I suspect it is a question of using one's ingenuity to try and come up with the best combination of methods possible to examine what you are interested in.
A challenge in any educational research is the fact that people are not all the same and that context can have a massive and usually unknown impact. I suspect that there isn't any nice neat way of dealing with this, though it is interesting that there doesn't seem to be that much of a culture in educational technology research of trying to produce counterexamples to other people's claims, presumably for practical reasons. I have always thought that as with economics, we really ought to try dividing the field into 'micro' and 'macro' fields, with the former looking at smaller scale instances of learning where we may be able to show more precisely what effect context has or doesn't have, and the latter looking at say larger experiences across whole courses or degrees.
On top of all this, I am very aware that research happens in a sociocultural context and that as well as learning how to do research, one also has to learn how to participate effectively in that context. This was something that I was naive about when I did my doctorate. I am not sure I necessarily know the best way to learn this, but at least I am conscious of its importance now in a way that I wasn't a decade ago.