The Search for Research Literacy

“Sit down awhile;

And let us once again assail your ears,

That are so fortified against our story . . .” – Hamlet

‘Research’, like any other word, can acquire and shift meanings. In the last few years we have seen a shift in usage from research as quantitative evidence to much more fluid ideas around qualitative and descriptive ‘data’.

Likewise, we used to talk about ‘research-based’, but now we talk about ‘research-informed’ practice. But unless we have criteria for establishing what research is worth being informed by, the phrase is meaningless. Like every other semantic shift in the endless parade of political ideas masquerading as educational practice, placing ‘professional judgement’ and ‘teacher autonomy’ above – or even on a par with – hard evidence is a way of giving ourselves permission to do whatever we like. It only takes a few vacuous blogs, effusive ‘Principal’s Welcome’ pages on school websites, or a glance at school performance appraisal policies to see that very fuzzy ideas abound in education.

At this point the common objection is that hard evidence is impossible to come by in education; that we have to accept that learning is primarily a social interaction, that the relationships in the classroom are the most important thing and that each teacher has make a judgement in their context as to what they will do. While relationships are obviously important, that is true in every human context, and it is an entirely separate issue from whether teachers engage in the most effective practices. The refusal to believe that there are some practices which are more effective than others is a deeply held conviction by many educators, at least many that I have worked with, and many I have encountered on social media.

Can there really be a set of effective practices? What would they look like? How would we know? And even if there were, wouldn’t it be a terrible loss to teacher creativity and classroom spontaneity, which are much more important to student motivation?

In short, yes, there are such practices, and some of them have been validated over decades, replicated through single-subject and group designs. The evidence in favour of certain practices is very strong, but knowledge of this research within teaching communities is scant to say the least. As Professor Graham Nuthall said, teaching is more of a craft than a profession, and we build our practice more on folklore than on evidence. It takes time to get to grips with decades of research literature, and teachers are busy people.

We wouldn’t, of course, accept this approach in, for example, medicine or aeronautics. No one these days would expect to see a doctor who cares more about autonomy than evidence, or to fly in an aircraft  designed by an engineer who values creativity above correct procedure. The old chestnut that medicine is a ‘hard’ science and teaching isn’t, is plain wrong – medicine is intensely personal, highly subjective and often requires a great deal of inference. Doctors still get things wrong, but the improvement in medical practice, and its impact on society over the last century, have been nothing short of astounding. This did not happen because the medical profession argued about what research is, or what good practice is. It happened because there was a consensus to learn from evidence, or people would suffer and die, and that wasn’t acceptable. Yes, there are plenty of examples of poor practice, but they stand in contrast to the enormous progress that has been made.

Teachers, too, could take a more robust view of what constitutes evidence, and a much clearer view of what constitutes effective research. Neither busyness nor philosophical arguments should deter us from a fundamental quest to equip our students with the best tools we can, as early as possible. The alternative is to accept the status quo: 20% of students in secondary school struggling to read and write; millions of adults unable to deal with daily reading requirements of the most basic nature – signs, forms, newspapers. The great majority of these students – usually labeled ‘dyslexic’ – need nothing more than effective teaching. If we knew the evidence, we would act differently, and students would have better outcomes.

There is a commendable move afoot to establish research leads in schools, but there is also a risk that the movement will falter and fade because the initiative had too much freedom and not enough rigour. Schools do not need so much to conduct research themselves as ensure that their staff are familiar with the most important literature and that they are required to implement the findings of robust research faithfully. That may reduce some teacher autonomy, but autonomy is less important in education than it was; the more we know as a body, the less room we have to exercise our own preferences in spite of the evidence. Until we are familiar with the literature, and how to sort the wheat from the chaff, we are not in a position to decide what further research needs to be carried out. Through knowledge, we will save ourselves enormous work, and our students will not need to be guinea pigs whose education is sacrificed by our ignorance.

Can we imagine how the profession might be different in two or three decades? Perhaps we might have a truly research-informed profession, where all new teachers require at least two additional years of training to master’s level; where a year of becoming familiar with educational research and how to evaluate it is compulsory; where a year of actually undertaking research and submitting it for academic assessment ensures that teachers have a grasp the practical difficulties. There would be a major shift in the way in which teacher recruitment and training are resourced and organized; the master’s level pathway would be slower, and more costly, but likely result in greater retention and more effective practice. School managers, who currently often have a very good grasp of the latest Ofsted handbook but less so the useful findings of the last fifty years in educational research, would have to be a different breed in order to lead professional debate rather than suppress it.

It cannot happen overnight, but it must happen, because otherwise we are saying that our personal choice is more important than our students’ outcomes. But we are not consumers in a market; we are those in whom a great trust has been placed. A future with more of the same grinding inequality will perpetuate outcomes that threaten the very fabric of our society. We have the opportunity to establish a different future, but I sense that the window is closing fast. The democratisation of education through social media and debate may have already seen its heyday; increasingly, the discourse is aggressive, and increasingly, the bureaucrats are gaining back control of the levers of power.

Knowledge of educational research is the best way to challenge political and ideological initiatives that do not serve students, that squander resources and stunt the growth of the profession. All teachers have a responsibility to confront the issues. We do not have to agree – not at first – but we should be working towards a professional consensus based on the best knowledge we have.

I am optimistic, if only because there really isn’t any alternative.

PS: This post by Jon Brunskill resonated with me and articulates some key reasons why primary school education will benefit from greater exposure to the findings of robust research.

 

 

 

 

 

 

 

 

 

 

 

Posted in Education | Tagged , , , , | 2 Comments

How Whole Language Lost the War

Ere I could make a prologue to my brains, they had begun the play. – Hamlet.

On an otherwise empty table, under a bare electric bulb, the machine sat squat and solid. Impregnable, inscrutable, the keys ranged along its front were like so many blunt tusks, giving it an air of silent pugnacity. It was called Enigma.

The young mathematician stared at it from the shadows. He had been working on the machine for weeks, and slowly, grudgingly, it had begun to yield its secrets. The underlying principles of the code were, he knew, almost within his grasp. He rubbed red-rimmed eyes, and flexed his fingers. He had had his short break; it was time to get back to work.

Just then heavy steps sounded on the wooden steps of the hut. The door creaked open and the burly figure of Major Goodman hurried in, accompanied by a blast of freezing air. The door slammed shut. Goodman stared at the machine for a long moment, then turned to the young mathematician. “Well?” he barked. “Got it sorted yet, Turing?”

“We’ve begun to make good progress. We’ve worked out that the key changes daily, and that’s why the apparent irregularities – ”

“Begun, begun! Never mind beginnings. We’ve been at the beginning for weeks. The top brass is breathing down my neck. We need to generate some messages, d’you hear me?”

Turing rubbed his forehead wearily. He could never reconcile the major’s overbearing confidence with his utter lack of understanding. “There are over ten to the power of twenty-six possible combinations if all five wheels are employed – ”

Goodman cut him off again. “Look, Turing, no hard feelings, you’re a nice enough young fellow and I’m sure you mean well, but it’s all a bit dull. We can inject some colour and energy into this exercise by looking at it from another perspective.”

Turing stared blearily at the major’s florid face. His mouth moved, but his brain could not formulate the words to describe the hours of back-breaking, concentrated labour that he and his team had put in, twenty-four hours a day, in bare huts and isolated from friends and family. It was a labour he was prepared to offer, if at the end he had developed a system for decoding any message that the enemy cared to send. But Goodman had moved on.

“Come. Look at these messages. You see this? What’s that mean?” He had unfolded a set of papers and jabbed a stubby finger at the jumble of letters and numbers. Turing studied it.

“The first thing to do is to determine the date it was sent. Then when – ”

“Beginnings, beginnings, Turing.” Goodman plumped himself into the chair in front of the machine. “You’ve been looking at this from the wrong end of the telescope. I mean, does the government need the code, or the meaning of the message?”

“Well, it’s not possible – ”

“Of course it’s possible. Consider: We’re trying to read German messages. Reading is a psycholinguistic guessing game.  It involves an interaction between thought and language.  Efficient reading does not result from precise perception and identification of all the elements, but from skill in selecting the fewest, most productive cues necessary to produce guesses which are right the first time.”

“Who said that, sir?” Turing asked distractedly. He had an unpleasant feeling that he already knew the answer.

“I did. Now, we know that this was sent just before major troop movements in Normandy. So it’s about troop movements and those troops did in fact move. We can now predict, d’you see! Look, this part here seems to be about where the troops went to, and this part here . . .” He paused. “Not really cricket of the bally Germans to leave out the pictures,” he muttered. He searched the page. “This part here – this V O R – that’s probably Von Runstedt, their Normandy commander, what what? And here, these dots – if you stand back and squint, they form a sort of a picture. Looks a bit like . . . well, a bit like a wave breaking on the beach, and here would be the beach huts – obviously that’s the secret bit – and over here, looks like a man walking a dog.” Goodman slapped his forehead. “Great Scott! That’s it! Von Runstedt will be working his men like dogs on the beaches of Normandy.”

“Remarkable,” Turing said mildly. He felt he had lost even the power of sarcasm.

“Not remarkable at all,” snorted Goodman. “Efficient reading results from skill in selecting the fewest, most productive cues. These cues are selected from perceptual input on the basis of the reader’s expectation. They are guided by constraints set up through prior choices, his language knowledge, his cognitive styles and strategies he has learned. And while these cues provide partial information, the reader forms a perceptual image using these cues and his anticipated cues. Elementary!”

“Mmm,”said Turing.

“Then, the reader searches his memory for related syntactic, semantic, and phonological cues. This memory search may lead to selecting more graphic cues and to reforming the perceptual image.”

“Oh? Why?”

Goodman became grim. “Don’t be clever with me, Turing. These cues are necessary to produce guesses which are right the first time. And so they are. I now have a clear message to show to the top brass. Just get on with using the psycholinguistic guessing approach, and send me some more messages. Right, tip-top, must get this to HQ.”

“You have constructed a message, sir,” said Turing, leafing through the remaining papers. “But we’re guessing and the Germans aren’t. If we don’t know exactly what they meant, the intelligence is no use, and we’ll lose the war.”

But the door of the hut had swung closed, Goodman had disappeared into the night, and the machine stared back at him smugly from the table. A chill engulfed him that had nothing to do with the frosty night.

Enigma 2

You can read the real Kenneth Goodman’s thoughts on reading (many reproduced above verbatim) in his 1967 paper Reading: A psycholinguistic guessing game. These ideas formed the basis of the whole language approach to reading that has seen a consistent fifth of students arriving at secondary school unable to read at the necessary level. For a more entertaining critique of Goodman, you may wish to read Martin Kozloff’s paper, Rhetoric and Revolution.

Posted in Uncategorized | 6 Comments

To everything, a season

And he . . . Fell into a sadness, then into a fast,

Thence to a watch, thence to a weakness, 

Thence to a lightness, and by this declension

Into the madness wherein now he raves,

And all we mourn for.  – Polonius, Hamlet.

Anecdotally, there seems little doubt that we are seeing a major rise in mental health issues amongst adolescents. Whilst there may be some truth in the suggestion that some of these ‘needs’ are manufactured, there is also some evidence that there has been a sharp rise in both the number and severity of mental health cases. In my own school, the number of severe case has more than doubled this year.

In response to this situation (some would say crisis), we hear again the plaints of those  demanding that schools ‘need to do more’, just as schools have also been asked to ‘do more’ with sex and relationships education, drug education, healthy eating, radicalisation, British values, character, financial education, community cohesion, equality, diversity, and citizenship – to name but a few of the many additional tasks that schools are expected to take on – in addition to the more ‘core business’ of raising academic standards, preparing students for an endless, churning mill-race of curriculum and assessment changes, narrowing the achievement  gap, meeting floor standards, succeeding in Ofsted inspections and competing in league tables.

Quite understandably, it is a cause of some disquiet to teachers that serious, potentially life-threatening, and time-consuming issues should be tossed into that mill-race. Apart from the risks to the students, attempting to intervene in mental health issues could severely compromise teachers’ abilities to effectively carry out their ‘core’ functions of planning, teaching and assessment. And it is not as if teachers’ own mental health is exactly in tip-top shape, particularly under the often soul-destroying pressures pervasive in the current incarnation of the education system. Teachers are being responsible when they resist calls to become counsellors and amateur psychologists.

And yet.

It is not so simple in the real world, where we are not able to neatly segment our lives, and our students’ lives. We should maintain professional boundaries, but this does not mean that we can ignore the pain in front of us. Most of us would do our best to support a colleague who was experiencing mental health problems. Why would we ignore our students? The fact is, we can’t.  Mental health is a specifically named set of needs in SEND legislation, and students are entitled to support where such needs affect their ability to access the curriculum. There is also a swathe of guidance from the DfE on safeguarding, so any teacher who thinks that they can ignore mental health issues in their classroom is likely to be on very shaky ground.

So – what do you do when confronted with a student with mental health needs?

The answer, of course, depends on the needs, and the situation. Is there an imminent risk of harm, or has the student already been harmed? How are other students being affected? To what extent is there support available from home, and what support services are in the school? Being able to ask ourselves questions like these, and knowing who to go to for advice and referrals, are the equivalent of a mental health first aid kit.

Schools have policies on safeguarding, child protection, bullying, and wellbeing. Linked to these policies should be clear procedures for referring students’ needs to the appropriate service within the school. Ideally schools will have specialist staff who are trained and who are linked with external agencies to ensure that a secure net of support is woven around the student both at school and at home.

So we would hope – but of course, in reality, the agencies are overstretched, in-school provision is patchy at best, and specialist staff are not only hard to come by but funding to employ them is being squeezed. In these circumstances, there are two dangers: that we attempt to do more than we should in response to the urgency of the need, or that we attempt too little because we are overwhelmed by the scale of the problem. In the former scenario teachers can get in over their heads very quickly, sometimes with painful results. In the latter, apathy allows disaster to engulf young lives when the right conversation could have led to better support.

So how should schools deal with mental health?

First, schools need pro-active policies in place to ensure that mental health needs are understood as normal human experiences, and that it is healthy to seek support when you need it, whether informally through friends and family, or formally though pastoral services. Secondly, all staff need training in the basic ‘first aid’ procedures for mental health: assessing the immediacy of risk, notifying appropriate personnel, and managing incidents to minimise harm and disruption. Thirdly, schools need to have sound provision in  place for students who need specialist support, and a clear procedure for inter-agency working to ensure round-the-clock care where it is needed.

The last item is the one that costs the most money. And even if we find the money, who will deliver the service? This paper by the DfE sets out principles for setting up a service in a school that is embedded, but independent (see p.21). We use a similar model in my current school and I’m very grateful we have it.  I wouldn’t dream of suggesting that we can’t improve, but even with the many compromises required to keep a small organisation working within a much larger one, the service does an excellent job at holding students who would otherwise not be able to access the opportunities that school has to offer. In other words, we keep them coming back to school. They haven’t given up.

Should teachers attempt to take on the roles of counsellors and therapists? No. But nor can we ignore the needs in front of us. As responsible adults, we should ensure that systems, procedures and resources are in place to protect the most vulnerable. This is good safeguarding practice, and not optional. While I understand the desire for teachers to maintain their professional boundaries, and work within their expertise, school leaders have a responsibility to ensure that provision is in place and that everyone knows the process to follow when we become aware of student needs.

Slogans and dismissals won’t do. In the world of broken, mixed-up people we work with (people like ourselves), effectively responding to mental health needs can literally be a matter of life and death.

 

 

Posted in Uncategorized | 3 Comments

Of one who went before

“He was a man, take him all in all. I shall not see his like again.” – Hamlet

Mr Bremner was unusual for a teacher in a rural New Zealand school in the 1960s. He drove a French car, a ‘Pew-joe’, and regularly travelled to Europe with his wife in the summer holidays. He was locally born, and evidently came from a strict background. He told us that his mother told him off for using the word ‘darn’ even though he was only asking her to repair his socks. But I felt safe around him, even while the rest of my life was subject to unpredictability. He had a lean, suntanned face, and the kind of authoritative stillness that no one would dream of challenging, inside or outside the classroom.

At break times, Mr Bremner would walk around the playground with various children accompanying, holding or clutching him. In the classroom, there was constant variety. We made our own drums, so we could learn about percussion. We learned about Captain Cook’s journeys and his skill in mapping, to coincide with the bicentenary of his first visit. We listened in awe to the school intercom as the first man stepped on to the surface of the moon. We learned about the French-British wars in Canada, and drew pictures of famous battles in the New Zealand wars.

Mr Bremner had a good singing voice and he taught us folk songs and traditional hymns. He would stop and correct us if we were drifting from the melody, as children will, and insisted on faithful reproduction. Once we were sure of ourselves we would join in with gusto. He had a Swiss army knife which he used to divide up any food we might wish to consume in his classroom: the rule, he said, was that it was all right to eat, but everyone had to have some. He introduced us to Swiss cheese, and olives, in a setting where a hamburger was a culinary novelty.

I pitched up in Mr Bremner’s class because I was doing too well in the class below. The solution to this problem was to put me into a situation where I was more ‘challenged’. As a nervous eight-year-old, I joined a class of older children and fairly quickly felt out of my depth. Mr Bremner ran a much more orderly classroom than I was used to, and he was, if my memory is accurate, my first male teacher. Mr Bremner was ramrod straight, alert and not at all motherly. In his class I felt all over the pain of beginning again. He had a different set of standards.

There was a hinged blackboard which he opened out on that first morning to reveal a 12 x 12 grid which had been etched into its surface. Inside each square was a neatly chalked number between 1 and 12. Mr Bremner would choose a student. He would point to a row or column, and give them a number by which each item in that row or column had to be multiplied. I was alarmed; I had not experienced this kind of public assessment before. But then he held out a stopwatch. And timed us.

I don’t really remember how long my first attempt took, but I think I had to multiply by six and I think it took 36 seconds, by far the slowest in the class. It may have been longer and I have simply repressed the details. I do remember the sense of humiliation and embarrassment.

Lest, dear reader, you imagine that this is a tale of how authoritarian practices shamed me and destroyed my self-esteem, let me spare you misplaced indignation. As a result of this experience, I went home and practised. I practised a lot. I became much faster. I practised until I was the fastest in the class, well, at least first equal – Susannah and I both managed correct runs at nine seconds each, but much to my chagrin, I never managed to beat her. When I left high school years later, mathematics, though not my favourite, was my highest scoring subject, thanks to Mr Bremner.

I hated swimming, not because I don’t like swimming but because I just couldn’t seem to get the hang of breathing only when my mouth was out of the water. As soon as my face went in, a kind of panic came over me. It didn’t help that I was as skinny as a beanpole and would be shivering violently if it was a cool day. But Mr Bremner insisted that I get at least one sticker on my swimming certificate, and made me keep going. So I gasped and splashed my way around the pool, and won my sticker. Which was a lot more than any other teacher got out of me.

In a world before calculators, computers, the internet, Google and the horrors of twenty-first-century non-skills, Mr Bremner knew that we needed our times tables. He knew this not because he was a cruel authoritarian, a despot, or some kind of child-hater. He knew it because he was a well-informed, well-travelled man who cared about his students and was an expert in how to teach them well. No one ever taught me with quite the same impact on my confidence, and that came not from being told I was wonderful but being told to do it again, and better this time. He was kind, but he wasn’t there to let me fail.

Mr Bremner took us on nature walks which still live in my memory in the same way that Wordsworth felt about his daffodils – I return again in my mind to see him point out a lark over the fields, the birds’ nests, to roll back a log to reveal creatures from another world, all of whom he knew by name. I remember the hot sun beating down on the dusty gravel road, vanishing as it ran towards the blue-hazed hills on a December afternoon. I remember the freedom, and a deep satisfaction.

When the weather wasn’t so good, we would sit in the classroom, half-dreaming, as he read us another chapter of the Adventures of Grump, an old tramp true to his name who was always falling into scrapes and escaping again by quick thinking and not a little luck. Mr Bremner didn’t just read the story: he was Grump, in all his moods and movements. He was unafraid of the dramatic pause, for none of us would dare to disturb his storytelling. For many of us, this was restful; for me, it was sustenance. I grew inside.

The next year I was still in Mr Bremner’s class, and had found my feet. But my parents lost their business and left town abruptly. I don’t remember saying goodbye, and on reflection I don’t think I could have borne it.

But I have carried those days through the rest of my life, and draw upon them still when I think of beauty, of peace, of safety. They are part of why I teach: because I believe that education has that power, to gift us parts of a life we would otherwise never have known. And because, in my heart, I hope that I can be a little like Mr Bremner.

Posted in Uncategorized | 2 Comments

Mastery and all that

Yea, from the table of my memory I’ll wipe away all trivial fond records,

All saws of books, all forms, all pressures past,

That youth and observation copied there – Hamlet

Mastery is once again being debated. This is a good thing, since it drives us to ask questions about what mastery is, why we need it and what happens when we don’t have it. Our view of mastery is intimately connected with our view of knowledge and of learning.

One of the criticisms recently made, for example, is that mastery means different things to different people and that this will lead to shaky implementations. The problem with this criticism is that it does not actually define mastery. What it does highlight is that education is still immature as a discipline because we lack a shared technical language. Our language is politicised rather than technical. Other terms that have different meanings to different camps are fluency, practice, mindset, and character. Ethical and political concerns are inevitable and should be discussed freely. Our failure is that we have not succeeded in any kind of scientific consensus on even the most fundamental learning processes. And this is not because the science does not exist; it is because the science is not politically acceptable.

Take the term ‘mastery’. One of the comments made on David Didau’s blog (in response  to Jo Facer’s post on a mastery curriculum) is that ‘deeper learning results from forgetting and re-learning’. Is this mastery?  Such a definition of ‘deep learning’ implies that a good education involves forgetting everything that has been learned, and then re-learning it. It seems to me that it is difficult to argue that anything has been learned if it cannot be remembered. Secondly, the colossal waste of time and resources implicit in such an approach would cost billions with limited impact on the population (which suggests that such an approach may well be at the heart of UK education policy).  Clearly those who argue for forgetting and re-learning have an entirely different view of mastery from those who believe that students should remember what they have learned.

The painful irony is that decades ago, clear definitions with solid supporting evidence were appearing in the research literature. Like Auden’s expensive, delicate ship, the profession sailed calmly on despite something amazing having appeared – in this case, not a boy falling from the sky, but useful research findings. For example, White and Haring as far back as 1980 proposed a concept of mastery recognising different dimensions of learning:

  • Accuracy – being able to do, solve or recall with a high level of accuracy.
  • Fluency – being able to perform a skill with a high degree of accuracy and speed.
  • Retention – being able to recall the skill or knowledge over time
  • Generalisation – being able to adapt the skill or knowledge appropriately to different contexts.

One of the fears of those who question mastery is that it will narrow the curriculum and prevent students from generalising skills. One of the fears for proponents of  mastery is that without accuracy, fluency, and retention, generalisation will be halting, temporary and often ineffectual. Both, of course, have a point. A mastery curriculum needs to address all of these stages of learning – and mastery teaching must ensure that individual students achieve mastery in each dimension. Despite long-standing evidence, consisting of thousands of individual studies which identified precise criteria for fluent mastery, you are unlikely to hear such a view of learning expressed in a teacher training institution. It is too ‘simplistic’, too ‘technical’, too lacking in ‘heart’ for the New Romantics who have colonised education.

An even deeper test of mastery is how previous learning impacts on new learning. When more foundational skills and knowledge have been mastered to a high degree of fluency, they are not only remembered more easily but also combined with other knowledge or skills to enable ‘leaps’ in knowledge. If new learning is a struggle, if key concepts are misremembered or misapplied, if facts are recalled slowly resulting in the overload of working memory, it is clear that what went before was not mastered. But in the pressure of the modern education world, it is scarcely conceivable that a teacher would insist on more practice before moving students to new knowledge. There is a curriculum, there are schemes of work, there is a timetable, there are assessments, and there are observations, work scrutinies and performance reviews. In this environment, those who most need practice most are least likely to get it.

A real mastery curriculum will ensure that learning happens in small steps “such that the learner could induce mastery in a few minutes.” (Engelmann and Colvin, 2006 ).  Constant small successes are the key to mastery – along with generous amounts of carefully sequenced (not random) practice. At first students will not be making the fabled ‘rapid’ progress – but they will know that they have learned and understood and they will begin to feel more successful and confident. Over time, they will combine new learning with previous learning with little effort to recall what was previously studied, and will begin to grow in knowledge, in understanding, and in self-efficacy. They will accelerate through the curriculum. They will become increasingly independent, and own the knowledge that they have acquired for themselves.

One of the problems with mastery is that we have not paid enough attention to the work of those who have gone before. For teachers, the guardians of knowledge, to ignore their predecessors is a tragedy. Another tragedy is that perhaps we cannot understand mastery learning because we have never really experienced it for ourselves. There is no unified body of knowledge for teachers to master; everyone must have their own ‘philosophy of teaching’. Romanticism, individualism, and the platitudes espoused as ‘good practice’ in teacher training institutions have left us slaves to ‘every wind of doctrine’.

Perhaps, as Hamlet suggests, there are more things in heaven and earth than are dreamt of in our ‘philosophies’.

References

Binder, C., Haughton, E., & Bateman, B.  (1992) Fluency: Achieving True Mastery in the Learning Process.

Engelmann, S., & Colvin, G. (2006) Rubric for Identifying Authentic Direct Instruction Programs.

Johnson, K., and Layng, T. V. J. (1992) Breaking the Structuralist Barrier: Literacy and Numeracy Through Fluency.

White O., & Haring, N. G. (1980) Teaching Exceptional Children.

Posted in Uncategorized | 4 Comments

Dangerous Conjectures

“’T were good she were spoken with; for she may strew dangerous conjectures in ill-breeding minds.” – Hamlet

I raised an eyebrow when I saw the tweets about the traditional / progressive debate being ‘boring’. A little further investigation revealed that the allegation was not so much about the level of excitement that the debate generated but rather the idea that the debate was not worth having, stemming from a post on the teachertoolkit.me blog.

I would normally ignore such comments as uninformed. In this instance however, because of the writer’s own claims to be widely followed, I am inclined to the view that these ramblings might have detrimental consequences for the profession and need to be challenged.

The blogger claims that:

“… this debate is fast-becoming the most boring debate of 2014 and (also) 2015! Can we face yet another year? Are we still in a position where we need to argue that one is better than the other?

In reality, teachers at the chalkface actually don’t care what it’s called, they just get on with teaching, using whatever methods suit them and their students. And because of their workload, most have little time to be concerned.

It appears to me that various camps who blog and tweet; those who publish various articles in defence/support of an ideology they believe in, believe one methodology works best. It is my belief that one cannot exist without the other, and one is no more important than the other.” [sic]

There are a number of problems here that should give the reader pause for thought.

  1. ‘Progressive’ and ‘traditional’ are terms that have become shorthand for two different approaches to teaching. They are philosophies of teaching, not ‘methods’. This is an important distinction.
  2. The idea that teachers ‘don’t care’ about ‘what it’s called’ [sic], especially as they are ‘too busy’ does not bode well for their ability to choose wisely ‘whatever methods suit them and their students’.
  3. Such a sentiment is hardly a ringing endorsement of the intellectual quality of the profession – rather, it is trotting out the old myth that as a ‘professional’, I get to decide what I do and how I do it. I have yet to see such a definition of professionalism applied to civil engineering, aviation, or law, for example. It did used to be applied in medicine – but as Greg Ashman recently pointed out, and Douglas Carnine before him, that didn’t work out very well. I doubt many of us would commit our children to the care of practitioners who ‘use whatever methods suit them’.
  4. There is a bold and sweeping statement here that might well be worthy of a book: the idea that the two sides of the debate are equally important, and dependent for their existence on one another. Unfortunately the writer does not elaborate, so we have no sense of his reasoning. Philosophically, it is an intriguing if unlikely proposition.

Why is the distinction between methods and philosophies important? Because methods can be independent of aims. Philosophies of education are fundamentally concerned with the purpose of education. Methods are just things we do in the classroom as steps on the way. The debate is about where we are going.

The essentially romantic progressive philosophy aims to improve society by creating individuals who are caring, co-operative, empathetic, concerned with justice, open to new ideas, and tolerant. It seeks to do so by promoting autonomy, authenticity, co-operation and diversity in the classroom. Knowledge is secondary to attitudes – indeed, in recent years knowledge has become irrelevant to some educators. In this philosophy, society will be improved by making the classroom a microcosm of what we would like the world outside to become.

What is generally labeled traditional education in the current debate is not, fortunately, very traditional. If we look back at our educational past, there are practices that very few of us would want to see return: violent punishment, ignoring safeguarding, tolerance of absenteeism, poor co-operation with external agencies. But there are also traditions worth conserving, because they will benefit our students: community, discipline, work, and knowledge. In this philosophy, while attitudes are important, they are secondary to knowledge. Knowledge is prized because it not only carries forward the accumulated learning of previous generations, but because it also equips students to succeed in the world as it exists beyond the classroom.

The key difference in the two approaches is the attitude to knowledge. To argue that teachers do not need to have a view on knowledge is like saying that doctors do not need to have a view on health. Such an argument is the result of reducing two profoundly different philosophies of education to mere ‘methods’ from which teachers may choose at will. On the contrary, the debate is real, the debaters passionate and sincere, the stakes as high as they could be: the future lives of our students and the ability, or inability, of our institutions to achieve our ambitions for them.

There is, I believe, something profoundly anti-intellectual in turning one’s back on a debate of such importance. To encourage others to do so is at best ignorant, at worst mischievous. Whether we like it or not, if we want to be called professionals, we have to have an intellectual grasp of the issues of our profession.

As C. S. Lewis put it, “Good philosophy must exist, if for no other reason, because bad philosophy needs to be answered.”

Or no philosophy at all.

Posted in Education | Tagged , , , | 28 Comments

Not all who wander are lost

“O woe is me, to have seen what I have seen, to see what I see!” – Ophelia, Hamlet

The journey home from Research Ed at Swindon in the November twilight slipped into night, the tail lights of the traffic ahead gleamed brighter in the gathering gloom, and brilliant flashes of colour erupted and fell again into the shadows. Fireworks, the heralds of winter darkness, announced their fleeting presence.

Another kind of darkness had troubled the edges of my mind throughout the day. At the conference, Professor Ray Land had spoken on the subject of threshold concepts: those points in our experience where we grasp a new concept that changes the way we see the world, and ourselves. The characteristics of such learning experiences, he explained, were uncertainty, anxiety, and a sense of revelation as a new perspective was discovered. They are often accompanied by a sense of loss, for our old perspectives, and our old views of ourselves. This is why such knowledge is sometimes described as ‘troublesome’. Land did not offer any evidence for his assertions, unless one counts quotations from peers; but the resonance with experience was apparent.

Examples of such concepts included gravity, the Copernican view of the solar system, evolution, and the banishment from the Garden of Eden. The serpent, in Land’s view, offers the opportunity to enter a new and exciting world which is much more interesting than the tame and confined experience of the Garden. It was this example that set me thinking on a darker road. For there are experiences that lead us to a knowledge we might wish we never had, or which distort our view of the world, and ourselves. They may qualify as thresholds, but new experiences are not good simply because they are new.

I think of the child who has suffered sexual abuse, who cannot perceive human intimacy without perceiving threat; or the child whose abuse though neglect has been so casual, so persistent, so normal, that a lifetime may not erase the message that he or she is of no value. I think of the woman who has been tortured, the humiliation and bewilderment of a marriage betrayed, of a parent’s trauma when a child falls into addiction and self-destruction. These indeed are liminal experiences, thresholds we may cross and from which there is no going back. Even returning, we will never be the same. They are life-changing, but they are not liberation. They are not ‘more interesting’. They are disasters that we would not wish on anyone.

I make the point because Land didn’t. The implication of his speech was that anxiety, uncertainty and liminality are inherently good, because they lead to change, whilst rest, peace and certainty are reductions of the human condition. Perhaps he didn’t mean this; perhaps he simply didn’t think it was worth clarifying because it was so obvious. But the subversion of the Eden story to suggest that seduction leads to a better outcome than fealty struck me as a perverse kind of wishfulness. The story of Eden is the coming of darkness into the human spirit. That such darkness exists is evident every day in the sorrows of those I encounter. Not all thresholds are to be crossed, nor should we wish them to be. Not even all intellectual endeavours are worthy of pursuit.

The road passed out from the temporary comfort of artificial light into the unlit countryside. Black hills appeared against the eerie, transient glow of the skyrockets. I was reminded then that the longest nights of all are where our predecessors chose to celebrate another story – the story of another crossing, of another kind of threshold: where the light broke into darkness, and the darkness could not overcome it. It may be true that we cross some thresholds to our sorrow, and can never be the same. Yet there is hope too, not that the serpent promises of life will prove true, but that grace can do what regret cannot.

It is, as Hamlet might have put it, a consummation devoutly to be wished.

Posted in Uncategorized | 6 Comments

Vested Interests?

“Madam, how like you this play?”

“The lady doth protest too much, methinks.” – Hamlet

A popular tactic in arguments is to attack the person bearing a displeasing message. This ad hominem response works by slapping emotive labels on opponents in an attempt to stop other people thinking about what they are saying. At times I have seen this tendency rise to feverish heights in politics, and in education. It’s a tendency that currently seems to be growing in the Labour Party.

Even the academic world is fraught with ambition, rivalry and conceit. The same threat of vilification and ostracism can just as easily apply to researchers as it does to politicians, or to teachers who happen to espouse a view that is is contrary to the progressive zeitgeist. For instance, in a speech to the Baltimore Curriculum Project in 2012, Shepard Barbash discussed the rejection of Direct Instruction by the educational establishment in the US, and related this anecdote. He had been corresponding with an academic  researcher who had praised his book, ‘Clear Teaching‘, on Direct Instruction, but gave this response when asked for a public endorsement:

Dear Shep,

I enjoy discussing these issues — that’s what I do. But there is no chance that I will end up writing a note to Amazon that endorses your book or Direct Instruction. 

I worry that anything I write in praise of your book could be seen as an endorsement of DI and that implication — however indirect –might diminish the impact of my research.1

I have certainly seen some questionable practices by academics at first and second hand. Those experiences, and engagement with academics  less inclined to toe the party line than Barbash’s example, have developed in me a healthy element of skepticism when it comes to educational research.

I have critiqued two studies released by the Educational Endowment Foundation on this blog: Switch-On Reading and Accelerated Reader. Both report mediocre effect sizes as ‘positive effects’. In reality, the most favourable interpretations of the studies’ data suggested that they led to three months’ gain over three months in the case of Switch-On Reading, and three months’ gain over four and a half months for Accelerated Reader.

Why is this inadequate? Because, for students who are reading significantly behind, such progress is ineffectual. Let’s take a student who is 36 months behind. At least 7% of secondary students fall into this category, so in a community with low socio-economic resources, it’s likely that in a school of 1000 pupils there would be at least 70 in this situation.2 And let us say that we ensure that they receive one of these interventions.

A student following Switch-On Reading for three months makes a gain of an additional three months. So now they are just 33 months behind. The deficit has been closed by one twelfth. At this rate, if they do Switch-On Reading every term, it will only take another 11 terms for them to catch up, or three and a half years. So this intervention would, if it had sufficient materials and teaching programmes, enable Year 7 students reading 36 months behind to catch up late in Year 10. Of course, it doesn’t have these resources – it is a ten week programme. If it did have the wherewithal, it would cost not £672 per pupil, but 12 x £672, or £8064. Per pupil. Even that would be something. But in fact, the intervention offers three months’ gains in ten weeks, and that’s it. As for Accelerated Reader, students would gain about six months per year, meaning it would take six years to catch up. A ten or twenty week intervention that moves struggling readers just three months further on is the educational equivalent of giving a course of aspirin to a TB patient.

Yet according to the EEF studies, both programmes have ‘positive benefits’ with medium security for the reliability of their findings. The fact that these findings are absolutely no use in the real world, for students with real learning needs, seems to have escaped the researchers.

Recently the lead researcher of the Switch-On study, Professor Stephen Gorard, tweeted his objections to this post (which was about Reading Recovery research). There were three:

  1. I did not make sufficient distinctions between the Switch-On study’s use of effect size and Hattie’s.
  2. He believed I had an agenda.
  3. My anonymity.

I had expected to be chastised on some technical grounds at some point (few issues are settled in the world of educational statistics). Professor Gorard wrote:

Screen Shot 2015-10-17 at 15.39.28

Screen Shot 2015-10-17 at 15.39.46

He is correct that Hattie uses different types of effect size calculations in his meta-analysis. How much this matters is another issue. Regardless, the argument still stands, even if we remove all reference to Hattie and the ‘hinge point’. By the Switch-On researchers’ own reckoning, students gained three months – so the effect of the intervention was insufficient to enable a student to catch up if they were reading just four months behind, let alone four years. Would schools invest if they understood this?

This yawning gap between educational practice focused on students’ needs on the one hand, and academia focused more on the security of a research design than the utility of findings on the other, is a major concern. Not only does it call into question the government funding provided to the EEF, but it ultimately undermines confidence in the value of educational research.

In this context, Professor Gorard’s complaints about anonymity are perhaps best seen as an attempt to dismiss the entire discussion rather than to go into awkward territory. Instead, he pursued the issue of identity in order to prove an unspecified ‘agenda’.

Screen Shot 2015-10-17 at 15.48.41

It was as if he had read my recent post on reasons to be anonymous on the internet, and was at pains to prove me right. When John Walker innocently tweeted a comment, this was the accusation:

Screen Shot 2015-10-17 at 15.57.25

Either an argument stands up or it doesn’t. Whether the author is a millionaire, a teacher or an academic, a supporter of free speech, rail nationalisation or fruitarianism, the argument and the evidence are separate issues from their identity. Of course the writer and their writing are related: but if the only way you can oppose an argument is to decry the author’s motivations, then you have lost the argument.

Crying ‘vested interests’ can also be a two-edged sword. Does receiving income from your educational work mean that you are precluded from expressing views on education? If so, no one employed as a teacher has a right to express a view on education. Indeed, no MP would have a right to express views on politics. Such a view is clearly ludicrous, but this does not prevent its frequent appearance in the modified form: ‘if you receive private income rather than from the state, you have a vested interest.’ Piffle. The money smells the same regardless of how it was (legitimately) earned. There is no moral superiority attached to being funded by the taxpayer. In this sense, civil servants, union members and boards of directors all have vested interests. So, for that matter, do academics who rely on government-funded bodies for research grants.

Professor Gorard, for example, led the teams on the Switch-On Reading study, the Accelerated Reader study, and the P4C study (which has been criticised by others for its statistical weakness and poor interpretation). Defending that funding stream could be construed as a vested interest. Does this mean Professor Gorard has no right to comment? Of course it doesn’t. His views should stand or fall on the evidence and the logic. And it is reasonable that he should accord the same right to others, focusing on the debate rather than innuendo.

‘Vested interest’ is rarely an argument. It is a principle used to decide who should be part of a decision-making process, for example when allocating funds. It can’t be used to prove that something is or is not true.

Back to the nub of the issue:

Screen Shot 2015-10-17 at 16.00.09

Apparently looking for two months’ additional progress per month of intervention is ‘absurd’. When researchers focus their expectations of what is ‘normal’ on statistical experience, and write about findings in this light, they do students a grave disservice. No amount of crying ‘vested interests’ makes a three-month gain ‘effective’ for struggling readers. Schools need to look beyond the bland headlines when researching interventions. The Education Endowment Foundation needs to think carefully about whether or not their current approach is providing the service to schools for which they are funded.

Footnotes:

  1. Barbash, S. (2012). Keynote address to the Baltimore Curriculum Project, Baltimore USA.
  1. Source: Department for Education, National Key Stage 2 Results, 2012-2015. Based on percentage of students at or below Level 2 in English at the end of Year 6.
Posted in Uncategorized | Tagged , , , , , , , , , , , | 6 Comments

Five reasons to be anonymous on the internet

‘Ay, sir, to be honest as this world goes, is to be one man picked out of ten thousand.’ – Hamlet

  1. People whose staple response is ad hominem get really frustrated. Which is why they attack anonymity – they are trying to find something to work with.
  2. If helps conversations stick to the issues.
  3. It saves other people who may be connected to you from trouble or embarrassment if you make a blunder.
  4. It helps to protect you from those who are in a position to harm you if they feel threatened by your views (but of course it cannot guarantee this).
  5. It means you can never be accused of self-promotion.
Posted in Uncategorized | 2 Comments

A question of rigour

Sure, he that made us with such large discourse,

Looking before and after, gave us not

That capability and godlike reason

To fust in us unus’d.” – Hamlet

There is a scene that recurs in action movies, usually towards the climax, where a large vehicle (train, bus, lorry or aeroplane for example) careers out control, smashing into one barrier after another, slowly disintegrating even as it demolishes the structures we expect to contain it. While this can be cathartic in an action movie, it is less uplifting when one encounters it in a report that cost £147,000 and that was supposed to tell us something useful.

The EEF report on Accelerated Reader published in February of this year at first seems neither problematic nor particularly inspiring. A randomized control was run across four secondary schools involving 349 pupils. The first sign of a possible derailment is when we learn that the overall effect size was 0.24, equivalent to three months’ additional progress during the twenty weeks, or half a school year, over which the trial was run. This is a mediocre effect size; if we compare it to the findings of John Hattie’s massive meta-analysis, it does not even reach the ‘hinge’ 0.4 which is the average impact of educational interventions. In other words, it does not reach Hattie’s criterion for ‘worth investing in’.

This does not, however, prevent the EEF from declaring that AR is “an effective intervention”. This is consistent with other EEF studies which also scored around the 0.25 mark. One possible justification for assessing such a grading as effective is that the cost – cited at £9 per pupil per year – means that the intervention is cheap to implement even though it only achieves what the EEF admits is a “modest” effect.

To evaluate this, we would need to consider not only the £9 cost of the most basic licence type for each student, but also the other resources which the EEF states are required for implementation: computers or tablets, internet, and access to a wide variety of books graded according the Renaissance Learning ‘readability formula’. Schools in the trial bought books in order to set up Accelerated Reader, and employed staff to band books and to inventory them. This resourcing was paid for by schools using funding from the trial. However, although the EEF must have precise figures on this expenditure, no detail of these costs is made in the report.

Screen Shot 2015-08-26 at 13.07.06

We know that the overall funding for the trial was £147,000. In the absence of a breakdown of the project’s funding, one can only guess at the true cost of implementation. Assuming that the training and evaluation cost £70,000, and that licences cost £9 per head for a total of £3,140, we can estimate that the remainder of funding went to the resourcing described above – about £74,000. This works out at £206 per pupil – a discrepancy of about 2,300%. So much for rigour.

There are, of course, other costs that researchers and schools often fail to include. The opportunity cost of taking students away from other lessons seems never to be considered, perhaps because it is difficult to estimate. In this case, students spent 180- 200 minutes per week reading books with Accelerated Reader  – in one school after lessons, in others during the school day. In  some cases at least three lessons were being missed. Two questions arise:

  1. What was the impact on their learning in the subjects that they missed? Is it justified by the progress they made?
  2. Could this time have been used with greater impact on their reading through another intervention?

The first question is simply ignored, though the three months’ additional progress suggests they may as well have stayed in class. The second could not be answered because the design of the study simply compares the treatment group with a ‘business as usual’ control group. Such comparisons do not yield enough information to move educational practice forward. Why, when we have the money, the expertise and the opportunity, do we not compare interventions to see which is more effective? Rather, we have a trickle of data that has almost no impact but instead generates a slowly rising tide of antipathy towards research.

The study then lurches into collision with the most obvious problem we could have expected: what to do with children who can’t read independently? The teachers did what any self-respecting teacher would do: they helped the individual students with their reading. We are not told how many children were helped in this way, or how much. So we have no idea how much this one-to-one support, which was in addition to the AR intervention, contributed to the admittedly modest gains described. Having identified this problem, the report carries on without suggesting any solutions.

The last, unexpected part of the disintegration comes when we look at how the effect size is generated. The post-test is the NGRT Digital, a common, well-researched and statistically strong test. No problem there. What is the pre-test? Answer: none. Students were allocated to the intervention based on their Key Stage 2 English points. This measure alone has been used to establish comparability between the randomized groups. The effect size is simply the difference between the mean reading post-test score of the treatment and control groups.

The writers appear to argue that, because the randomization process made for comparability between the groups, no pre-test is required.1 But one of the long-running complaints that secondary teachers have about the Key Stage 2 scores is that they often seem to bear little or no relation to the children in front of us. Given the potentially distorting effects of pressures on schools, there is good reason to be wary of relying on these scores for anything – quite apart from the variations in teacher interpretation that can occur between individuals and schools. In other words, the grouping criterion is not reliable.

The report stresses that the NGRT was simple to administer as a post-test. Then why not use it as a pre-test to ensure comparability between students? One of the problems this would have identified was that of students who were not able to read independently, thereby eliminating the major problem in interpretation of the results (referred to above). It cannot, given the size of the budget, have been a cost-saving measure – or if so, it suggests very poor judgement.

In short, the EEF’s report has provided little evidence to help schools decide whether to use Accelerated Reader. AR is, as far as we can tell, better than nothing, but may cost a lot more than it first appears. Far from revolutionizing education through evidence, this report and others like it appear to be creating opacity, not clarity. A cynic would say that four schools had found a way to fund extra books and computer tablets, a view that appears to be supported by the writers’ complaint of a lack of interest by any of the schools in analyzing and reporting on their own results. A more optimistic view is stated in the report, which opines that AR is a well-researched intervention and that this trial provides the largest and most reliable study so far.

I do not agree that AR is well-researched; it is certainly not well-evidenced. Nothing here convinces me that it is better than having a well-stocked library, with clear goals for each student, and the time to get them reading.

Notes:

  1. If they are arguing something different, I haven’t understood their point and am happy to be corrected.
Posted in Uncategorized | 3 Comments