In praise of complex, ‘messy’ practitioner research

In this blog – the first of a series of three – Dr Lynne Taylerson, the author of a recent thematic review into the outcomes of the Outstanding Teaching Learning and Assessment (OTLA) 6 projects, reflects on the value of ‘messy’ research into complex, contextualised FE practice. Dr Taylerson’s review looked at the outcomes of OTLA 6 projects, which spanned 2019 and 2020, focusing on teaching and learning in English.

There is recognised value in FE practitioners developing a knowledge of evidence-based research. The ETF Professional Standards (2014) define common expectations for the sector and reinforce the importance of updating knowledge of educational research and the value of applying theoretical understanding to develop evidence-based practice.

With these principles in mind, ETF has funded Outstanding Teaching Learning and Assessment (OTLA) action research projects, bringing together Further Education (FE) and Skills practitioners in collaborative work. The projects encourage teachers to explore new strategies for improving the quality of teaching and learning, empowering teams to exercise agency in their professional development. OTLA phase six (2019–2020) focused on teaching and learning in English, involving 45 providers and over 400 practitioners in action research.

While evidence-informed research gives us key principles to guide and frame our practice, each FE group, each learner, is unique. Practitioners operate in highly specialist contexts across a diverse sector. Kemmis (2017, in Powell, 2018) argues that evidence-based practice can involve practitioners using best practice takeaways from past research by other teachers, who will often be working in unrelated contexts. Use of generalised principles risks over-simplifying complex learning processes.

The practitioner action research undertaken by OTLA teams, by contrast, is current, highly contextualised and site-specific. OTLA teams report a ‘two-way flow’ of collaboration allowing English teachers to better contextualise sessions to suit a vocational specialism and vocational teams being better able to embed engaging English activities into their practice. Practitioners also report increased confidence in the use of evidence-informed approaches.

For a teacher working in a scenario similar to the OTLA researchers, reading the outcomes from the OTLA reports also has clear value. Fellow teachers can pick up strategies to experiment with and gain the benefits of ‘lessons learned’ during the research.

The OTLA work brings the complex factors at play in the messy reality of unique teaching spaces into sharper focus. For OTLA researchers, this messy practitioner research can involve compromise in terms of the interventions possible, the time which can be devoted and the reconciliation of conflicting findings. Though this type of practitioner research can certainly be messy, OTLA 6 projects provided evidence of innovative professional development and improved learner engagement and progress. One of the strengths of these projects is the evidence richness, the thick descriptors (Geertz, 1973) provided by researchers.

There can be disadvantages to these types of descriptors in research reports such as OTLA studies. We need to accept qualitative judgments and interpretations from one researcher, or from a small group, presenting a potential for bias or conflicts in interpretation. That said, thick descriptors provide a powerful way of allowing us to understand meanings or influences underpinning a scenario, giving us the fine-grained details of human experience key to understanding complex situations (Geertz, 1973). Availability of rich detail is also instrumental in allowing a reader to consider how they might apply similar learning theories and strategies to their own scenario.

Gardner, Holmes and Leitch (2008:95) advocate for this type of nuanced, small-scale research, warning against the prevalent “obsession” with large-scale, quantitative data. The search for big data can lead a researcher to neglect “subjective, anecdotal or impressionistic” evidence even though small-scale, subjective data is “a powerful source of evidence” (ibid:96).

So how can we advocate for the rigour and validity of small-scale, time-limited practitioner research in FE? The answer lies in foregrounding the value for teachers in describing and analysing their authentic lived experiences in rich detail.

Participation in action research gives opportunities to draw on multiple evidence sources. Teams called upon the learner voice, tutor and support team testimonials, views of parents, carers, community groups and volunteers. There was inter-team collaboration between vocational tutors and English teachers, learning support colleagues, learning resource centre staff and volunteers. Rich evidence drawn upon included samples of learners’ work and also some quantitative measures such as attendance and participation data. Evidence was gathered on a micro scale; messy, certainly, but working in-depth in this way with individual learners allowed practitioner researchers to become more familiar with learners’ wider lives and more aware of their needs.

Gardner, Holmes and Leitch consider that discrete, diverse pieces of evidence such as this can come together into a “mutually supportive… arguably credible source” of evidence of impact. If multiple, corroborating arguments are threaded together in the reporting of research investigations, they can form a strong cable “whose fibres may be ever so slender, provided they are sufficiently numerous and intimately connected” (2008:97). In the case of the OTLA 6 work, different research teams investigated strategies in the same area, for example development of learners’ oracy skills or how to foster a love of reading for pleasure.

The presence of multiple, consecutive, related studies presents an exciting possibility of a jigsaw of practitioner-generated evidence which, when considered together, enables us to see a wider picture, to draw out key themes and conclusions. Impact can be identified through interpretation of the rich anecdotal data because “where there is smoke, there is (the potential) for fire” (ibid:98).

Importantly, OTLA reports interpret situations as well as simply describing them, allowing practitioners to engage in powerful reflection on the effectiveness of their practice and so to reframe and develop it. Findings are valuable for the teams engaging in the work, of course, but also contribute to a powerful, growing research evidence base for the whole sector.

References:

The Education and Training Foundation (ETF) (2020) Outstanding teaching, learning and assessment: a summary of projects in the OTLA phase 6 (English) programme (The ‘Green Book’). London: ETF.

Gardner, J., Holmes, B. & Leitch, R. (2008) Where there is smoke, there is (the potential for) fire: soft indicators of research and policy impact. Cambridge Journal of Education. 38: 1, 89—104.

Geertz, C. (1973) Thick Description: Toward an Interpretive Theory of Culture, in The Interpretation of Cultures: Selected Essays. New York: Basic Books.

Kemmis, S. (2017) in Powell, D. (2018) Practice architectures and ecologies of practices: new ways of seeing into and making changes to teaching, learning and assessment practices. [online]. Available from: https://www.skillsforlifenetwork.com/attachments/OTLA/A%20new%20way%20of%20seeing%20into%20TLA%201.pdf

 

Share this article: