Dr Greg Hall left the CSA Centre team in September 2018. Greg wrote this blog when he was in our team, we hope you find it interesting.
Dr Gregory Hall, our Deputy Director for Knowledge & Practice Development, outlines some of the challenges involved in evaluating and commissioning effective services and how the CSA Centre is working to improve understanding of ‘what works’.
At the CSA Centre, we’re focused on improving knowledge around child sexual abuse in two main areas. First, by improving knowledge of the scale and nature of abuse, something I mentioned in a previous blog post. Second, we want to help improve understanding of what works to address, prevent and respond to abuse.
The challenge of finding ‘what works’ stuck out as a concern in our recent research with commissioners. Commissioners wanted to support programs which were as effective and accountable as possible, but saw that it was problematic to impose onerous and potentially ill-fitting evaluation and accountability regimes, particularly on small-scale services with limited resources.
Our Evaluation Fund has been working with a number of such services to help them establish more robust frameworks, markers and measures which fit with their needs and capacity. This has included listening to service users about ‘what works’ for them, as well as work clarifying practice frameworks, and more systematically identifying how practices are linked to outcomes. Apart from directly supporting 17 services across England and Wales, the Fund will provide learning in terms of the usefulness of different markers and measures, and learning about the process of improvement – particularly for smaller services with limited resources, and how they might be supported and encouraged by commissioners.
We recognise the importance of building knowledge from the ground up and integrating different approaches to evidence. This means balancing and understanding how different voices contribute to knowledge rather than relying on a ‘ladder of evidence’ which automatically assumes that findings from highly systematic and experiment-oriented models such as randomised controlled trials (RCTs) provide the best insights into effective practice.
Snakes and ladders
Assumptions from such a ladder reflect a particular scientific emphasis which is perhaps most helpful where practice involves clearly defined programmes and consistent use of a detailed model of working.
Challenges emerge however, where services are less able to articulate clear frameworks or where practice may itself be less amenable to being guided by manuals, less controllable, or involve multiple agendas.
In addressing child sexual abuse, professionals work in dynamic and often messy, uncontrolled contexts. Practices such as social work visits to a family home are difficult to constrain into clinical or therapeutic models.
Such practices play out through conversations where emotional labour bumps up against resource constraints, administrative workloads and untold local operational issues. These experiences on the ground are rarely dwelt on in models of practice.
The danger of the ladder approach is the higher you climb in the search for evidence, the further you get from the ground.
‘What works?’ for professionals and the individuals and families they engage with is not always straightforward. How things appear on the ground can be quite different to how things are perceived in models or apparent in reporting and inspection as a recent blog from Cardiff University highlights. The manuals on which RCT findings are developed can be left on the shelf as new challenges emerge and practice assumes its own reality at ‘street level’. Flexibility can itself be an important feature of responding to children and young people.
The ladder hierarchy can also lead to experiences of individuals and families being viewed as low status evidence. In the public domain however, claims to knowledge from less systematic and controlled approaches are increasingly contributing to thinking about social problems such as child sexual abuse. For example, narratives from personal experience and case-informed accounts from individual practitioners have been instrumental in unearthing systemic problems and poor practices around child sexual exploitation. Stories of experience are also a key aspect of recent and current inquiries into child sexual abuse, as evident in the Truth Project set up by the Independent Inquiry into Child Sexual Abuse.
Recognising diverse perspectives
Our approach to building knowledge for practice encourages transparency about the benefits and limitations of diverse forms of evidence. Different methods and forms have different strengths and weaknesses. This is not to say all methods are equal, but being more transparent and critical rather than assigning absolute value (or worthlessness) can help us understand their usefulness for practice. Highly systematic approaches have strong claims, but their limitations can be missed in assumptions about intrinsic value.
The challenge is to draw less systematically developed knowledge into dialogue with more systematic research. The opportunity is that greater dialogue and inclusion will unearth a more rounded picture, hidden from any one perspective.
There has been much good work done in recent years around active engagement to build knowledge. This has invited professionals to use research evidence to inform rather than dictate practice, and seek out a balanced view informed by more than one source of expertise. It has also recognised the benefits of researchers working with practitioners rather than attempting to impose solutions.
At the CSA Centre, we have developed our own focus on bringing practice environments into dialogue with researchers through the creation of a multi-disciplinary team. This team links our research efforts to advisors grounded in practice experience and service improvement.
Practice improvement advisors
Our practice improvement advisors have diverse specialist backgrounds and connections, including with social workers, police, health and paediatric teams, multi-agency partnerships, the education sector and practitioners working with marginalised groups. The CSA Centre also has an advisor with a particular focus on links to Welsh government and developments in Wales.
The advisors’ practice backgrounds and connections present opportunities for the CSA Centre to identify knowledge held in practice environments and explore how we can help practitioners and organisations make it more robust and transferable.
We are equally interested in critically assessing findings from research through a practice lens. We do this through an extended peer review process involving multiple practice experts, as well as through structured practice-research dialogues which test the relevance and robustness of findings, and support more accessible presentations and resources for practitioner education and development.
Our advisors are also leading engagement with the views of children and young people. We are currently listening to children’s perspectives on our research, as well as developing understanding of how young people perceive professional interventions around their risk and safety.
Find out more about the advisors and their areas of focus.
