… or, why it’s not all about the numbers!
Conversation within the CO team often veers towards the latest sporting events: the plight success of Forest Green Rovers, the highs and lows of the recent Rugby World Cup… It’s fair to say that not all members of our team (we’re not going to name names) are equally enthused by people chasing a ball round a field. But this week, England men’s cricket hero – and BBC SPOTY 2019 – Ben Stokes became the latest to catch our attention, with a match-winning performance against South Africa in Cape Town.
Sometimes the people who make the best stories, who create legends and grasp the attention of the country, are not those with the greatest stats, but the ones who seize the moment and pull victory from the unlikeliest situations.
“We live in an age when analysts believe almost everything can be measured. Win-rates, strike-rates, heart-rates, economy-rates… you name it, someone can put a figure on it.
And there’s no doubt such statistics can provide insight and illumination. But not everything that counts can be counted.”
George Dobell – “Love, loyalty, stamina: the secrets of Ben Stokes’ immeasurable brilliance“
Of course numbers are useful. We use them to measure the scale of an issue like digital exclusion, and to evidence the impact we make. But relying solely on the numbers can be very misleading.
What’s the catch?
Targets for digital inclusion work are often based on activity numbers:
- number of sessions provided by a digital champion
- number of session attendees
- number of skills learnt
- number of site page / video views
- number of e-Learning courses completed
But while these measurements tell us something, they are also liable to miss some crucial aspects. They don’t tell us about the effectiveness and durability of the learning that has taken place. Training that doesn’t leave people feeling more skilled, confident and motivated three, six or twelve months down the line may be of little real value.
Our Switch approach builds digital skills provision that is meaningful and that lasts.
Just counting the numbers doesn’t help anyone understand whether 5 sessions with a learner is 5 times better than 1, or why.
Is a structured 30 minute lesson more, or less, effective than 6 x 5 minute interventions? That will depend on who the learner is, how the Digital Champion applies their skills, and on the local context.
We know from our own research into the work of Digital Champions that a lot of what they do isn’t captured by the numbers they report. Sometimes there isn’t time to make a note of all that’s done, and it’s impossible to measure the more intangible benefits of support and connection: people feeling heard and cared about.
Building partnerships
More broadly, if there is little ongoing support in the community to help people keep learning and staying safe and confident online – if there’s no investment in the necessary facilities, volunteers, or organisational transformation – then the work that goes into reaching people may bear little fruit.
When looking for value and success in a digital inclusion project, we believe it’s much more than a numbers game.
We’re always working to improve the reliability and validity of the evidence base for what we do – and we know our approach isn’t perfect – but we’re convinced that lasting impact is about more than (just) the numbers.
It’s not just about the number of organisations involved, but about how well they work together, and how capable, confident and committed staff are. It’s about local knowledge, empathy and understanding. It’s about leadership and action even against the odds. Just ask Ben.
After working in over 50 communities in the UK over 20 years, we’ve seen short-term projects come and go without reaching deeply enough to sustain lasting benefits.
By contrast, we measure success in terms of building a community of providers, supporting area-wide partnerships of organisations with shared goals, who help each other do more. This is what sustainable digital transformation looks like.
