'How do you measure your impact?’
This is often the first question asked by somebody with a private sector background when encountering a public or voluntary sector organisation. It is a reasonable question to ask of any organisation. But it is not always reasonable to expect a clear answer. The trouble is that private sector organisations, thanks to the luxurious simplicity of measuring performance in profit (and loss), are used to having clear answers to this question. As a result it is easy to feel bafflement and frustration when such clarity fails to materialise elsewhere.
To get a better perspective on the challenges this question poses, private sector organisations could ask themselves how they would measure their own impact beyond profit and loss. Take Microsoft for instance. Of course we know how profitable it is by country, by product and probably minute by minute too. But does Microsoft have a robust way of measuring the impact that Powerpoint has had on the workplace? Does Powerpoint produce better decisions? Does it increase productivity? Does it reduce costs? Does it increase sales for the user? Does it improve morale?
There could be ways of contriving measures for any one of these using some form of randomised control test but these would be expensive, complicated and almost certainly inconclusive. How could you isolate the impact that Powerpoint has amongst all the other variables of a workplace? At some point the effort put in to get these numbers becomes absurd. Instead, I suspect that Microsoft tests the quality of Powerpoint and its other products by using a combination of detailed user testing and the judgement of its own experts.
The same goes for many services provided by the public and voluntary sector. Assessing the impact that any specific intervention has on the future of a child, or the quality of life for an elderly person is an inexact science. There is nothing wrong with trying to find good, quantitative measures, but only up to a point. The best way to assess and manage complex services is often through seeking the opinions of those who use them and trusting the opinion of the experienced people who are providing them. This is not a soft touch - in good organisations the user testing and the recruitment of experts are pursued rigorously – but it is usually a better use of resources than investing time and money creating pseudo-scientific measures to satisfy unhelpful expectations.
This is often the first question asked by somebody with a private sector background when encountering a public or voluntary sector organisation. It is a reasonable question to ask of any organisation. But it is not always reasonable to expect a clear answer. The trouble is that private sector organisations, thanks to the luxurious simplicity of measuring performance in profit (and loss), are used to having clear answers to this question. As a result it is easy to feel bafflement and frustration when such clarity fails to materialise elsewhere.
To get a better perspective on the challenges this question poses, private sector organisations could ask themselves how they would measure their own impact beyond profit and loss. Take Microsoft for instance. Of course we know how profitable it is by country, by product and probably minute by minute too. But does Microsoft have a robust way of measuring the impact that Powerpoint has had on the workplace? Does Powerpoint produce better decisions? Does it increase productivity? Does it reduce costs? Does it increase sales for the user? Does it improve morale?
There could be ways of contriving measures for any one of these using some form of randomised control test but these would be expensive, complicated and almost certainly inconclusive. How could you isolate the impact that Powerpoint has amongst all the other variables of a workplace? At some point the effort put in to get these numbers becomes absurd. Instead, I suspect that Microsoft tests the quality of Powerpoint and its other products by using a combination of detailed user testing and the judgement of its own experts.
The same goes for many services provided by the public and voluntary sector. Assessing the impact that any specific intervention has on the future of a child, or the quality of life for an elderly person is an inexact science. There is nothing wrong with trying to find good, quantitative measures, but only up to a point. The best way to assess and manage complex services is often through seeking the opinions of those who use them and trusting the opinion of the experienced people who are providing them. This is not a soft touch - in good organisations the user testing and the recruitment of experts are pursued rigorously – but it is usually a better use of resources than investing time and money creating pseudo-scientific measures to satisfy unhelpful expectations.
