Due to the increasing co-operation of actors in public service systems the evaluation of service outcomes involves a complex set of information sources and processes. Organization-specific evaluation practices yield only a partial picture of the effectiveness of services. I wrote about this earlier in ‘What defines performance of a service system?’
In another study we recognized several shortcomings in performance measurement and evaluation practices and especially in the use of performance information. These shortcomings related to different stages of information management cycle [see Choo, C.W. 2002. Information Management for the Intelligent Organization, Art of Scanning the Environment]
First, the identification of information needs entails familiarization with the context and aims to specify the underlying phenomenon (see figure below). It is also important to carefully consider why we are engaging to measurement exercise in the first place.
Second, after the information need has been identified, the next step is information acquisition. In case of a service system this task is more complex than within an organization. Information acquisition is further hampered by the increasing trend of outsourcing data administration.
Finally, organization and use of performance information closes the cycle and brings in the decision-making part and adaptive behavior. This brings us back to information needs and raises questions like: Who is responsible for ensuring that the system operates efficiently and effectively? Who is responsible for the performance of the service system and what level of performance is required or desired?
We summarized our empirical findings into the key argument of the paper; many challenges in the evaluation of system-level service effectiveness relate to problems in knowledge transfer and a lack of appropriate knowledge management tools and practices.
We suggested that knowledge management would help in changing employees’ attitudes to data collection and increase their understanding and awareness of the important performance drivers. Moreover, management and evaluation perspective needs to be taken into account when developing information structures and systems. The analysis and development of information processes is an effective way to streamline administrative processes and improve the performance of the public service system.