Natural England

Evaluating Knowledge Management (AP03/24 - Annex 3)

Annex 3

KM Evaluation Findings (Analysis of interviews by theme have been removed for brevity but are available from barney.smith@countryside.gov.uk)

The Countryside Agency

Knowledge Management Strategy Evaluation 

Prepared by Simulacra Media and Sparknow

23 May 2003

CONTENTS

Foreword

Summary of progress and intentions

1.        1.    Summary of main findings

1.1       

1.2        Skills training, roles and responsibilities

1.3        Culture and behaviours

1.4        Information management and systems

1.5        Development of internal networks and communities

1.6        Possible futures for the KM programme and KM Team

2.        Recommendations of the consulting team

3.        Analysis of interviews by theme

3.1       

3.2        Skills training, roles and responsibilities

3.3       Culture and behaviours

3.4       Information management and systems

3.5       Development of internal networks and communities

3.6       Possible futures for the KM programme and KM Team

Conclusion

APPENDICES

FOREWORD 

When we wrote our original strategy documents for you, we felt it important to characterise the documents as working documents rather than finished assets to be shelved.   It seems pertinent to re-emphasise this on this occasion.   The point of an evaluation is not to be a finished product but to act as some kind of punctuation or pause, which allows you reflect on the past, discuss your reflections and then use this combination of record and dialogue to inform your future actions.

It is inevitable too, that we as outsiders who have interviewed only a small sample of people, cannot necessarily always detect the meaning or truths behind what has been said, or the weight we should ascribe to it.   In this sense you might describe the patterns below as a geometric index, rather than a capitalisation-weighted one.   And since each conversation took its own turn, systematic comparison is not viable, only a kind of inference by us that some clusters might have deeper meaning and resonance.

Added to that, this document comes forward for your consideration at a time when the Agency has undergone considerable changes, in particular a fundamental restructuring and change programme, and change of key personnel who led the implementation of the original strategy.   To disentangle personality and leadership from what has (or has not been achieved) is not appropriate in this document, but is a key point for you to consider.   Equally, the conditions in the whole Agency are different now, and it is hard to isolate the impact of KM from other activities.  

More challenging, perhaps, if knowledge management is really working at its best, it is invisible, embedded inside other things or taken for granted.   So how do you get people to notice it?   You might not even want people to notice it.   Invisibility might be a key criterion of success. You, as insiders, will be the more honest judges of that. This may require a shift of perspective from ‘them the KM team doing that thing to us which we (chose from reluctantly/enthusiastically/sceptically/anxiously) sign up to’ to ‘we, as individuals together with the Agency embracing a philosophical approach’

So the way to use this document is to consider it as work in progress:   a tool to support active dialogue, and then lead to action.   It is a selective, but by no means necessarily representative, set of perspectives against which to weigh your own views and experiences. the structure of the main document is deliberately in two voices (excluding Ian’s voice):   the voices of interviewees, reorganised into patterns, and then a consulting commentary which is intended to provoke some new thinking. We have not drawn any conclusions about specific action plans or projects.   There are no easy recommendations at this stage, and it is not our job to pretend that there are.

At the outset we have also included a summary by Ian Bilsborough of actions to date, and intended future actions and we have also referred to this in the Executive Summary, but you should be aware that we have not validated this.

Finally, to make full use of this Evaluation, you would benefit from reading and reflecting on the original reports and appendices submitted by us and the key documentation developed by Barney Smith and Liz Thompson.   These are all available from Ian.   The scope of our current assignment has not allowed us to reincorporate materials from these as actively as we would have liked to.

We would like to thank you for the invitation to come back and visit after 2 years.   This is a privilege, which we feel proud to have conferred on us.

Claudine Arnold, Sparknow

Gill Stewart, Simulacra Media

Victoria Ward, Sparknow

SUMMARY OF PROGRESS AND PLANNED ACTIVITY 

Following completion of the evaluation report Ian Bilsborough has provided a summary of progress made since the interviews were undertaken, together with details of planned activity. These are summarised below by category and should be read in conjunction with the findings of the report.

There has been a positive trend in communications, which includes growing dialogue by the knowledge management team (km team) with the regions, visits, twinning between activities, developing new media for group meetings (e.g. video and online conferencing techniques), publications and external speaking engagements.

That said, all interviewees said that more work could be done both in internal and external communications.   In particular, the relationship between internal communications, knowledge management and the publishing strategy needs to be considered as a strategic whole (as we recommended in our original reports). 

Not enough emphasis is placed on understanding the different internal and external audiences of the Agency and matching the communications approach and language to their needs.   This matter goes beyond the remit of a knowledge management programme.

There is perhaps too much emphasis on publications and IT based solutions.

1.2    Skills training, roles and responsibilities 

There are few identified KM roles (Learning Network facilitator is one).   Most people felt that there was a lack of core KM skills within job descriptions and the appraisal processes. There was a strong desire to make this more explicit.   It is also true to say that at the time the interviews were conducted, the new Work and Development Plans, which embed core knowledge objectives, has not been fully communicated.   This should be done. 

Our assumption is also that the new Change programme (which we have not considered in detail) shifts the role of appraisal from a box-ticking exercise to a meaningful explicit process, so the role of a knowledge objective can have real influence rather than be lip-service to organisational documentation.

Some effort also needs to be made to shift mindsets to recognise the value of sharing information so that this becomes part of the daily routine, rather than being seen as an extra activity. This could be achieved by better training and induction programmes and better sign posting to sources of help and information.   A combination of stick and carrot is preferred.   There was also a feeling that people have not quite understood the value of KM and its relevance to their job.

In the jargon of competence and incompetence, it seems that the Agency, and a reasonably broad base of individuals in it, are moving from ‘unconscious incompetence’ to ‘conscious incompetence’.   That is to say that people are becoming more tuned in to shortcomings, and aware of what might be.   There were some very useful practical suggestions from interviewees in this respect.

One disjunct we noted was between the relatively radical shifts in library space from library to resource centre, which do not seem to be accompanied by an equally radical shift in the role of the information specialists.

1.3   Culture  and behaviours 

The early challenge was getting the message over that knowledge management should be taken seriously, individually and collectively and at all levels of seniority.   Despite some patchiness, it is clear that this is now recognised, both in the official alignment of plans and knowledge objectives, and in the more informal commentary and observation.

There is evidence of a change in behaviour and an increased recognition of the need to share information.   Some of the improvements in communications and better IT systems support this, and for some this is increasingly reflected externally and so having a positive impact on the Agency’s reputation.    However, it is difficult to judge the extent to which the KM programme has influenced this rather than simply the behaviour of highly-motivated individuals – people feel that the Agency very broadly splits itself into two camps: those who actively take initiative and ownership and those who do not.   

There are still vestiges of past organisational cultures, historical and current silos, this effect being compounded by high staff turnover.    More can be done to change mindsets. The high staff turnover is an opportunity as it does offer the opportunity to create active induction into fresh and enthusiastic recruits.

As a visible manifestation of commitment to change, the early decision to invest in the creation of a physical space (the Resource Centre), was a strong positive signal, and an unusual perceptive early move compared to other knowledge strategies we have observed.

Some nice little prompts, nudges and reminders are being distributed by the km team in the form of, for example, handy tips and hints on how to run a good meeting.   This seems nicely judged in size and tone.

1.4    Information management and systems   

There has been a heavy, and valid, early investment in attention to information management and systems, from databases to taxonomies to Learning Networks (which are largely outside the specific scope of this report.)   However, an inevitable consequence of this is that the organisational structure (IT serves KM) looks more to the uneducated observer like the opposite (KM serves IT).   This relates directly back to some of the key communications challenges for the km team in the future.

There are an increasing number of databases and sources for information, some of which are good examples that are welcomed by their users.   The success of these databases is linked to the amount of proactive management apportioned.

However there is a general feeling that as new systems develop this is leading to information overload and a lack of clarity and guidance on what is held and where. This is compounded by insufficient connectivity between systems, no effective search mechanism nor consistent taxonomies.   Taxonomies that are currently under development are in danger of being inwardly-focused.

There is a desire for clear linkage between systems, supported by guidance and protocols, so that information can be more easily found and delivery is tailored to the needs of the individual.

It will be important in future activities to make sure that the KM programme is seen as a portfolio of activities with a balance between explicit systems and structures and tacit systems and structures (narrative work).   The new plans for a portal (metallic in its metaphor) should be quite clearly balanced by other projects, like narrative work, which have a more organic flavour.   The portal and taxonomy also open the gate for a more radical overhaul of the role of the information specialists in analysis, advice and coaching in information management, research, record-keeping and publishing skills.

1.5    Development of internal networks and communities 

There is recognition that networks, both formal and informal, are valuable.   The key to their success depends on the extent to which there are formal roles involved (e.g. Learning Networks).    There is a stated desire in horizontal groups across the organisation to establish networks.   It appears that allowing networks to emerge has a better success rate.   Thus continuing to offer help and support to emergent networks has a role which is equally to that of spawning new official networks, and one just needs to be grown-up about the fact that the metrics on such developments are far from straightforward.   (See also much that is written by us and by others about the risks inherent in overformalising emergent structures.   This was something we highlighted in our original recommendations too.)

1.6    Possible futures for the Knowledge Management (KM) programme and the KM team   

There is a general feeling that the activities fostered by the KM team have been successful and the existence of the programme has initiated a number of valuable activities. There is however uncertainty about the overall purpose of the programme and how it differs from other initiatives in the Agency.

There is a need for more clarity on both the role of the team and programme, supported by better communication processes and linkages with other initiatives in the Agency. 

 
RECOMMENDATIONS OF CONSULTING TEAMPractical and philosophical suggestions for action and attitude are threaded throughout this document.   Our experience tells us that this is a useful characterisation for recommendations.   This is what we think will help you most if you can pull it off:

From our conversations with the sponsor we have understood that the programme to date has been underpinned by technology development. There is a desire now to shift the focus over the next 12 months to the more human related aspects of KM and develop the appropriate behaviours across the Agency through the development of skills, training and networks.

This in itself presents a risk. Whilst you devise a new and complementary approach to change behaviours, you should not lose sight of the activities already developed and, taking on board lessons learned from both success and failure, you need to continue to support these activities in an appropriate way.  

This shift is about to occur in a changed organisational environment with a new set of key people with whom you need to forge relationships and partnerships; and you need to place these emerging relationship in the context of existing ones which already work well.  

And that all this needs to be undertaken in an external environment where regionalisation is increasingly important and will bring with it increasing demands on the Agency from government and other bodies.

There is a dual aspect here: the need to understand the risks involved in any given activity and that of seeing all actions as part of a bigger portfolio and so managing it as such. 

As a priority you need to identify key sets of people   (e.g. Richard Wakeford and the executive, 4 Directorate heads, the 8-10 Programme Directors, 40 programme managers, key mid office communities (finance network), 60 editors (rising to 120), individuals in the regions, who will promote KM actively and help you define the programme going forward. 

Then you need to plan an appropriate campaign of communications and activities over time with these stakeholders, drawing on effective dialogue and story to illustrate your approach.   At outset for e.g. short face to face conversations with people on a regular basis can be used to great effect to gain support. This should mitigate the risk that the KM programme will lose its voice and momentum in the face of other pressures.