Natural England

Research Strategy (AP03/29)

1.   This is the first of two papers concerning a Research Strategy for the Countryside Agency.   This one presents the Board with a proposed framework for the strategy.   If the framework is approved, a fully worked up version will be presented in November.   This will allow the new strategy to be used in business planning for 2004/05.

2.   Research is a key function for the Agency: it underpins our remit as statutory champion and watchdog, it informs the way we influence and inspire solutions to rural problems, and it helps ensure that whatever we do, we add value.

3.   Many valuable research projects have been completed, often in partnership, since the Agency was created in 1999.   Our current research activities reflect both our strategic priorities and the long experience of our parent bodies in collecting and using evidence on countryside matters.   Research on specific topics is handled by the appropriate specialist teams and cross cutting research is led by the central research branch (Evidence and Analysis).  

4.   There are many good examples of Agency research.   Recent research notes on how Village Design Statements influence new development (CRN 53), and an evaluation of pilot Town Design Statements (CRN 56), show how research can build cumulatively.   An example of cross cutting research is the work that underpins the affordable housing indicator theme, developed in Evidence and Analysis but used by colleagues within the Agency and by external partners.

5.   However we do not have a clear summary statement of our research priorities, nor do we have an Agency-wide set of quality assurance mechanisms - that is, research definitions and guidelines.    The proposed research strategy will deliver these in a transparent and accountable way.

6.   The production of this strategy is not just driven by internal needs.   We welcomed the creation of Defra, which has resulted in a higher national profile for rural affairs.   Some of our work is complementary, but our roles, including our research roles, differ.  

·         Defra’s evidence base must underpin and inform:

a.        existing and proposed government policy on rural affairs;

b.       progress on delivering Rural White Paper commitments; and

c.        the next government spending review. 

·         Research by the Countryside Agency supports: 

d.       our advice across the whole of government on rural proofing;

e.        our Chairman in his role as Rural Advocate; and

f.         our statutory duties, where we are responsible for delivery (in landscape designations, demonstration projects, advice on countryside matters and access administration).

7.   Whatever the outcomes of the Haskins report, robust research that explores the needs of and concerns for England’s countryside will continue to be vital.   We require agreed, prioritised research themes, precise enough to reflect the special needs of the sector, but flexible enough to respond to change.   We also need to articulate the principles that guide research activities, so that quality is assured.   A thematic, prioritised, quality assured approach is proposed in this paper. 

Progress to date

8.  Evidence and Analysis – The Agency’s central support team, Evidence and Analysis (E and A) has recently been expanded.   The role of E and A is to support research activity across the Agency, leading in developing an informed understanding of rural issues, in a wide context and with a long-term view.   In doing this, the team uses specialist expertise and a cross-disciplinary approach.

9.  Business Plan Process – Agency staff work to a series of targeted, regularly reviewed business plans.   These define what research is being done across the Agency at any one time.   The Business Plan process allows for the tracking of individual projects and is an essential tool for effective research planning and management.

10.  Creating a centre of expertise in national and regional data collection, sharing and analysis - A good start has been made in developing regional State of the Countryside websites into effective tools for influencing rural policies within specific regional agendas. 

11.  Research database and learning network    - Both the software and hardware for the research database is nearly ready.   Data to populate that database will initially be provided by the recently revised research listing (see Annex 1).   Creating a research learning network is in this year’s E and A business plan.

12.  Research definitions and guidelines – A draft set of research definitions has been produced (see Annex 2).   These are currently being discussed with Finance Section, which is looking at ways of improving coding.   These definitions need wider consultation, refining and testing before they are formally adopted.   Work on a full set of research guidelines will follow immediately.

13.  Consultation – Regional and Programme Directors and Managers have been consulted on a draft of this paper.   DEFRA is aware that work on a research strategy is in hand and will be consulted before the full strategy is brought to the Board in November.

Proposals – a research framework

Research themes – the ‘what?’

14.   We propose that all Countryside Agency research should fall within at least one of the three themes listed below.   These themes have been designed to cover current research and potential future research activity. The themes will be explained in more detail in the full strategy. 

g.      Countryside aspects (relatively narrow targeted research);

h.      The holistic countryside (possibly – the ‘joined up’ countryside – a theme that encompasses the range of cross-cutting research activities);

i.        Research that supports rural proofing and countryside advocacy.

Research principles – the ‘how?’

15.   The list below sets out research principles that are already in frequent use.   These principles are a clear demonstration, for both internal and external audiences, of our commitment to quality research.   The full strategy will contain more detailed guidance, as the way they are used and the priority given to each principle will vary between projects.   The principles are listed in suggested priority order.   Our research must be: 

j.        Fit for purpose  – each project must relate to at least one theme.   Some projects will be reactive, exploring situations and problems that others present to us, that require further understanding.   Some projects will be pro-active, where we take the initiative and explore new topics, or look forward (such as this year’s ‘State of the Countryside 2020’);

k.      robust – our research must be credible and defensible; it should be focused, so that our understanding of specific themes and topics is increased; it should be politically informed, to underpin the advice that we give to government on countryside matters; our methods must be sound, so that we are confident in our evaluations of our practical, innovative programmes. 

l.        timely – our research must be sensitive to current and future needs and sensibly prioritised;

m.    well managed    - we need to hit a balance between centralised and devolved research management.   

n.      visible and understandable    - this means not just communicating with immediate clients and partners, but getting messages, in appropriate language and format, to wider audiences;

o.      efficient in adding value   - where appropriate, projects should demonstrate additionality, by building on existing work.

The ‘research hub’

16.   One of the defining qualities of much Agency research is the way it is done by specialist teams, located both in our headquarters and in our regional offices.   Such a system produces highly focused, fine grained and targeted research but it also carries risks.   To overcome these, we need more transparency about how we deliver consistent, high quality research that is effectively communicated both internally and externally.   In fact we need to strike more of a balance between devolution and centralisation.   We believe that this balance is best achieved by developing a ‘research hub’ function for the central E and A team.   The ‘research hub’ will inform, advise and coordinate.   It will act as an information exchange and advice centre for all teams undertaking research, and will help Finance Section and Internal Audit monitor and evaluate research activities. Before bringing the full strategy back to the Board, we will consult on ways of integrating ‘research hub’ activities with the annual business planning cycle.  

The ‘research toolkit’

17.   A key role of the ‘research hub’ is to help deliver quality research.   This will be done through a ‘research toolkit’, incorporating the research definitions that are already in draft and other good practice guidelines on topics such as methods, developing research proposals, evaluation and communication. 

18.   E and A will produce the ‘toolkit’ in cooperation with other teams such as Human Resources (training), Finance Section (coding) and Internal Audit (monitoring and evaluation).   Once the toolkit is in operation, E and A will keep it under review, working with other teams to ensure understanding and compliance.   More detail of how ‘toolkit’ would work will be presented to the Board in November.

19.   An outline Action Plan for implementing both the ‘research hub’ and the ‘toolkit’ are in Annex 3.   A diagrammatic representation of the way the hub will link with other teams, is in Annex 4.

Financial and staffing implications

20.   The full implementation of the research strategy should result in better value for money for the Agency overall.   Rolling out the ‘research hub’ and the ‘toolkit’ will require some reallocation of staff time and responsibilities.

Next Steps

21.   Evidence and Analysis will produce a full draft Research Strategy for the November Board meeting, after further consultation with Regional and Programme Directors and Managers, and with our main external partners.

Risks and Mitigation Measures

22.   The imminence of the Haskins report means that there is uncertainty - and thus risk - about the organisational structure in which the Research Strategy will operate.   In mitigation, we need a research strategy to demonstrate that our research is fit for purpose.   This means two things.   First, that we are researching the right topics (hence the themes) and second, that our work is robust and high quality (hence the research hub and toolkit).   Clarifying these things at this stage is invaluable.   They can be carried over into whatever organisational structures emerge.

23.   There is a risk of lack of transparency and consistency in a research programme that is largely devolved.   Moving to a completely centralised approach to research carries different risks, that we would lose our current finely focused approach to topics and localities.   In mitigation, developing the proposed ‘research hub’ and associated ‘research toolkit’ would retain the benefits of devolution, combined with central oversight.

24.   We will consult our main partners on the themes and principles before the Strategy returns to the Board.  

Annex 1

COUNTRYSIDE AGENCY RESEARCH

CURRENT BUSINESS PLAN COMMITMENTS

Work on a Research Database is nearing completion.   To populate that database, Evidence and Analysis and Knowledge Management have compiled a Research List, from data provided earlier this year by Business Managers.   The information relates to projects defined by each team as research, which are either ongoing or planned during this financial year.

The figures below relate to a first stage analysis of the figures.   So far we have not gone back to the teams to discuss the figures, nor have we analysed the data using the draft research definitions (see Annex 2).   We are aware that there may still be some projects in the current list that may fall outside a strict definition of research.   We are also aware that some Agency activities that involve a considerable amount of data collection and analysis – such as the Access Mapping programme – are not at present included in the Research List.

The list shows that in the 2003-04 financial year the Agency is directly engaged in research projects worth £9.139 million.   Activities in directorates A and B account for more than £8 million, with the remainder coming from Directorate C.

Split by current Corporate Plan objectives, the details are:

Directorate A:   Making Life Better for People in the Countryside   £4.277 million

1.1 Communities                                             

1.2 Local governance                                      

1.3 Voluntary sector                            

2.1 Service providers                                       

2.2 Local services                                            

2.3 Housing                                                     

2.4 Transport                                                   

2.5 Disadvantage                                           

2.6 Advocate                                                     

3.1 Economies                                                 

3.2 Market towns                                             

        Misc.                                                          

Directorate B:Improving the Quality of Life for Everyone   £4.130 million

1.1 Finest countryside                                     

1.2 Agriculture                                                 

1.3 Rural development                                    

1.4 Countryside capital                                   

2.1 Open country                                             

2.2 Diversity barriers                                      

2.3 Public health/recreation                          

2.4 National trails                                            

3.1 Urban fringe                                              

3.2 Aggregates levy                                         

        Misc.                                                          

Directorate C:Rural Intelligence and Influence   £0.732 million

1.1 Strategic research                                      

2.1 Recognition/endorsement                      

2.2 Knowledge management                                     

3.1 Influence                                                     

Annex 2

DRAFT RESEARCH DEFINITIONS AND DECISION TREE

This note is the first exercise towards a suite of good practice guidelines that could become the Countryside Agency’s ‘research toolkit’.   It was prepared in the E and A team.   An earlier version has been discussed briefly with Finance Section.   Other teams have not yet been consulted.   The ‘decision tree’ does not at present have a role in our business planning or accounting processes.

Why the Countryside Agency needs research definitions

The Countryside Agency requires a definition of research that is specific enough to be useful and accountable, but broad enough to cover the range of research activities that we undertake.

We need this because:

·         Our evidence base must be built on work that is widely accepted as research; 

·         We require greater transparency and consistency in the way we deliver research across the Agency;

·         We need to show how research activities affect the accounting class ‘payments for goods and services’ (B35), under which research now sits, in our ‘Resource Accounting’ processes.

The definitions that we adopt must meet the above needs.   It would be useful if the definitions help us ‘map’ the way research is classified within programme teams, and how that classification links through to accounting processes.   Although the requirements of our accounting system should not necessarily be seen to lead research practice, it would be good to have consistency and clarity.    If Finance Section and the teams doing research were to use different approaches to classifying research, we run the risk of not being as clear and consistent as we should be.

A draft definition and typology of research

To be of most use, our definitions of research need to clarify not just what research is, but what its uses are.    As a government body we are committed to using the definition of research developed for the OECD and explained in the Frascati Manual.   The Frascati summary definition says that:

Research and experimental development (R&D) comprise creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications.

OECD 2002 Frascati Manual   p30

This definition contains key words that we could use in our decision tree, but on its own it does not provide enough help in dividing research into its various types.   For more detail, Patton’s typology (1990) of research purposes, although designed for qualitative research, appears to cover all the different types of research that the CA undertakes.   We need to consult other teams on the relevance of this typology.    This typology is not a research hierarchy, and in reading it you should not assume that ‘basic’ research is somehow better or more important than ‘applied research’.   The key question is, is the research fit for purpose?

Patton’s types are:

Basic research 

Basic research is about gaining knowledge as an end in itself.   Ideally such knowledge should be applicable across space and time. We need internal discussions on whether any Countryside Agency research could have such broad application.   We always have to say why a proposed project is relevant to the Agency’s corporate plan.   (Also, our research is based on culturally specific notions such as ‘countryside’ – which limits how far you can generalise from our data).

Applied research

The purpose here is to understand the nature and sources of human and societal problems and issues.   The actual focus, approach and methods, depend on what questions, issues or problems a society or a sector thinks important at a specific time.   This is what much of the Agency’s research is all about.   Examples include the Rural Services Survey and Countryside Quality Counts.   If robust procedures are followed – for example, in selecting sample frames - some generalisation is possible, within the defined context.   The type ‘Applied research’ needs further subdivisions. Further discussion will guide us on the most useful divisions, which would be based on a number of criteria, including corporate plan objectives and methods.

Summative evaluation

This kind of research can also be called monitoring and evaluation – it determines the effectiveness of human interventions and actions at a summative level – for example looking at the achievements of whole programmes, such as the Market Towns programme.   Generalisations across interventions with similar backgrounds or goals are possible, but such generalisations should be treated with caution.

Formative evaluation

Formative evaluation aims to improve specific interventions, by focusing on specified projects or products.   The recent evaluation of the Forest of Dean Integrated Rural Development programme is an example.   Generalisation is only possible within the setting studied, although lessons learnt in formative evaluations can be tested elsewhere.

Action research

This is active engagement in a programme or community, to solve a problem.   To qualify as research it requires detailed documentation.    The results are only applicable in that context.   Many of the Countryside Agency’s demonstrations could qualify as action research, but only if there is consistent attention to collecting and analysing the information gathered.

Using the research typology – a draft decision tree

This purpose of this ‘tree’ is to ensure that we allocate research projects to the most appropriate code.   This information is needed for business planning, financial management and reporting to DEFRA.   In due course it will be presented as a flow chart.   The language of this version is probably too technical – we will consult on appropriate language.

1.      Is it research – (i.e. is it creative and systematic, leading to an increase in our stock of knowledge? (Frascati))

Is it also primarily of benefit to the CA? 

if yes, go to 2, if no go to 3

2.      yes – it is coded to ‘support costs’ for reporting to DEFRA – go to 4.

3.      no  – it is coded to CA e.g. D10, and allocated to ‘programme spend’ for reporting to DEFRA – guidelines developed by others, not E and A.

4.      Is the proposed research basic – i.e. knowledge as an end to itself? (this one is tricky, but we might include some State of the Countryside reporting here)

  If yes, go to 5, if no go to 6.

5.      How is it done?   If by our own staff, code it BA/B35; if it is contracted out, code B35

6.      Is the proposed research applied  – that is, it helps us understand the nature and sources of human and societal problems and issues, generalisable to the context studied?   (for example, surveys of rural deprivation and social exclusion; landscape character assessments in specific parts of England)

If yes, go to 7, if no, go to 8.

7.      How is it done?   If by our own staff, code it BA/B35; if it is contracted out, code B35.

8.      Is the proposed research summative evaluation – i.e. it assesses the effectiveness of human intervention, generalisable to interventions with similar goals? (for example, national monitoring and evaluation of a large programme such as the Market Towns, or Vital Villages programmes)

   If yes, go to 9, if no, go to 10.

9.      How is it done?   If by our own staff, code it BA/B35; if it is contracted out, code B35.

10.  Is the proposed research formative evaluation – i.e. it looks at ways of improving an intervention, programme or product, generalisation limited to the setting studied? (for example, evaluation of small scale pilots such as early versions of whole farm plans in specific LMI areas)

   If yes, go to 11, if no, go to 12

11.  How is it done?   If by our own staff, code it BA/B35; if it is contracted out, code B35.

12.  Is the proposed research action research – i.e. it involves direct engagement in an organisation or programme, to solve immediate problems, not generalisable?   (for example, location specific research to inform a single Village Plan)

  If yes, go to 13, if no, go to 14.

13.  How is it done?   If by our own staff, code it BA/B35; if it is contracted out, code B35.

14.  The activity is not research, even if, at the beginning of this process, we thought it might be.   We need to look again at whether it should be supported by the CA, and if so under what heading.

Conclusions

These definitions and the decision tree are very much works in progress.   They are examples of the issues that must be addressed if the Countryside Agency is to be transparent and consistent in developing its evidence base.

References

OECD 2002 Frascati Manual – proposed standard practice for surveys on research and experimental development   OECD

Patton, M. Q. 1990 Qualitative Evaluation and Research Methods. Sage.   London etc. P160.

Annex 3

COUNTRYSIDE AGENCY RESEARCH STRATEGY

OUTLINE ACTION PLAN

Annex 4

THE RESEARCH HUB – ROLES AND CONNECTIONS

Role of the Research Hub (based at E&A): 

·Leads in cross cutting research

·Leads centre of expertise in data collection & handling

·Leads with research guidelines

·Source of advice for all, on implementing research strategy

·Designs & reviews toolkit