Archive for the ‘Policy’ Category

EBP TOOLKIT FOR PROGRESSIVE POLICY MAKERS IN DEVELOPING COUNTRIES

August 19, 2008
 

EVIDENCE BASED POLICY : IMPORTANCE AND ISSUES

 

WHAT IS EBP?

o    Evidence Based Policy is adopted in UK by New labor government from 1997 includes lessons and approaches which are valuable to developing countries.

 

o    Useful for the progressive policy makers and advisors of public sectors in developing countries

 

DEFINITION OF EBP

o    EBP is an approach that helps people make well informed decisions about policies, programs and projects by putting best available evidence from research at the heart of policy development and implementation (Davies : 2004)

 

o    A set of methods which informs about policy processes, rather than one which aims the eventual goals, includes systematic evidence and rational analysis.

EBP

o    It is a shift away from opinion based policies to more rigorous and rational approach that gathers, critically appraises and uses high quality research evidence to inform policy making and professional practice.

DYNAMICS OF EBP

WHAT EVIDENCE IS USED?

o    Research based

o    Any systematic approach to increase the stock of knowledge (OECD,1981)

o    Data collected through systematic approach

o    Critical investigation, evaluation, theory building, data collection, analysis, and codification related to development policy and practice.

o    Action research – self reflection by practitioners oriented towards the enhancement of direct practice.

 

CONSIDERATIONS

o    Not all forms of evidence share equal importance, relevance, weighting

o    Hierarchical judgments of what evidence to use, where and how- often deeply embedded in assumptions regarding validity and power.

o    Using only empirical data and hard evidence is narrow

o    Tacit forms of knowledge, practice-based wisdom, the voices of ordinary citizens are equally useful

o    EBP is thus to take wide breadth of sources of research, not just hard evidence

 

WHAT SHOULD GOVERNMENT CONSIDER?
(Court, Hoveland, Young 2005, Shaxson 2005)

 

o    Accuracy – Is the evidence correctly describing?

o    Objectivity- Quality of approach to generate evidence and objectiveness of the source, extent of contestation of evidence

o    Credibility- Reliability of the evidence, dependability for monitoring, evaluation and impact assessment.

o    Generalisability – Extensive information, not just selective cases and pilots

o    Relevance – Timing, topicality  and policy implication of evidence

o    Availability – Existence of good evidence

o    Rootedness – Grounded in reality

o    Practicalities – access to evidence in a useful form to policy makers, and feasibility and affordability of research in policy making

 

INCORPORATION

o    Evidence is incorporated into policymaking by

n     Different stages of policy processes

o    Agenda setting

o    Formulation

o    Implementation and

o    Evaluation

n     Different types of evidences are required to different parts of the policy process

n     Time considerations influences mechanisms to mobilize evidence

 

ISSUE IN USE OF EVIDENCE (Pollard and Court 2005)

Stage

Description

Different evidence issues

Agenda setting

Awareness and priority given to an issue

Identify (new)problem, evidence building for its magnitude to realize its importance, credibility of evidence, communication of evidence

Formulation

Determination of policy options,

Selection of preferred option

Detailing of understanding of specific situations and different options, making informed decisions about policy and implementation, linking activity with outcomes, asses cost and impact scale, quantity and credibility of evidence is important

Implementation

Actual practical activities

Focus on operational evidence to improve effectiveness of initiative, analytical work, systematic learning around technical skills, expert knowledge & practical experience, Action research, pilot project, evidence should be practically relevant across different contexts

Evaluation

Monitoring and assessing process, impact of an intervention

Develop monitoring mechanism, Comprehensive evaluation procedure to determine effectiveness of the policy for future decision making, Objective evidence is thorough and relevant and its successful communication into policy process

 

IMPLICATIONS OF EBP IN DEVELOPING COUNTRIES

o    Have potential to show greater impact on socio-economic outcomes

o    Less established in developing countries than in developed countries

o    Its better use in policy and practice reduce poverty and improve economic performance

o    Evidence improves, lack of evidence devastates

o    EBP use in Tanzania health policy helped reduce 40% of IMR bt. 2000-2003 in two pilot districts

o    Its ignorance deepened HIV-AIDS in some developing countries regarding its causes and prevention

 

EBP TRANSLATION – DEVELOPING COUNTRY CONTEXT

o    To consider the diversity of cultures, economic and political contexts make valid generalizations difficult

 

KEY DIFFERENCES IN DEVELOPING COUNTRIES (Court 2005)

o    Weaker economics and scarcer resources

o    Difficult political environments – Limited political freedoms, weak public accountability systems, Political volatility lead to negative impact of policy also

o    Problems with accountability, participation, corruption and lack of incentives/capacity to draw in evidence in policy implementation

o    Academic freedom, media freedom, strength of civil society matter for effective EBP

o    Limited capacity to generate rigorous evidence

o    Conflict – civil wars and low intensity conflicts

o    Lack of performance management

o    Lack of indicators at political level monitoring service provision

o    Lack of institutional mechanism

o    Lack of ongoing evaluation

o    All factors effect supply and demand side of evidence based policy

 

REQUIREMENTS

o    Demand for greater incentive to improve policy and performance and to improve accountability

o    More open entry points into policy making and fewer constraints on communication are available in democratic contexts which is required. Local technical expertise often contribute to improve political frameworks within the context of a democratic polity. Tanzania is other good example.

* GETTING EVIDENCE INTO POLICY : APPROCHES AND TOOLS IN UK

August 19, 2008

APPROACHES

n     Policy makers to understand value of evidence, more informed about research, gain access to it, be able to critically appraise it (Davies, 2004)

n     To arrive at an agreement of what research means by policy makers and researchers

n     Ministers and senior officials to own the evidence and gain commitment from it

n     Commitment to use findings in non-hierarchical, open and democratic environment

n     Provide incentives to establish evidence of effectiveness and efficiency ( Eg: UK – Treasury).

n     Allocating incentives to summon and use sound evidence in devolution of budgets to frontline agencies and decision making bodies

n     Systematic reviews of what is known, increased use of routine assessments and audits

n     There is a need for ongoing evaluation of health systems and policies in Sri Lanka ( Hornby and Perera – 1997)

n     Increased communication between the research and policy worlds to strengthen integration of policy and evidence.

n     Using in-house and outside researchers in policy process

n     Integrated teams to understand the questions to answer (Eg: Employment Retention and Advancement (ERA) demonstration project (Davies: 2004)

n     Setting up of intermediary bodies.

n     UK has new set of institutions to organize and create knowledge in health.

n     Co-location of policy makers and internal analysts, if required to have sustained interaction

n     Use of secondments to exchange of staff between government departments and universities.

n     Learning each others languages, more fora for discussion, joint training and professional development opportunities for policy makers and researchers (Davies, 2004)

*SPECIFIC TOOLS USED BY UK GOVERNMENT

August 19, 2008

OVERVIEW AND CHECKLIST

(Impact Assessment and Appraisal : Guidance checklist for policy makers)

 

STRATEGY AND POLICY EVALUATION

       Strategy and Survival Guide

       Magenta Book: Guidance notes on policy evaluation

       Green Book: Appraisal and Evaluation in Central Government

       Regulatory Impact Assessment

 

ENSURING KEY PERSPECTIVES

       Incorporating Regional Perspectives into Policymaking Toolkit (Sub-national)

       International comparisons in Policy making toolkit

       Gender Impact Assessment: a toolkit for Gender mainstreaming

       Managing risks to the public: Appraisal guidance

 

TESTING POLICY IDEAS

       Policy Pilots

 

PUBLIC ORIENTED TOOLS

       Concern Assessment Tool

       Community Engagement How to Guide

       Connecting with Users and Citizens

 

GETTING BETTER ADVICE AND EVIDENCE

       Expert Advisory Bodies for Policymakers

       Improving Standards of Qualitative Research

 

 

1. IMPACT ASSESSMENT AND APPRAISAL

August 19, 2008

 (London, Cabinet’s Office, PM’s Strategy Unit (PMSU), 2002)

 

n     WHAT IS IT?

q    To deliver a fairer, more inclusive and competitive society, key issues and needs of different groups to be considered.

 

q    A web-based checklist prepared by PMSU to help policy makers to identify issues and highlight tools to provide effective advice to ministers

n     When is it Used?

q    At the time of policy proposal and at the end of policy process

n     How does it work?

q    As most up-to-date guidance to screen the potential impacts of the proposal

 

METHODS AND TOOLS USED BY UK GOVERNMENT

n     Costs and broad appraisal issues : Defined in economic and equity terms

n     Impact Assessment

q    Value for money

q    Access

q    Choice

q    Information

q    Fairness

n     Regulatory Impact Assessment

n     Public sector impacts

n     Quality of Life

q    Social progress

q    Environment protection

q    Prudent use of Natural Resources

q    Maintenance of high and stable levels of economic growth and employment

n     Scientific Evidence

n     Risk, Public health and Safety

n     Legal issues

n     Treaty obligations

n     Devolved administration

n     Environmental appraisal

n     Area implications

n     Policy appraisal for equal treatment

2. STRATEGY AND SURVIVAL GUIDE

August 19, 2008

(London, Cabinet Office, PMSU, 2004)

WHAT IS IT?

       Support strategy development and strategic thinking in government

       It encourages project based approach with four typical phases

       Foster strategic thinking through range of skills, useful tools and approaches

       Reference guide, not off-the-shelf solution

 

HOW DOES IT WORK?

        It has two sections

       1. Strategy development

        Justification and set up

        Research and analysis

        Strategy formulation

        Policy and delivery design – all links to

       Typical tasks

       Example outputs

       Management issues to be considered

       Typical questions to be asked

       Relevant skills

        Strategy skills section

       Managing people and the project

       Managing stakeholders and communications

       Structuring the linking

       Building an evidence base

       Appraisal options

       Planning delivery

        Number of helpful tools and approaches for skills make up a toolkit for strategy practitioner, right tool for right job improves the efficiency and effectiveness of strategy work. Appropriate blank templates also available. (eg: SU childcare project)

        Team has specific key players in thinking through implementation

        Setting up working groups and specified key deliverables

        Delegated a detailed work to establish ownership, buy into specific tasks and overall conclusions

        Team plan in table presents specified key conclusions, outputs, activities, lead responsibility, key stakeholders and time-table

        For Ministerial version an additional column is left for further comments

        To publish plan for key stakeholders to account for against it

3. MAGENTA BOOK : GUIDANCE NOTES ON POLICY EVALUATION

August 19, 2008

Government’s Chief Social Researcher’s Office, 2003)

l     What is it?

l     Series of guidance notes to evaluation and analysis

l     Helps intelligent customers and intelligent providers to determine high quality work in the field

 

WHERE DID IT COME FROM?

l     From post 1997 demand for evidence based policy making and changing needs of analysis

l     From demand for high quality evaluation, appraisal, analysis and policy making.

 

WHEN DOES WE USE IT?

l     Throughout the Policy making process, from commissioning to eventual evaluation

 

HOW DOES IT WORK?

l     As user friendly guide for specialists and generalists, at the time of commissioning, undertaking and management of policy research and evaluation.

l     When technical details are necessary to expand on methodological procedures and arguments

l     Subject approach from the perspective of analysis, and avoids disputes from different schools of thoughts

 

PROVIDES GUIDANCE ON

l     How to refine a policy question to get useful answer

l     The main evaluation methods used to answer policy questions

l     The strengths and weaknesses of different methods of evaluation

l     The difficulties that arise using different methods of evaluation

l     The costs involved in using different methods of evaluation, benefits to be gained

l     Where to go to find out more detailed information about policy evaluation and analysis

 

COST ANALYSIS IN ECONOMIC EVALUATION

l     Cost-analysis

l     Cost-effectiveness

l     Cost-benefit analysis

l     Cost-Utility

4. GREEN BOOK : APPRAISAL AND EVALUATION IN CENTRAL GOVERNMENT

August 19, 2008

GREEN BOOK

HM treasury, 2003

§        What is it?

§        Provides guidance on economic appraisal of cost and benefits of policy options.

§        General approach to carry out options’ appraisal of all government intervention.

§        Requirement for all expenditure and for all new policy actions which have impact on businesses, charities of public sector

§        Discusses risk and uncertainty in general terms

 

WHERE DID IT COME FROM?

§        What is it?

§        Provides guidance on economic appraisal of cost and benefits of policy options.

§        General approach to carry out options’ appraisal of all government intervention.

§        Requirement for all expenditure and for all new policy actions which have impact on businesses, charities of public sector

§        Discusses risk and uncertainty in general terms

 

HOW DOES IT WORK?

§        It includes techniques and issues that should be considered while carrying out assessments

§        To ensure that no policy, program or project is adopted without first having to answer these questions

§        Are there better ways to achieve this objective?

§        Are there better uses for these resources?

 

WHEN IS IT USED?

§        All new policies, programs, projects of revenue, capital or regulatory should be subject to comprehensive but proportionate assessment so as best to promote the public interest.

§        Green books is at agenda-setting stage of the policy process, before the policy is adopted.

 

ACTIVITIES COVERED

 

Policy & Program development

Decisions on the level and type of services or other actions to be provided, or on the extent of regulation

New or replacement capital projects

Decisions to undertake a project, its scale and location, timing, and the degree of private sector involvement.

Specification of Regulations

Decisions, for example, on standards for health and safety, environment quality, sustainability, or to balance the costs and benefits of regulatory standards and how they can be implemented

Major procurement decisions

Decisions to purchase the delivery of services, works or goods, usually from private sector suppliers

 

5.REGULATORY IMPACT ASSESSMENT (RIA)

August 19, 2008

(From Welch, 2004)

 

WHAT IS IT?

n     Tool developed by Cabinet Office, aims to improve chain of causality and advice (Shaxon, 2005).

 

n     Methodology to design precise, targeted regulations that achieve legitimate policy aims with the minimum burden on those affected.

 

WHEN IS IT USED?

n     RIAs must be completed for all policy changes that affect public, private sectors, charities, voluntary sector or small businesses.

n     Used throughout the policy making processes

n     At the beginning also to develop better policy

 

HOW DOES IT WORK?

n     Provides a guide to develop a good quality policy

n     Include thorough analysis of full range of options available to government for addressing a policy problem

n     Calculation of costs and benefits to ensure that new measures are justified fully

 

STEPS IN RIA

n     Title of Proposal

n     Purpose and intended effet of the proposal

n     The policy problem

n     Options

n     Impacts

n     Distribution of impacts

n     Results of consultation

 

WHAT ARE THE BENEFITS OF USING RIA?

n     Improving quality and efficiency of government interventions

n     Enhancing competitiveness

n     Increasing transparency and accountability

n     Reducing opportunities for corruption

n     Producing a tool for policy monitoring and evaluation

 

WHAT ARE THE CHALLENGES ATTACHED TO USE RIA?

n     Lack of awareness and acceptance of RIA within government and Civil Society

n     Institutional capacity

n     Problems of data availability

n     A lack of a coherent, evidence-based and participatory policy process within developing countries- Policies are often made by the minister, after consultation with one or two advices.

 

THE RIA AND CHANGES IN POLICY ENVIRONMENT

n     RIA implementation requires

n     Changes in institutional and policy environment

n     Resources

n     Capacities/skills

n     It leads in shift in balances of power in 3 dimensions of

n     Institutions at the Center government

n     The Centre government and line ministries

n     Ministers and officials

6. INCORPORATING REGIONAL PERSPECTIVES INTO POLICY MAKING TOOLKIT (SUB-NATIONAL)

August 19, 2008

Cabinet Office of the Dy. Prime Minister, 2002

WHAT IS IT?

¢    Policymaking is informed by a proper understanding of regional issues and priorities

¢    Practical resource to support new thinking an a set of ideas and techniques to design better policymaking processes.

¢    Toolkit contains exercises to promote learning, encourage successful working and promote integrated policy development

 

WHAT DOES IS SAY?

¢    It highlighted seven building blocks of sound policy development, lead to long-term joined-up success.

¢    Culture

¢    Mapping needs

¢    Strategic fit

¢    Networks and Communication

¢    Project planning and accountability

¢    Organizational Capacity

¢    Evaluation and feedback

 

HOW IS IT USED?

¢    Explores building blocks in more depth and provides prompts, exercises and ideas to engage regions in policy making

 

¢    It has two sections

l     Sec-1- Diagnostic checklist to rate regional or departmental performance against these seven building blocks and to help you focus on areas where development is most needed.

 

l     Sec-2 – Develops action-oriented exercises for each of the building blocks

 

         Potential barriers to outward looking

         Regional and Central exercises

         To establish good foundations

         Developing a specific new policy

 

Cabinet Office of the Dy. Prime Minister, 2002

 

 

WHEN SHOULD IT BE USED?

         Used at three stages of policy making process

         New policy initiative

         Reviewing and updating established policy

         Quick initiatives responding to crisis events and public concerns.

7.INTERNATIONAL COMPARISON IN POLICY MAKING TOOLKIT

August 19, 2008

(Government’s Centre for Management and Policy Studies (CMPS), 2002)

 

WHAT IS IT?

p    A toolkit for ‘Modernizing Government’s Whitepaper’ to become more forward and outward looking

p    Learning lessons from other countries and integrating international dimensions into policymaking to ‘expand horizons’.

p    To provide practical help and guidance to policymakers in the use of international comparison in policymaking

 

WHY IS IT IMPORTANT?

p    To use international comparisons that provide what work and what doesn’t in practice reveal new mechanisms for implementing policy and improving the delivery of public services.

p    Policymakers can also learn from the way in which other governments undertake the process of policymaking.

 

WHAT DOES IT SAY?

p    While searching for international comparisons for ideas and values we have to look not only into national governments, but also at sub-national, regional, local, businesses and not for profit organizations working with governments.

 

p    To identify existing sources of information, expertise and institutional memory, social sciences, researchers, economists, statisticians, scientists and librarians in their organizations.

 

WHEN SHOULD IT BE USED?

p    To cultivate networks of contacts in other administrations and international organizations and in academic research community.

p    To make aware of current practice and relevant developments in other countries, to be able to incorporate comparative information into your analysis and advice as a matter of routine.

 

HOW DOES IT WORK?

p    The toolkit highlights

 

n     Policymakers to scan the horizons for interesting approaches and innovative development

n     To select one or more promising comparators for closer systematic examination

n     To understand how the comparators work in practice

n     The CMPS International Comparisons Workbook provides practical help to explore the critical factors in social, economic, cultural and institutional environment

n     To analyze the relevance of the comparator

n     To establish the possible lessons from case studies