Copy
View this email in your browser
Edition 20 / 24 February 2015

Progress: November 2014 to January 2015


We are close to finishing many evaluations from 2013/4 and some from 2014/15 and there will be a regular stream to Cabinet from this point.

This is an exciting time as findings go public and we move into the improvement phase.

About

Evaluation Update summarises progress with government evaluations in South Africa, led by the Department of Planning, Monitoring and Evaluation (DPME) in the Presidency. 
Twitter
Twitter
Website
Website
Email
Email

Evaluations milestones


11 evaluations are completed, up from eight and 28 more are at different stages (see the evaluation status report).Three evaluations went to Cabinet in November and the key findings are presented below. The evaluation reports can be downloaded from the Evaluation Repository on the DPME website.
 

Table: 1 Status of evaluations as at February 2015

Approved reports Improvement plans being implemented Served at Cabinet Research underway TORs approved Preparation stage Stuck
11 10 6 23   13 2

The Implementation evaluation of the Restitution Programme


The Restitution Act of 1994 as amended enables all those who lost their land under the repressive land legislation of the past to lodge land claims before 31 December 1998. The Restitution Programme is the vehicle for implementing this.

The evaluation is based on a process assessment of the programme’s implementation (from the lodgement of claims through to their finalisation), and covers the period from January 1999 to 31 March 2013, i.e. since the completion of the first Ministerial review. The purpose of the evaluation was to assess whether the Restitution Programme has been implemented efficiently and effectively, and to identify how the programme can be improved ahead of the next phase of the restitution process.

The programme has settled over 85% of the claims lodged since its inception. However, the findings of the evaluation reveal a range of systemic and operational weaknesses which compromise its efficiency and effectiveness, and undermine the achievement of its developmental purpose.

Recommendations include developing a clear definition of the function of the Commission as an independent entity dedicated exclusively to the administration of the restitution process. The evaluation report was approved in February 2014 and presented to Cabinet in November 2014. Click here for the full report.
 

Evaluation of the Land Recapitalisation and Development Programme (RADP)


RADP was launched in 2010 to focus on struggling land reform farms acquired since 1994 that have received little or no support, but have the potential to become successful, if assisted. The main purpose of the evaluation was to establish whether RADP is on track to achieve its objectives and to advise on how to strengthen implementation of the programme.

Findings indicate that RADP has made some progress towards achieving its intended objectives, but there is room for significant improvement.  About 540 additional jobs were created on the 98 farms included in the evaluation after RADP was implemented, varying across provinces with KZN much more successful.  However, the number of jobs created is too small to justify the investment in RADP given the high job losses in the agricultural sector.  Most RADP stakeholders interviewed believe that food security has improved after RADP. 

As regards agricultural production, it is on-going in 70% of the projects included in the evaluation. An area in which RADP does not seem to have made much progress is facilitation of market access for farmers – a finding highly consistent with conclusions on national agricultural surveys as conducted by Statistics South Africa. There are questions regarding whether the grant funding approach in RADP is sustainable given the limited resources available and the suggestion that it promotes dependency on state funding among beneficiaries. In addition, the same beneficiaries are targeted repeatedly, over time. 

The evaluation recommends a redesign of public agricultural support programmes and doing away with the existing silos of funding agricultural support services. The evaluation steering committee approved the report in October 2013. The management response and improvement plan were received from the Department of Rural Development and Land Reform (DRDLR) in February 2014 and the evaluation report was presented to Cabinet in November 2014. The full report can be accessed here
 

Evaluation of the Comprehensive Rural Development Programme (CRDP) 


CRDP was launched by the DRDLR in 2009 to improve access to basic services, promote enterprise development and village industrialisation. The purpose of the evaluation was to assess whether the CRDP is achieving its policy goals, how the programme can be strengthened and up-scaled and whether the institutional arrangements to support the implementation of the CRDP are appropriate.

The evaluation found that there has been mixed progress in achieving CRDP goals. It is a high cost intervention with investment per ward of up to R42 million which will be difficult to scale-up as currently designed. The modalities for strengthening coordination across the spheres of government and developing the capacity of local institutions, especially local municipalities and the Council of Stakeholders, so as to ensure comprehensive delivery on rural development, are weak.

The evaluators recommend ways of strengthening the CRDP’s institutional arrangements and integrated planning process and of improving the CRDP’s attainment of its programme goals. The model itself needs to be reviewed with clear norms and standards for rural development. The evaluation report was approved by the steering committee in October 2013. The Department of Rural Development and Land Reform has finalised the Improvement Plan and the report was presented to Cabinet in November 2014.The full report can be accessed here.
 

Policies and guidelines


There are 18 guidelines and templates on the website that can be accessed here. Guidelines on contracting and evaluation ethics are planned. If anyone has useful material on evaluation as opposed to research ethics please contact Jabu Mathe at jabu@po-dpme.gov.za .

 

News on capacity development

  • A call for an evaluation training panel went out to allow for flexibility on commissioning training, as well as for delivering training in Western Cape.
  • DPME is delivering a module for the MPhil in Public Policy and Practice at UCT, focusing on the use of planning and M&E as part of change process. For further details contact Christel Jacob at christel@po-dpme.gov.za.

Support for Planning Implementation Programmes

  • On 6 August 2014 Cabinet approved a memorandum on programme planning which includes:
    • Piloting capacity development in programme planning;
    • Departments to use the guideline for developing new programme plans; and
    • Programme plans submitted to clusters should be scrutinised to see they conform with the guidelines.
  • DPME will launch a call in February 2015 for proposals for an audit of programmes, which will assess how many programmes there are in a number of departments, and the state of those programmes.
  • In addition an evaluation is starting on the departmental strategic planning and annual performance planning system. This will incorporate the linkage with implementation programmes
  • For any questions on this contact Antonio Hercules Antonio@po-dpme.gov.za.

Quality Assurance

  • The second contract for the quality assessment of evaluations is underway, funded by DFID. There are over 100 evaluations in the Repository, of which 90 passed the minimum of 3. This assignment is also developing the evaluation tracking system. For details contact Mark Everett at mark@po-dpme.gov.za
  • An assignment on professionalisation of evaluation has started in partnership with SAMEA, using funds from DFID. This will look at how professionalisation should be taken forward over the next ten years, and will draw from experience in Canada, Mexico and Colombia. There will be extensive consultation around this assignment. For details contact Jabu Mathe at jabu@po-dpme.gov.za.
  • An assignment on evaluability assessment is underway being undertaken by the University of the Witwatersrand. We have managed to draw from a range of international resources for this, a tool has been developed which is being piloted on a national evaluation (Extension Recovery Programme) and on a provincial evaluation in the Western Cape. This is not focusing on whether an evaluation can be done but rather at what type of evaluation methodology is appropriate, considering the commitment of stakeholders to the evaluation, and the state of the data. For details contact Thabisile Zuma at thabisile@po-dpme.gov.za.

Research

  • Work is well underway on a research diagnostic to assess the role DPME should play to support the use of research evidence in the policy-making and implementation. A reference group meeting was held with DST, PSPPD, DPSA and GTAC, HSRC and NRF.
  • An assignment on the workload of DG’s has started, looking at seven national and three provincial DGs.
  • DPME is also collaborating with a second BCURE project to reflect on how DPME’s tools could be improved, and integrated. This focused on support to smallholder agriculture and land reform, and is drawing from the outcome monitoring work, the five relevant evaluations, MPAT data and external data including two relevant systematic reviews completed recently by the University of Johannesburg (on smallholder farming and urban agriculture).
  • A research repository has been developed, initially as a platform to share relevant research outputs within DPME, moving towards access for external stakeholders.
  • Using the support of 3ie, it is hoped to test a rapid evidence assessment (REA), which has a faster turnaround, using the housing sector, and ideally then carry it on to a full systematic review to assess the value of the REA. REA’s typically take three to six months while a full systematic review can take 18 months. REAs typically focus only on literature available electronically, and limit the languages covered. For details contact matodzi@po-dpme.gov.za
  • For details around the work on research contact Harsha Dayal at Harsha@po-dpme.gov.za

Sharing evaluation and research evidence

  • DPME has written a paper on the South African experience for a special edition of the Journal of Development Effectiveness. This has been published and is available on the DPME website. In addition DPME has written two chapters and a case study in a book on Evaluation Management in South Africa and Africa, edited by Fanie Cloete, Babette Rabie and Christo de Coning, available from Sun Media at orders@africansunmedia.co.za.
  • Twelve papers have been written for a special edition of the African Journal of Evaluation focusing on the launching of the South African evaluation system, and in partnership with SAMEA. Having been peer reviewed using an internal workshop process final versions are currently going out for external peer review and the edition should be published in June 2015.
  • A Study on Collaborative Partner-Donor Evaluation Work (CPDE) was mandated and commissioned by the Evaluation Network of the OECD–DAC (EvalNet) in November 2012. Ian Goldman presented (by skype) on the South African system at a synthesis and systematization workshop held from 2 to 4 December 2014 in Manila.
  • A communication strategy is being developed (funded by DFID), including the conversion of Evaluation Update to an electronic newsletter. For further information contact matodzi@po-dpme.gov.za

Status of evaluations in the Western Cape Department of Agriculture


Dr DP Troskie, Director: Business Planning and Strategy in the Western Cape Department of Agriculture reports that during 2013 Agriculture adopted a multi-year Departmental Evaluation Plan (DEP). The plan is driven by the Head of Department (HoD) and enjoys high level support within the department. The HoD expects programme managers to report progress on DEP activities at monthly departmental management meetings.

During the 2013/14 financial year the department successfully completed four evaluations following which improvement plans were produced. A notable achievement of the departmental evaluation system has been the revision of agricultural learnership training as a result of the impact evaluation of this intervention. In the current financial year (2014/15) the department plans to complete seven different evaluations including a diagnostic evaluation of farmers’ service needs;  a design evaluation of CRDP implementation and an impact evaluation of the food garden programme on household food security.

The full report can be viewed here.
 

International linkages

  • DPME is working with Uganda and Benin as well as the CLEAR initiative to develop a regional project on M&E called Twende Mbele, which will support regional sharing on M&E for countries that are committed to the use of evaluation.
  • DPME is a member and Ian Goldman a board member of 3ie. Howard White has moved on and Manny Jimenez is joining 3ie as the new executive director. Howard gave a lecture in October as the first of a series of annual Howard White lectures. This can be viewed here.

Collaborative management

  • Provincial Evaluation Plans (PEPs) – Mpumalanga and Northern Cape have approved their provincial evaluation plans for 2015+, and there has been significant progress with Free State, Limpopo, and North West. All look likely to develop PEPs for 2015/16.
  • Panel of service providers – the call for the evaluation and research panels has closed and selection will be completed in February 2015.
  • With 39 evaluations in progress, monitoring is much more complex. An evaluation management information system is being developed by PDG, including an improvement plan tracking system, and linked to the system for quality assessment of evaluations. This assignment also includes reviewing the peer review system.
  • Staff – we are interviewing for an Assistant Director: Evaluation, but otherwise are at full complement.

Priorities for the next two months

  • Completing a number of evaluations and taking a number to Cabinet, ensuring the improvement plans are underway and progress reports are being received. The assignment on tracking improvement plans is urgent to deal with these.
  • Making sure that all DFID-funded assignments are underway as the DFID-funded SPME project closes in September 2015.

Contacts

 
Dr Ian Goldman, Head of Evaluation and Research Unit ian@po-dpme.gov.za
Mr Jabu Mathe, Director: Evaluation jabu@po-dpme.gov.za
Ms Christel Jacob, Director: Evaluation christel@po-dpme.gov.za
Mr Antonio Hercules, Director Evaluation antonio@po-dpme.gov.za
Ms Matodzi Amisi, Director Evaluation  matodzi@po-dpme.gov.za
Mr Mark Everett, Deputy Director Programme Administration mark@po-dpme.gov.za
Ms Harsha Dayal, Director: Research harsha@po-dpme.gov.za
Ms Thabisile Zuma, Evaluation Officer thabisile@po-dpme.gov.za
Mr Bongani Maluka, Evaluation Officer bongani@po-dpme.gov.za
Ms Ntando Buthelezi, Evaluation Officer ntando.buthelezi@po-dpme.gov.za
Ms Priscilla Mutondi Rambau, Evaluation Officer priscilla@po-dpme.gov.za
Ms Ayanda Ndlhovu, Evaluation Officer ayanda@po-dpme.gov.za
Ms Ahn-Lynn Crouch, Research Officer ahn-lynn@po-dpme.gov.za
Ms Criselda Mashabela, Acting Evaluation Officer  criselda@po-dpme.gov.za
Ms Nkamang Tsotetsi, Assistant Director Administration nkamang@po-dpme.gov.za
Ms Refilwe Masikane, Secretary refilwe.masikane@po-dpme.gov.za
Mr Thabang Kgongoana, Intern  thabang@po-dpme.gov.za

 

Department of Planning, Monitoring and Evaluation

The Union Buildings, East Wing Government Avenue, Pretoria, 0001

Postal Address

Private Bag X944 Pretoria, 0001
 

Contact Numbers

Tel: +27(0)12 312 0000 Tel: +27(0)12 323 8246