Q and A's: Developmental Evaluation

A few weeks ago, Rise had the opportunity to co-present on developmental evaluation at the Minnesota Council of Nonprofits' Annual Conference. With the Twin Cities Medical Society (TCMS), we discussed how Rise used developmental evaluation to help TCMS build an innovative advocacy training program for med students. It was a great chance to talk about what developmental evaluation is (and isn't) and how nonprofits can use it as a tool to develop new approaches to complex problems. Check out some of the most commons Q and A's about this approach below.

Q: What IS developmental evaluation anyway?
A: Developmental evaluation is a type of evaluation that helps social innovators develop solutions to complex and dynamic problems. It applies the thinking and methods of evaluation to problems that have multiple and overlapping causes (like poverty), where the environment is constantly shifting (like the start of the pandemic), or where there are many different stakeholders with conflicting views (like pretty much any social system).

Q: How does developmental evaluation differ from other types of evaluation?
A: Traditional forms of evaluation help us understand programs that are already well-defined, meaning that activities and  outcomes are known in advance. Traditional evaluations typically happen after a program has been implemented.  Developmental evaluation is different because it happens as an initiative is developing. Activities and outcomes aren't necessarily known in advance. While the methods are similar (surveys, interviews, quantitative data analysis), developmental evaluation relies on rapid-cycle data collection and analysis to help innovators learn about their initiatives - and adapt them - in real time. It also prioritizes systems-level thinking over program-level thinking, which means thinking about relationships between and across settings rather than discrete program components.

Q: Is developmental evaluation approach for my program, initiative, or organization?
A: Well, it depends. Developmental evaluation works best in situations that involve complex or changing environments, where you want to build or develop a new approach (i.e. innovate). Highly structured programs with strict accountability measures are not a great fit. For this approach to be successful, program staff and organizational leadership need to be comfortable with ambiguity and willing to adapt program components and methods in response to new information.

Q: How does developmental evaluation differ from our continuous improvement efforts?
A: Like continuous improvement, developmental evaluation involves ongoing data collection and analysis to provide information on a program or initiative. With continuous improvement, the focus is on the program itself. Data inform how the program is working - is it reaching the intended audience, achieving expected outcomes, etc. With developmental evaluation, the focus is on how the system is changing as an initiative develops. Data are used to inform key elements of a  initiative and may lead to large changes in the activities and expected outcomes.

Q. How do I convince leadership to use developmental evaluation?
A. Supportive leaders (and funders!) are critical to the success of developmental evaluation. At Rise, we've found that it helps to identify the limits of traditional types of evaluation - which are great for understanding programs that are well-developed but less great at changing systems to enable big change.

Want to learn more about whether developmental evaluation is right for your organization? Let's connect!