A few weeks ago, Rise had the opportunity to co-present on developmental evaluation at the Minnesota Council of Nonprofits' Annual Conference. With the Twin Cities Medical Society (TCMS), we discussed how Rise used developmental evaluation to help TCMS build an innovative advocacy training program for med students. It was a great chance to talk about what developmental evaluation is (and isn't) and how nonprofits can use it as a tool to develop new approaches to complex problems. Check out some of the most commons Q and A's about this approach below.
Q: What IS developmental evaluation anyway?
A: Developmental evaluation is a type of evaluation that helps social innovators develop solutions to complex and dynamic problems. It applies the thinking and methods of evaluation to problems that have multiple and overlapping causes (like poverty), where the environment is constantly shifting (like the start of the pandemic), or where there are many different stakeholders with conflicting views (like pretty much any social system).
Q: How does developmental evaluation differ from other types of evaluation?
A: Traditional forms of evaluation help us understand programs that are already well-defined, meaning that activities and outcomes are known in advance. Traditional evaluations typically happen after a program has been implemented. Developmental evaluation is different because it happens as an initiative is developing. Activities and outcomes aren't necessarily known in advance. While the methods are similar (surveys, interviews, quantitative data analysis), developmental evaluation relies on rapid-cycle data collection and analysis to help innovators learn about their initiatives - and adapt them - in real time. It also prioritizes systems-level thinking over program-level thinking, which means thinking about relationships between and across settings rather than discrete program components.
Q: Is developmental evaluation approach for my program, initiative, or organization?
A: Well, it depends. Developmental evaluation works best in situations that involve complex or changing environments, where you want to build or develop a new approach (i.e. innovate). Highly structured programs with strict accountability measures are not a great fit. For this approach to be successful, program staff and organizational leadership need to be comfortable with ambiguity and willing to adapt program components and methods in response to new information.
Q: How does developmental evaluation differ from our continuous improvement efforts?
A: Like continuous improvement, developmental evaluation involves ongoing data collection and analysis to provide information on a program or initiative. With continuous improvement, the focus is on the program itself. Data inform how the program is working - is it reaching the intended audience, achieving expected outcomes, etc. With developmental evaluation, the focus is on how the system is changing as an initiative develops. Data are used to inform key elements of a initiative and may lead to large changes in the activities and expected outcomes.
Q. How do I convince leadership to use developmental evaluation?
A. Supportive leaders (and funders!) are critical to the success of developmental evaluation. At Rise, we've found that it helps to identify the limits of traditional types of evaluation - which are great for understanding programs that are well-developed but less great at changing systems to enable big change.
Want to learn more about whether developmental evaluation is right for your organization? Let's connect!
Across Minnesota, counties collect data in the normal course of administering human service programs. When someone calls the county in need of food assistance, for example, a county worker will record information about their family, job, income, and assets. This information is used to determine eligibility for programs like MFIP (cash welfare) and SNAP (food support), monitor program participation, and provide referrals to community organizations.
Few MN counties use this data for program improvement.
I'll admit, I was shocked when I learned this. And not just because I love data! Not using existing data is a lost opportunity – even simple analyses can help programs better understand the families that they serve, their needs, and whether these needs are being met.
So, why aren’t counties using the data they already collect for program improvement? There are actually a few reasons:
Admittedly, some of these problems are easier than others to solve. And to be clear, there are still major limitations of existing datasets - namely, that they are embedded in systems of structural racism and can perpetuate disparate treatment and outcomes by race and ethnicity.
But even recognizing these limitations, there is still much that can be accomplished if counties can better understand program participants, activities, and outcomes. Using existing data - with necessary caveats - can enhance efficiencies and minimize the additional burden placed on program participants.
So what can we do?
Want to know more about using data for program improvement? We'd love to chat! Contact us to learn more.
You can’t get very far these days without hearing about the need for systems change. The term is used to describe everything from transforming public assistance to reforming democracy. But what exactly does it mean?
In a nutshell, systems change is an approach to solving complex social problems.
At Rise, many of our partners are working to solve problems that don't have an easy solution. These so-called ‘wicked problems’ are distinguished by their complexity: they are hard to define, are intertwined with other problems, and are subject to varying interpretations by different stakeholders. Poverty, for example, is the product of many interconnected problems including the lack of educational opportunity, declining real wages, rising housing costs, and systemic racism.
Complex problems are tricky to solve at a program level. A program that provides housing assistance doesn't address educational inequities, low wages, or institutional racism. Complex problems need solutions that change systems, in addition to changing programs.
And that's what systems change initiatives do. They aim to change the underlying structures and practices that make a system or set of systems function in a certain way.
That sounds like a bunch of jargon, I know! Let me give you a real-life example:
For several years the Rise team has been working to reduce ethnic and racial disparities in Minnesota’s juvenile justice system. This is a complex problem with multiple, overlapping causes. Programs certainly matter for addressing disparities. Implicit bias trainings, for instance, are important for changing the behavior of front-line staff. But systems-level interventions are also critical.
That’s why Rise is working to change how the system itself perpetuates ethnic and racial disparities, by:
Working to change systems requires the ability to work across programs, agencies, and stakeholders and to bring different groups of people together in solving complex problems – skills we actively work to cultivate at Rise. If you want to connect with us to talk about how we can help your organization work at a systems level to tackle complex problems, email me at firstname.lastname@example.org.