Connect
Perspectives
Other

Digging Deeper: Insights from Our Evaluation of Community-Centered Health

August 29, 2022
By: Sarah Smith, Director, Learning and Evaluation
A Yearning for Learning

In my free time, you can find me immersed in YouTube videos of evaluation consultant, Michael Quinn Patton, or using Emergent Learning tools to plot my reality game show win. Safe to say, I am a learning and evaluation super fan, but I didn’t become a superfan overnight.

Ten years ago, if you said evaluation, I would have conjured up images of straight-lined logic models and sharp-edged boxes made for compliance. Uninspiring to say the least. However, having been part of the development of learning and evaluation within two foundations, the images that come to mind today are much more abstract with curvy lines and interconnected shapes and elicit a stronger emotional response.

I am energized by the continued momentum to reimagine evaluative practices in philanthropy, in particular to center equity and think more broadly about what it takes to create social change. 

I am energized by the continued momentum to reimagine evaluative practices in philanthropy, in particular to center equity and think more broadly about what it takes to create social change. I am also challenged by the continued work we need to do as individuals, as a sector, and as a society to create the conditions necessary for evaluation to be in service of these ambitions. This momentum and energy, as well as all of the challenges, keeps me coming back for more.

The role of learning and evaluation at foundations is dynamic (read more on this topic here). The day-to-day of learning and evaluation at the Blue Cross NC Foundation involves a variety of work – from facilitating relationships with external evaluations to leading internal strategic learning efforts to clarify goals, assumptions, and generate insights. While this requires close attention to methods and data, I have found it is just as important to focus on change management, relationship building, and dialogue.

A key aspect of our learning and evaluation function involves facilitating collaboration between our program team, grantees, and external partners to design, implement, and disseminate evaluations of our work. This allows us to identify and explore questions to help move strategies forward and generate insights and lessons learned. And while this provides us the opportunity to take a good look at ourselves in the mirror, it also provides an opportunity to shine a light for others to see. For us, sharing what we are learning is just as important to what it is we are discovering. Which brings us to why we are here today.

Learning and Evaluation at Work. Lessons from a Placed Based Approach.

Nearly a decade ago, we launched Community-Centered Health to increase the capacity of community-based organizations and health care entities to act on the root causes of health and health inequities. It is a multi-dimensional approach including investments in diverse multi-sector partnerships that amplify community voice in setting local health priorities. Community-Centered Health has helped the Foundation focus more explicitly on racial equity, understand the value of power shifts, elevate the perspectives of residents and community-organizations, and become a more thoughtful, community-centric investor in supporting health equity. While a formal evaluation was not built in at the start of the initiative, we saw a valuable opportunity to pause and to gain a deeper understanding of how it was working and, ultimately, utilize the insights gained to be more flexible and effective in our strategy moving forward.

To accomplish this, our Foundation, with grant support from the Robert Wood Johnson Foundation, partnered with Engage R+D to conduct a retrospective evaluation of the Foundation’s three initial Community-Centered Health partnerships. While we leverage different types of evaluation, we have found retrospective evaluations valuable to unearth more nuanced insights and lessons learned that emerge when you look at something over a longer period. The evaluation examined early progress and enduring outcomes through interviews with each community’s grantees, partners, and technical assistance providers; perspectives from Blue Cross NC Foundation staff, other funders, and researchers involved in similar community-driven initiatives; and a review of initiative-related documents, as well as secondary population-level data.

I won’t summarize all the findings here. For that, I will gladly point you to the full evaluation report. Instead, I wanted to take this opportunity to elevate some important insights and correlations related to our overarching approach to learning and evaluation that resulted from this undertaking.

The social change we are hoping to influence is long-term, and the pathways to get there are complex and messy. Recognizing this, the evaluation of Community-Centered Health explores many interim factors related to its partnerships’ structure, values and practices, and progress such as increasing community residents’ engagement.

  • Social change takes time and is nonlinear- evaluation must reflect this
    Community-Centered Health is focused on social change – or changing the ways people and organizations interact to transform cultural and social institutions over time. The social change we are hoping to influence is long-term, and the pathways to get there are complex and messy. Recognizing this, the evaluation of Community-Centered Health explores many interim factors related to its partnerships’ structure, values and practices, and progress such as increasing community residents’ engagement. While there seems to be growing acceptance in the fields of philanthropy and health concerning how interim factors are critical to understand progress toward long-term outcomes, there is an opportunity to consider how these factors can better inform learning and decision making over the long run. As this retrospective evaluation demonstrates, our learning and evaluation function can help our organization and its partners better understand progress by focusing both on the long-term outcomes we are aiming to see as well as more interim factors along the pathways to change.

 

  • Context, context, and more context- evaluation needs more of it
    The work of the Community-Centered Health partnerships does not take place in a bubble. As with all collaboratives, there are histories and power dynamics between partners. Additionally, the partnerships are all operating within a greater political and social context. This information is elevated as part of the evaluation to not only situate the work in a broader ecosystem, but to also intentionally explore how partnership dynamics, structural and systemic barriers, local and state policy context, and community strengths impact the work over time. All of our organization’s learning and evaluation work is grounded in the increased understanding of the conditions we are working in and how those conditions impact our work and that of our grantees and partners. This provides us with a more nuanced understanding of success and challenges as well as deeper insights into what it takes to affect change.

 

  • Allow for emergence - evaluation needs space for this
    Each Community-Centered Health partnership identified pressing health needs within their specific community and developed the strategy to address them early on. However, as anyone who has planned anything from a relaxing vacation to a complex program knows, a plan represents someone’s best guess of how something will turn out. While the evaluation looked at the progress the partnerships made toward goals and outcomes related to specific health needs, there was also space within the evaluation to focus on what emerged over the course of nearly a decade. For instance, while not an explicit or intentional focus of ours at the onset of the work, acknowledging and addressing structural racism emerged as a critical factor to success. This became clear well before the heightened national focus on this issue the past two years; however, in 2020 many Community- Centered Health partner organizations leveraged windows of opportunities to more vocally address racism as a public health issue. Since the inception of Community Centered Health, the Foundation has also become more intentional about prioritizing racial equity as a critical dimension of supporting health equity. The evaluation elevates this story. Moving forward, it will be important for us to pay attention to how we think change will unfold and how that thinking changes over time based on unanticipated challenges, opportunities, and lessons learned . This outlook aligns with our approach to emergent strategy.

 

  • The process is important – evaluation must embrace this
    Often, all anyone outside of an organization can see is the final outcome or product from an evaluation. The reports are beautifully designed, and lessons learned are nicely summarized. However, much of the value of evaluative work is in the process to get there. From the outset, it was important to acknowledge the time and expertise required of our grantee partners and work toward a goal that would be additive and not duplicative of other work taking place in their communities. As we began planning, we realized we had not included participation in a formal evaluation as an expectation of the funded work. With this understanding, we moved to increase the size of the grant to compensate the communities for their participation in the evaluation activities. We also recognized that the partnerships were involved in several other evaluation efforts supported by various entities, and therefore Engage R+D spent considerable time conducting literature reviews, web searches, and speaking with other researchers, evaluators, and funders in the space to ensure participant time was well spent. We will continue to be mindful of these types of impacts as we grow our evaluation work.

Ultimately, our hope is that this approach to self-interrogation will benefit others in the field of philanthropy as well.

 

Looking Ahead

While we aspire for our learning and evaluation work to lift up the successes and challenges of our partners in the field, as we have discussed here, learning is bi-directional. What we have learned from this retrospective evaluation about ourselves will inform our overall approach, helping us to strengthen our practices to more effectively position learning and evaluation as a tool for social change.  And as we continue to explore different types of evaluation and learning practices, we will prioritize sharing what it is we are learning – this includes what we are discovering about the issues we are working on as well as how we are functioning as an organization. Ultimately, our hope is that this approach to self-interrogation will benefit others in the field of philanthropy as well.

Acknowledgements

I am grateful to our grantees and partners at Collaborative Cottage Grove, Healthier Highland, Mothering Asheville, evaluation partners at Engage R+D, Oktawia P. Wójcik, Director of Learning at the Robert Wood Johnson Foundation, and my colleague, Shelia Reich, Director of Racial Equity and Healthy Communities, for their time, engagement, and contributions to this evaluation.

Support for this evaluation was provided in part by the Robert Wood Johnson Foundation (grant number 77128). The views expressed here do not necessarily reflect the views of their foundation.

About Learning and Evaluation at the Blue Cross NC Foundation              

Our Learning and Evaluation team develops and implements practices and projects to help clarify programmatic and organizational goals and assumptions, assess progress, evolve and adapt strategies based on what we are learning, and share insights with the broader field. We are committed to the practice and principles of the Equitable Evaluation Framework  (believing evaluation and evaluative work should be in service to equity) and center our learning and evaluation work on three core tenets: how we work, what we are working on, and our values and principles. Our organization believes evaluative work should inform learning, decision-making, and action and recognizes that a variety of qualitative and quantitative methods are necessary due to the complexity of the issues and the dynamic environment in which our work takes place.