Campaign Monitoring and Evaluation: Measuring What Matters

Frameworks taught in the Community Organising Fellowship for effective monitoring and evaluation of campaigns, movement building and organising work.

Introduction

Monitoring and evaluating the success and progress of our campaigns and movement building work is a critical part of the action learning cycle: we make plans, we put them into action, we reflect on how it all went, and then we make new plans based on what we learned and prepare to do the process over again with greater knowledge, experience, and hopefully, power.

There’s plenty of reasons why monitoring and evaluating is important work. Through monitoring and evaluating, we can:

• Learn what is and isn’t working, and modify our plans accordingly
• Hold ourselves accountable for building leadership and winning campaign outcomes
• Report and communicate progress and success to our community and stakeholders
• Build our narrative of power and progress

In my experience, monitoring and evaluation work isn’t always effectively prioritised by changemakers. Sometimes all our effort goes into planning the strategy, without planning how to track if the strategy is really working. Sometimes a lack of emphasis on evaluation is part of a bigger problem where we favour reactive action over planning and reflection. It can seem a lot more compelling to dedicate our time to mobilising hundreds of people for an action rather than conducting an evaluation. Sometimes we’re anxious about setting, committing to, and being accountable for hard metrics, especially when it comes to relational-based organising work. Other times we fall into the trap of measuring, valuing, and rewarding the easy stuff of actions and outputs, rather than stuff that really matters like outcomes and changes in community, leaders’, or decision makers’ attitudes.

So how can we equip ourselves to design effective monitoring and evaluation approaches for campaigns and organising work? How do we ensure we track the stuff that really matters? And how do we do this in a way that is beneficial, rather than building another layer of bureaucratic chore?

For starters, it’s important to get clear about measuring outputs and outcomes, transactions and transformations. 

Outputs and outcomes 

For effective monitoring and evaluation, it’s critical to first distinguish between outcomes and outputs.

Outputs are the activities and actions of our campaign or organisation, while outcomes are the end results, consequences, or after-effects of those outputs – what we aim to achieve. Essentially, our outputs are designed to bring about our desired outcome. Effective evaluation will measure whether and how our outcome was achieved. So the first step is identifying and articulating the outcomes we seek.

A User’s Guide to Advocacy Evaluation Planning by the Global Family Research Project (2009), provides a useful framework for designing an advocacy campaign evaluation plan, including what will be measured and how.

The guide’s author, Julia Coffman, outlines four levels of measurement for advocacy campaigns:

1. Activity/Tactic Measures

Activity/tactic measures are metrics of output, which are the easiest evaluation measures to identify and track. They typically count the number of actions taken or tactics performed.

Examples of these measures include:

• Number of events, rallies, trainings, conversations, petitions signed, etc
• Meetings conducted with target decision makers or influencers
• Issue research report or policy agenda published

2. Interim Outcome Measures

Interim outcome measures track progress towards achieving our objectives/outcomes sought. Whereas activity/tactic measures are metrics of output, interim outcome measures are metrics of effect, which demonstrate consequences or changes brought about by our efforts.

Examples of these measures include:

• Policy agenda alignment with allies and stakeholders
• Key decision-makers or influencers demonstrating support for our policy agenda
• Shift in public/community attitudes and willingness to take action on our issue
• Growth in organisational capacity (eg. volunteers, donations, allies, media coverage)

3. Policy Goal Measures

Policy Goal Measures identify whether our policy goals/objectives/outcomes sought have been achieved.

Examples of these measures include:

• Policy commitment from key decision-maker
• Adoption and/or implementation of policy (eg. through legislation, regulation, funding)
• Unwanted policy formally blocked (eg. through disallowance, ballot, judicial review)

4. Impact Measures

Impact measures represent the impact of achieving our policy goals/objectives/outcomes. They show how our efforts have made a difference for our community, through improved services, systems, social and physical conditions.

Examples of these measures include:

• Reduction in annual asthma hospitalisations
• Better educational outcomes for children
• Increased employment in the community
• Greater accessibility and quality of disability support services

It’s important to remember that not all evaluation measures are created equal, so we need to prioritise which measures to focus on. Corporate frameworks, like Objectives and Key Results (OKRs) stipulate that we should have one ambitious policy goal and two or three interim outcome measures that give us strong indications of our progress towards that objective. Which measures give us the best insight into the effectiveness of our campaign strategy? Which measures are most persuasive to our community, organisation, or funders?

Lastly, we need to work out how to effectively collect, capture, and track the data or information required for each measure. Will we use surveys, interviews, polling, media tracking, parliamentary monitoring, case studies, or ballots?

Transactions and Transformations

A question I sometimes hear from the community organisers I work with is: “It’s easy to track advocacy campaigns, but how do we effectively track community organising and leadership development in more qualitative ways?”

Albert Einstein would answer this question in the abstract: “Not everything that counts can be counted, and not everything that can be counted counts.”

But that’s not very helpful, Albert.

It’s true that community organising demands a more nuanced set of metrics, because of the focus on leadership development and relationship building as core outcomes. Indeed, a growth in a leader’s confidence, sense of solidarity, and belonging can be harder to measure and articulate in an evidence-based way. Anyone who’s ever toiled to prepare a grant funding proposal knows this all too well. This is sometimes cited as a reason for underinvestment in work that builds movement power. In recognition of this challenge, Manuel Pastor and others from the University of Southern California’s Program for Environmental and Regional Equity published a 2011 report titled Transactions Transformations Translations: Metrics that Matter for Building, Scaling, and Funding Social Movements. Pastor et al offer a framework to support social changemakers in developing metrics that track important movement building work and bridge the dichotomy between quantitative and qualitative measures, or what the authors call transactions and transformations. These indices are key to healthy and effective movements capable of effecting big change.

Transactions represent the quantifiable measures that can be easier to track because they are more tangible. Transformations are more qualitative and measure the changes in individual and collective experience, relationships, knowledge, attitudes, and skills that develop through movement building work. It’s important to note the interrelation between transactional and transformational metrics. The two are not mutually exclusive, and should be used in combination.

In a community organising context, transactional activity/tactic measures might include the number of one-on-one meetings or training workshops, with the interim outcome measure being the number of new volunteers or leaders in your community. These classic number-based metrics give a snapshot of your organising reach. Transformational measures might include the shift in leaders’ attitudes from self-interest to understanding and prioritising collective power in their community, or their growth in empowerment, ownership, and belonging. 

In an electoral context, transactions might look like the numbers of youth voter enrollment, media hits, and votes shifted, while transformational measurements might gauge the shift in public consciousness, narrative, and power to see whether the campaign has created the conditions for further change.

The strength and power of a coalition or alliance can be measured transactionally by its size, make-up, and shared agenda, while transformational measures would focus on determining the levels of solidarity, unity, and trust built through connections, communications, and conflict resolution. These indices go directly to the coalition’s ability to outlive a single campaign, issue, or moment and lead to transformational, long-term movement power.

Our monitoring and evaluation metrics should always include a mix of both transactions and transformations, so we can get both the snapshot of activity, progress, and the deeper insight into how we are building movement power. There’s no question that transformational metrics are harder to measure, and require more resourced and nuanced forms of measurement such as polling, surveying, case studies, and interviews. Yet while conducting such evaluation may be a more demanding task, it will offer us valuable insight into our ability to achieve purpose.

Conclusion

Monitoring and evaluation is a critical component of successful social change and movement building work. This article offers a couple of frameworks to support our evaluation planning, with a focus on identifying what to measure and how. It is worth noting that no tool or framework is a magic formula or universal solution. There are some beneficial elements to each framework, and they’re most effective when used in combination.

Community organising and advocacy campaigns will always evolve and change as circumstances and conditions change. As such, it is important that we treat monitoring and evaluation as a live, ongoing, consistent practice. Our campaigns, communities, and organisations will benefit from regularly revisiting our plans and tracking progress towards our goals.

This brings me to the final point: community organising, campaigning, and movement building is a collective activity. We need to co-create, share, and test our monitoring and evaluation plans, measures, and learnings together! So how do you monitor and evaluate your community organising and campaign advocacy work? What are some of the indicators and measures of success you use in your work? Do your metrics include all four levels of campaign advocacy measures and a mix of transactional and transformational measures? Do you have a different approach? I’d love to hear from you!

About the Author

Maxwell Smith is a co-director of the Community Organising Fellowship. He was previously a Clean Air Campaigner at Environmental Justice Australia, a community organiser with the Australian Conservation Foundation, a training facilitator with the Australian Youth Climate Coalition, and an advisor to a federal Senator. Read other articles by Max. 

Explore Further

• 3 Templates for Campaign Evaluation
• 3 tools for advocacy evaluation
• Running Effective Campaign Debriefs
• How are we Measuring People Power in 2020+, and where do we go from here?
• Tip: “You know nothing, campaigners”
• ChangeMakers Organising School – Season Four, 2021 – How Small Groups Make Big Changes – Session 9: How can we measure success? – June 24, 2021