Our programs often make us feel good – but are we actually helping our clients? 

Amanda Woomer joins us today to discuss the much (but needlessly) feared monitoring and evaluation. Join us as we explore how M&E can help you advance your mission, impress donors, and ultimately make sure that you are actually doing the good you’re setting out to do.

Listen to the Episode Here!

Links

Amanda’s Instagram Page

Amanda’s LinkedIn

Timestamps

 (2:13) The value of M&E 

(4:42) Why you shouldn’t feel threatened by M&E (though we understand why you might)

(6:53) Building M&E into your program from the start

(12:22) How to use your data

(18:47) Being transparent with your results

Transcript

Dolph Goldenburg (0s):
Welcome to the Successful Nonprofits® Podcast. I’m your host Dolph Goldenburg. ​Today, we are going to be having a conversation with Amanda Woomer about measurement and evaluation (M&E). As I look back on the 200+ episodes we have done over the last four or so years, I am struck at how little attention we have paid to measurement and evaluation. I think it was Episode 3 and I think we recently recorded another episode on it. So up to this point, we have short sheeted measurement and outcome evaluation. 

Dolph Goldenburg (47s):
I’m really trying to make up for lost time. I’m so excited that we were able to get Amanda Woomer to come onto the podcast and discuss this with us. If we’re not thinking about how we measure what we do or evaluating our impact, we don’t have a real sense of whether or not we’re achieving what we think we’re achieving. All we really know, if we’re not engaging in M&E, is if what we’re doing is making us feel good. But we don’t know if it’s actually moving our mission forward. 

Dolph Goldenburg (1m 33s):
That’s why we’ve invited Amanda to join us today. She works full time in measurement and evaluation and on the side she does consulting in the conservation and peace-building space around measurement and evaluation. The other thing that I think she’s going to be phenomenal for as we move through the episode today is providing actionable tips for smaller nonprofits because she understands that not every nonprofit has an evaluation department. Hey Amanda, welcome to the podcast. 

Amanda Woomer (2m 11s): 

Thanks. I’m so excited to be here. 

Dolph Goldenburg (2m 13s):
My first question is: Let’s say, I’m in a nonprofit and I think that evaluation is really important and I think we should be doing it. How do I convince others? Whether that’s my boss or the board or someone else, how do I convince others that it’s important? 

Amanda Woomer (2m 29s):
Sure. I think an easy place to start is about funding. A lot of funders are going to ask you for different outputs of M&E or Key Performance Indicators (KPIs) that they’ll want to see if they’re going to give you money and after they give you money. Being able to say, “We have some of those indicators or KPIs” or “We have a system” is going to help you to get more funding and to keep your funders happy. The real reason though, like you said in the introduction, is that we know if what we’re doing is the right thing to do and that we’re actually achieving our objectives. 

Amanda Woomer (3m 17s):
I think a lot of times in different nonprofit work we assume we’re doing this cool or good thing. We know what we’re doing. So, of course we’re going to have the outputs that we want to see. However, that’s not always the case. And if we don’t have an M&E system in place, even a basic one, we’re missing opportunities to learn. We’re missing opportunities to have real impact. I think the third thing is M&E is scary. It’s often looked at as audits or other types of controls. What it is is a system for helping you know what you should do, if you’re doing it well and what you can learn and change from that. 

Amanda Woomer (4m 8s):
It’s not about being told that you’re wrong or that something’s not set up right or you shouldn’t get any more funding. It also doesn’t necessarily require a lot of money to get started. I think because of these ideas around what M&E is or what it isn’t can make it seem really daunting. And you think, “Well, I need an M&E person” or “I need this really expensive data collection system or data analysis” or “I don’t have the money for this”. But really you can start from where you are. 

Dolph Goldenburg (4m 42s):
I agree with you. I think for a lot of people in the nonprofit sector, the idea of M&E can be really scary. I also think that at times it feels threatening because you think, “What happens if people discover that this program is not effective?” You might be fearful that people will think you’re not competent or your program might be eliminated if the organization falls on hard times. So, I absolutely see why people could be threatened by it. And how do we help them not be threatened when we start these conversations? 

Amanda Woomer (5m 26s):
That’s a really good question. It definitely does come off as threatening. Even the teams that I’ve worked with as an internal M&E person, I still am sometimes seen as an outsider or someone that’s a little scary to work with. I think once we own M&E and we make it our own and really bring it into what we’re doing from start to finish that we start to see the value of it. Because it’s not waiting until the end to make a judgment. It’s from the design of our programs. It’s figuring out what the best design is. Collecting data throughout that gives us the information we need to know if things are going well and to make changes. It should be a system that’s traveling through the whole program cycle with you to serve you and your intended beneficiaries. 

Amanda Woomer (6m 31s): 

So, shifting our mindset about what it actually is a good place to start. Starting to develop that system, being very focused on the program team and the people that we’re trying to serve and not necessarily on the donors that we’re scared of losing or getting or keeping. 

Dolph Goldenburg (6m 53s):
I love the fact that you talked about the importance of starting your programs thinking about M&E and not waiting until you’re a year or more in. I think it was Anne May Chang that we had on the podcast a couple of years ago and she talks about the importance of creating a minimally viable product. You can run experiments and you can run tests and you can see what does and does not work or what happens if you change this or that. And doing it in the really early stage so that it’s not, “Oh my gosh, I spent three years on this and I failed” but instead “Oh, great. I learned this doesn’t work and now I can move forward with something else.” 

Amanda Woomer (7m 33s):
Exactly. I’ve worked with a lot of teams from the start to map out the logic of their programs and I find, especially if I’m working with a diverse team who has different technical experiences or comes from a different place, that everyone has made an assumption at some point that isn’t necessarily shared. Everyone has an idea of what the objective is and that sometimes differs between everyone in the program team. So, when you start with M&E from the beginning to help design your project or your program, you’re getting everyone on the same page. 

Amanda Woomer (8m 16s):
Another thing I do is something called constraints or problem analysis. Let’s say you’re trying to end homelessness in your area. That’s an awesome objective. And you might say, “Let’s start working on different activities that can help end homelessness in Atlanta.” A lot of people don’t start by analyzing the problem statement deeper. They don’t start by asking, “Why is there homelessness?” What are the root causes of homelessness in my city? But when we start from there, we get a better idea of what activities are going to have the impact that we seek and are appropriate to the situation. 

Amanda Woomer (9m 2s):
And then we follow that logic through to ending homelessness. Everyone gets on the same page. And at this point you can identify what data you need to know if this is going well. Sometimes we think, “Oh, I have to collect indicator data for these 50 things and talk to a hundred thousand people.” We get really overwhelmed. When at that stage, the question to ask is: what information do I need to know to see whether or not I’m doing is what I set out to do and whether or not it’s having the effect that I wanted it to have. Oftentimes that’s something really simple – having a conversation with someone or sending out a survey with a couple of questions. There are a lot of different options. Stop thinking, “What’s the gold standard of M&E?” and ask instead “What do I actually just need to know?” 

Dolph Goldenburg (9m 58s):
It almost sounds like the limping-cow phenomena. I read about this five or six years ago. Dairy farmers are having a real issue because cows were getting a high number of infections. Regardless of what the purpose of the cow was, cows with infections are much less likely to be productive for the farmer. And so they kept asking, “What variables impact this?” They would tweak different things and the infection rates were just not going down. 

Dolph Goldenburg (10m 38s):
At some point they said, “What are the things that we notice are happening at the same time?” And one of the things they found was that many of the cows that had infections also were starting to limp. And so they said, “Alright, what variables can we tweak that will keep the cows from limping?” And guess what? Once they kept cows from limping, they actually dramatically reduced infections. And so it’s sort of known as the limping-cow phenomenon.

Amanda Woomer (11m 15s):
That’s a great example. And one I’ve never heard before. It also makes me think about a couple of things in M&E. A lot of times we’re working with people and, unlike cows, we can talk to people and ask them, “What else is happening in your life? What else is going on?” I think sometimes we all get a little overanxious to help and we forget to talk to people about what they need. One of the organizations I work with works on housing. Sometimes we’re very eager to say, “You need a new roof” or “You need a new foundation.” When we should be asking people what they need or what the most important thing is to them. That makes our projects more sustainable in the long term. It also gives us a lot more information. I think that’s one place where the M&E mindset helps us – by asking people questions to get us to the design that we’re so eager to jump into. 

Dolph Goldenburg (12m 22s):
So let’s say that you’ve been collecting your data and now you’re ready to sit down and crunch your data. And you’ve got some good findings, whether that’s things we need to change or that things are going well. What should organizations be doing with that data? 

Amanda Woomer (12m 42s):
That’s a good question. It depends on the data that you have. But if we’re following the principles of being transparent, which I know can be challenging sometimes, one thing we should be doing with that data is sharing it back to the people we collected it from. Telling them, “Hey, you’re part of this group. This is what we found about the impacts of our work on your group.” Sharing it with donors is obviously something we’re all pretty used to doing. But share it in ways that make sense and that highlights key findings and key pieces of what you’re doing. 

Amanda Woomer (13m 22s):
There are a lot of people out there who talk about the fact that very long, 20+ page reports with findings don’t get read. And if you’re thinking about the audience that you want to share with, think really concretely about how to share that in a way that makes sense for them. It’s almost never a very long report. Maybe it’s a PowerPoint. Maybe it’s an infographic. A lot of these things you can do really easily with Excel. Some basic analysis can be done there and creating some graphs and charts that you can share with people in a visual way to give them an idea of the information you’ve collected. 

Dolph Goldenburg (14m 10s):
I have to give a shout out to Excel as well. It is actually a really powerful tool and I’m a little bit of a pivot table nerd. So when I’m working with a spreadsheet with a lot of data in it, to me, there’s nothing like doing some pivot tables. It will save me four hours a day. 

Amanda Woomer (14m 28s):
Yeah. I’m a fan as well. We all don’t have the monetary and time resources to learn more complex data analysis. But we also don’t always have to be collecting quantitative data or numbers for everything. People think, “Oh, the gold standard for M&E. I need to be able to run regressions” and all of these things. And we often don’t have time for that. You can also collect qualitative data by talking to the people we’re trying to serve and getting a feeling of how things are going. 

Amanda Woomer (15m 11s):
To go back to my example of homelessness, talking to people and just saying, “How are things going? Is this service I’m providing meeting your needs? What could I do better? What should I do?” These are ways to get qualitative data that you don’t need to analyze in a very time-consuming intense process, but that’s going to give you a lot of rapid feedback that lets you know maybe you do need to tweak the program now and not when you get to the midpoint. 

Dolph Goldenburg (15m 41s):
Absolutely. I know we had started to touch on how you disseminate your data. What are some of the best practices you’ve seen for sharing data with funders? 

Amanda Woomer (15m 51s): 

I think the biggest thing is to share things in a way that is accessible to funders. Again people don’t read long reports. Maybe you do have the long report because it’s required, but you can also come up with very visual and short ways to share findings. There are a ton of examples. If you just Google “one-page reports” or you can look at YouTube videos on putting together data placemats that have a visual representation. 

Amanda Woomer (16m 35s):
That’s a better way to engage funders than to have a report. You could also share quotes, as well. Give that to a funder as part of a discussion about, “Hey, this is the data we’ve collected. Here’s what we found. We think that this means we need to change the group that we’re focusing on.” Or whatever the case may be. That promotes a conversation versus checking the I-have-submitted-the-report-I-promised-you box. Make it readable, make it short, and make it as visual as possible to start a conversation. 

Dolph Goldenburg (17m 23s):
I also ask funders to schedule some time after I email them a report to sit down and walk through the report together. Because (A) they may not actually look the report, even if it’s just three pages long, and B) I also want to give them the insight behind those numbers. 

Amanda Woomer (17m 52s):
Yeah, that is so important. Depending on who your funder is, you may get insights from them about how you might change your program based on the data that you collect. But, they also have a better opportunity to learn from you if you’re having a conversation. They will miss a lot of that rich detail and background if they just look at the report alone. I think a conversation with a funder is a really good place to talk about your M&E and how you’re using it to learn. I think most funders will be happy that their money is going towards something that they know will be more effective than if you just waited until the end and did a fancy evaluation. 

Dolph Goldenburg (18m 47s):
It’s funny that you say that. I was the interim executive director somewhere a few years ago, and the fundraising office sent me an annual report that we were needing to give to a funder within 48hrs. And I read it and I had to go back to the fundraising office and say, “I’m sorry, this is just fluff. We can’t send this out.” And they said, “What do you mean?” I was like, “Well, we need to be really honest about both where we excelled and also where we didn’t and where we had challenges and why.” And then I pointed them to the Berkshire Hathaway annual reports, which are written by Warren Buffett. I don’t know if you’ve ever read one of those. They are a phenomenally good read. 

Dolph Goldenburg (19m 28s):
And I actually think everyone in the nonprofit sector should read his annual report. In part, because there are some legal requirements as a corporation when you draft your annual report and you’re a publicly traded corporation. He meets all of those legal requirements, but he also writes his letter to the shareholders in a very folksy way. It’s not short – like 25 or 30 pages. But he was very upfront about where he made good decisions that year and where he made bad decisions. For example, he will really say “Three years ago, when we bought XYZ business, we really thought that it was going to do this for Berkshire Hathaway and were we ever wrong. But that’s okay because we learned something from it. Here’s what we learned from it.” It’s so refreshing to read because so often we all want to pretend like we’re just going from success to success and ignore the fact that sometimes we don’t succeed. But we learn from it and people have mad respect when we own that. 

Amanda Woomer (20m 42s):
That’s so true. I’m thinking now of the founder of water.org and one of his principles has always been to be transparent about what’s working and what’s not. And that is really scary. I’ve worked for a lot of different nonprofits that tried to sweep the negative findings under the rug or couch them in the report in a different way or gloss over them as quickly as possible. I think it’s really important to own what’s working and what’s not. And the only way you’re going to know what that really is if you have some kind of an M&E system in place. 

Amanda Woomer (21m 22s):
It is scary to go to a donor and say, “Hey, we started the program this way. We thought it was going to work. We’ve found out it’s not and we need to change it.” But I think they’re going to respect that a lot more. If you can point to the data that’s shown you that, and you can point to these learning processes you have in place, you can say to them, “We now know we’re going to do a better job with your money.” I think that’s a trend you’ll see in nonprofits, that requirement for transparency and honesty. 

Dolph Goldenburg (21m 55s):
I’d say my experience has been that funders are so used to nonprofits not being transparent, that they are super appreciative when you are. And they are far more likely to trust you when you take risks if they know that you’re someone who’s just going to tell them the truth. 

Amanda Woomer (22m 10s): 

Yeah. I would agree with that. And I think this all starts again with not seeing M&E as a funder-required piece of what you’re going to do at the mid-point or at the end, but you’re starting with it from the very beginning. Not only will you have a better program when you think through the logic of it initially, but you will have an M&E system built in place that’s there for your own learning to then share that with funders and other people working in the same space as you. 

Dolph Goldenburg (22m 41s):
Absolutely. Well, Amanda, I have been saving up a great off-the-map question for you. I Googled you and I found out that you’re a pretty prolific artist in multiple mediums. 

Amanda Woomer (23m 3s):
I do some art, yes. I do some watercolor painting on the side in my spare time, which is not always abundant. And I also do a lot of landscape photography. Nature is where I recharge my batteries. So that’s been a fun outlet for me over time. It’s the place where I get in the zone. 

Dolph Goldenburg (23m 27s):
Listeners, I have to share with you, Amanda is being very, very modest when she says, “I do some.” We’re going to link to her artist’s Instagram page, if she’s okay with that because you’re going to see, it’s not some. She’s just really prolific and impressive and there’s some really amazing art. 

Amanda Woomer (23m 50s): 

Thank you. I really appreciate that. 

Dolph Goldenburg (23m 53s):
I read somewhere that you put your Ph.D. studies on hold for a little while so you could pursue your art and then you returned and finished your PhD. Is that accurate? 

Amanda Woomer (24m 5s):
I did. I had a hard time with my Ph.D. I was very focused on doing work with an organization rather than getting a PhD just to have a PhD. And I was having a really hard time finding a partner. So to take the pressure off, I switched focus for a bit and I found working on art and photography to be really grounding. And it worked out in the end. I found a great partner to do my PhD with and everything happened as it should have. 

Dolph Goldenburg (24m 39s):
We’re just going to continue this off-the-map a little bit, who was the partner? 

Amanda Woomer (24m 42s): 

I worked with Conservation International. They have a division of the organization that is looking at peacebuilding and conflict sensitivity and environmental conservation. Maybe because I’m an M&E nerd, I saw that M&E would be a great way to ensure that they were being conflict-sensitive in their work. And we had an awesome partnership that resulted in a great guide for conservation practitioners. 

Dolph Goldenburg (25m 11s):
Awesome. And again, I did a little research on you, so I think your master’s degrees thesis was also in partnership.

Amanda Woomer (25m 21s):
I worked with a Buddhist center in Atlanta to study how Americans were taking and making their own Tibetan Buddhism. The center is Drepung Loseling. It’s the North American seat of the Dalai Lama, and it’s also affiliated with Emory University. So really, really interesting. Not connected to what I do now, but I think it gives me a good perspective. 

Dolph Goldenburg (25m 54s):
Amanda, thank you so much for joining us today. I am grateful that you would share some really important information with our Listeners, not just about measurement and evaluation, but also why it’s important and why this is something they should embrace and they should be transparent about with their constituencies. So, thank you so much for coming on. Listeners, I want to make sure that you know how to reach out to Amanda – we will share the link to her LinkedIn in our show notes.

Dolph Goldenburg (26m 34s):
Also, Amanda is offering any organization that reaches out to her through LinkedIn a 30-minute consultation on monitoring and evaluation, on conflict sensitivity, or on facilitation needs. So Amanda, thank you for joining us, and thank you for that super generous offer. 

Amanda Woomer (27m 15s): 

It was great to be here. 

Dolph Goldenburg (27m 18s): 

If you were just super busy Googling the Atlanta seat of the Dalai Lama, then don’t worry. Go to our show notes at successfulnonprofits.com and we will have a link to Amanda’s LinkedIn page right there. Before we wind down the podcast today, I just want to read a quick listener review. 

Chad wrote, “This is the first podcast I have ever subscribed to, and I am glad I did. Thoughtful guests, extremely prepared hosts, and topics that inspire action.” Chad, I just have to say, I am truly honored and privileged that the first podcast you had subscribed to would be ours. Welcome to the podcasting world because there’s a plethora of great stuff out there for you. Listeners, if you’ve not already written a review, I’m going to ask you to do that today. It is one of the best ways that you can help other people find our podcast. 

Dolph Goldenburg (27m 58s):
And if you’re really feeling motivated, maybe even mention it to some of your friends that are also in the nonprofit sector. That, Listeners, is our show for the week. I hope you have gained some insight to help your non-profit thrive in a competitive environment. 

Dolph Goldenburg (28m 10s)
I am not an accountant nor an attorney and neither I nor the Goldenburg Group provide tax, legal, or accounting advice. This show is intended for informational purposes only and is not intended to provide tax, legal, or accounting advice. And while I always hate to sound snarky about this, we all should already know this, but if you do think that’s the kind of advice you need, trust me, it is best for you to find a licensed, competent professional and get the advice from them. If you’re not sure who to talk to reach out to me, because I know a lot of folks and I might be able to hook you up with someone who could actually help you out. 

**  We have edited this transcript because how you listen is not how you read. If you have a problem with this, remember you got this for free!