Measurement and ethics

I'm getting lots of requests to speak on ethics, or help with ethics policies, or consult on ethics. Seems to be a bit of a fad, sadly. It certainly is regrettable that Sarbanes, and the Senate Finance Committee are what it takes to motivate people to double check their ethical behavior. Let's think optimistically: Maybe it will be a long term trend, and not just a fad with a short half-life.

My class at Kellogg was, as predicted, very enjoyable. It was about the reasons for and ways to measure performance in nonprofits. It actually ran through this morning; I just did the kickoff session. But at that session, the issue of ethics surfaced.

A participant (we'll call her Lynn) asked a question about proving that what she did (in this case, develop young leaders) was actually having an effect, noting that to prove such a thing might take ten years. We agreed that it might, and that if Lynn could develop such a research project (and fund it), she could tell her other funders and community that she was really working on showing the long term effects of the program.

Then I asked the group; "But what should Lynn do if 5 years into the 10 year project, the preliminary data show that her program doesn't work, that it doesn't have any positive effect on leadership?"

There was a long, uncomfortable silence. You see, everyone believes that their program is wonderful, and worth their life's work. Otherwise, why would they do it for low pay and long hours? Mission motivates. But if the method we use to accomplish mission is shown invalid, what's our responsibility?

All this reminds me of Project DARE, the anti-drug effort that you may well have in your town. The idea was that by having intervention/education in elementary school, drug use would be reduced. DARE has been around for a long time (my 23 year old went through the program) and funds a huge number of police officers (Project DARE Officers) who come into the school twice a week. Kids get lots of drug education, and have a big graduation, take a DARE pledge, etc.

Sounds good? Doesn't work. Absolutely, positively doesn't work. Kids who go through DARE are no more or less likely to use drugs as kids who don't, at any age or stage in their growth. This is not news. We've known this for YEARS, and DARE keeps getting studied and keeps getting shown to have no effect, and we keep spending money on it.

Don't believe me? "It's SUCH a good program!", you say. So did I. Why do parents love this program? I think it's because we can rationalize that we don't have to talk to our kids about drugs (which is hard) since the school is doing it.....but it doesn't work: Here's a quote from a review of research published by the National Association of School Psychologists.

"When we examined use of tobacco, alcohol, marijuana and other illegal drugs at age 20, we found no differences between those participants who received the DARE program and those who did not. Similarly, our analyses revealed no effect of the DARE program on individuals' positive or negative attitudes towards these drugs. Also, there was no difference in individuals' ability to resist peer pressure as a function of having received Project DARE. The only significant effect for the DARE program that we observed was on self-esteem; counter-intuitively, at age 20, participants who received the DARE program in the sixth grade had lower self-esteem scores than participants who did not receive DARE."

Here's a link to the entire article: http://www.nasponline.org/publications/cq314projectdare.html

And we keep throwing money at this loser.

What if this kind of results had been for your program? What do you do? It's a tough, tough issue. How soon do you amend the program, pull the plug, go public? Very dicey stuff.

When you measure outcomes and performance, you almost certainly will get different results than you expect. Sometimes good, sometimes bad. But in every case, you have an ethical responsibility to let people know what you've found, and deal with the data in ways that make your program(s) better for the people you serve.