Q: I hear “research-based” and “data-driven” and don’t know what to believe about the work agencies do. Some agencies say that student workshops are all we need to solve all kinds of health and safety problems, but how do we know what really works when it comes to keeping kids and families safe from harm?

Excellent question. I am often confused to see what passes for “research-based.” It appears that non-profits and for-profits can use any term that sells their products or services. The “fact-checker police” don’t exist yet (though there is software in the works).

The only way to know if a particular product or process really changes health behaviors is to look closely at the evaluation behind it. Even then, analyzing research articles focused on behavior change takes special training. I’ve been doing it for decades and still find it tricky.

In our data leaders course we spend a lot of time on what we call “proving a hypothesis.” Essentially if some company or agency is saying, “If we do X, then Y happens” one needs to look very, very closely at what “X” was. For example, if we want to reduce childhood trauma, what makes the most sense from a data-driven perspective? Will it be a two-hour training for parents? A health class series for fourth grade students? Or might it be the building of a school-based wellness center with a staff of behavioral health care professionals who can now treat the emotional trauma of students and their family members? Data can guide us as we decide what to invest in (not what’s the easiest or cheapest activity to pull off).

Once you’re able to fully describe the intervention (the “X”), you can then begin to research what changes (the “Y”) as a result of it. Look closely and ask: With a training, was there a change in a person’s knowledge, attitudes or behaviors?” Was the change long-term?  Changing knowledge does not automatically mean behavior change. For example, people may learn  about the dangers of drug misuse but that won’t mean substance misuse won’t happen. If this sounds a bit complicated, it sure can be. What’s important is that all those working in the fields of human services, health equity, education and youth development need to get on the same page with what works and what doesn’t.  That means becoming BFFs with data. We can’t afford to be wasting resources on slickly-packaged curricula or websites that may look promising but fail the test when it comes to producing change. We seek change that is measurable and meaningful). Bottom line: we need to disrupt the spending of dollars and staff time on strategies that don’t work. Instead, get the data to get to results.