Author: Nigel Brunsdon

Reporting is the process of writing up your data and analysis. Reports fall into two broad categories:

Internal – meaning they are for the board, staff, or other stakeholders in your organization. These are often used for strategic planning and program development.

• Annual reports
• Board reports
• Needs assessment reports
• Financial reports

External – meaning they are for funders, partners, public entities, or the general public.

• Grant/donor reports
• Audits

Best Practices for Reports

These are some of the best practices for creating reports cited by harm reduction leaders:

• Make a plan – Make a plan for how you collect the necessary data and when you will do the write up – and stick to it. Note that this may include who you will need to contact for specific information.
• Don’t procrastinate – Do not put off gathering your data and information until just before your report is due – it only makes you miserable.
• Decide on who and what you need – When you begin writing, figure out who you’re writing for (donors/funders/board etc.) and what you need from them (more money/ a healthy relationship/ more support etc.). This will help guide your writing.
• Be brief – Keep brevity in mind when writing and try to edit for length.
• Make a skeleton – Make an outline before you start writing.
• Make a template – To save time with frequent reports, make an outline that you can save as a template for future reports of the same type.
• Illustrate your points with stories – Where possible and appropriate, use anecdotes to illustrate your point. Data can be dry and abstract while stories offer details for people to connect to.
• Read the instructions – This should go without saying, but make sure to do this even when you think you know what they are.
• Use another set of eyes – Have another staff person or colleague you trust read through, review, and edit your writing. For very small programs this is an excellent task for board members.

Effective program evaluation is a systematic way to improve and account for public health actions by involving procedures that are useful, feasible, ethical, and accurate. Essentially, evaluation looks at whether or not a specific intervention was effective from several different perspectives. While monitoring creates raw data, evaluation is the analysis of that data. Evaluation differs from quality control and assurance because it is primarily for reporting to others outside the organization, including board members, donors, and funders; while quality control and assurance is for internal use.

Evaluation often looks at process and outcome goals or measures. These are sometimes confusing but in general:

• Outcome measures – describe a program’s results and determine whether the intended outcomes were achieved. Outcome measures track what happened.
• Process measures – describe a program’s services, activities, policies, and procedures. Process measures look at the processes an organization undertook and the “how and why” of organizational achievements.

You may remember these from the section on strategic planning because evaluation allows you to look back at your strategic planning to see if you met your goals and objectives.

In thinking about evaluation, it is essential for leaders to begin by asking some of the following:

• What will be evaluated? (That is, what is the program and in what context does it exist?)
• Who is the evaluation for?
• What aspects of the program will be considered when judging performance?
• What standards (i.e., type or level of performance) must be reached for the program to be considered successful?
• What evidence will be used to indicate how the program has performed?
• What conclusions regarding program performance are justified by comparing the available evidence to the standards selected?
• How will the lessons learned from the evaluation be used to improve effectiveness?

Quality assurance is the process of creating plans to ensure that your program is meeting the needs and expectations of your participants. Quality assurance is proactive. Activities that fall under “quality assurance” include:

• Needs assessments
• Informal surveys
• Service needs questionnaires

The quality control process is a corrective process within quality assurance. It is the task of actually ensuring that a service meets the quality assurance benchmarks. Examples of this include spot checks and post-service participant evaluations.

Monitoring includes all of the raw data regarding your program. These are your “program stats” and include things like:

• The number of unique individuals seen
• The number of syringes provided
• The number of HIV or HCV tests provided
• The number of outreach shifts provided.

Monitoring can also refer to “disease monitoring”, which is tracking the number of people in a given community who have a certain disease. The main feature that distinguishes monitoring from evaluation is that monitoring does not usually refer to any sort of analysis of data. Monitoring data is frequently used as the basis for evaluation.

There is often confusion about the methods and processes of collecting data and monitoring vs. quality assurance vs. quality control vs. evaluation because all of these processes provide data and information about how well a program is doing.

All of them can be used in reporting but they are not all the same and it is important to distinguish between them for a variety of reasons, not the least of which is because many funders will expect you to distinguish them and they are often the basis for ongoing funding.

It’s important for SSPs to ground any research or data collection they do in the insights and best practices for working with marginalized folks. The following are some of the best practices for collecting data from participants in harm reduction programs based on the experiences of harm reduction leaders:

• Be transparent with participants – Explain to your participants why your program collects data, why you ask for specific data, and how it is related to things like grant reporting that help the program. SSP participants are often suspicious of disclosing their information because they have been harmed by it. Being explicit about your motivations and needs regarding data collection helps build trust with participants.
• Be transparent with staff – If staff do not understand why you collect certain data, they cannot possibly be expected to explain it to participants.
• Be human first – Data collection that comes across as mechanical or rote can alienate people and make programs and interventions less effective.
• Ask about behavior – Because some questions can make folks feel unseen, judged, and defensive, try to minimize any questions that refer to people’s identities and completely avoid those that assume anything about them. Instead, try to ask direct questions about behavior or ask questions in a manner that does not presume how a person may identify.
• Be direct – Asking direct, nonjudgmental questions is always preferable to speaking euphemistically.
• Ask open ended questions – Open-ended questions are questions that are open to interpretation and do not offer a finite response. “Where did you grow up?” is a closed-ended question because there is only one response, but “Can you tell me more about where you grew up?” is an open-ended one because it invites the responder to offer more information. Open-ended questions aren’t always appropriate or desirable, but they often feel less judgmental and presumptive especially when discussing behaviors folks may be sensitive about.
• Don’t assume identity – Asking demographic information can sometimes be awkward because assuming identity can sometimes make people feel unseen and resentful. On the other hand, not assuming an identity that may seem self-evident can also make people feel invisible or make the interaction awkward. Harm reduction leaders report that a middle path, asking if a seemingly obvious identity is correct, is often the best method for collecting this data. This may look like asking “You appear to be (demographic), is that true?” or “I would assume you’re (demographic); is that right?” both of which acknowledge appearance while still leaving room for correction.
• Train staff effectively – Train staff on both why data is being collected and how to do it in the most effective and efficient manner.
• Minimize data collection, especially during service provision – Data collection is necessary, but it can act as an artificial barrier that interferes with social interaction and rapport building, and it can alienate participants. This is deeply problematic given that harm reduction relies on building community and trust to be effective.
• Study up – There are many resources and trainings in the public domain on how to do effective public health program evaluation; please see the resources sections of the Hacks for first stops. These resources can help you build more effective and impactful programs because the best programs rely on peer reviewed evidence.
• Ask for help – One of the primary purposes for technical assistance partners like NASTAD and the National Harm Reduction Coalition is their ability to help guide and support programs with evaluation.
• Do not be afraid to question your funders – If you feel like a particular set of data or data collection devices is too invasive or burdensome for your program or participants, don’t hesitate to ask your funder if it is necessary. If you choose to do this, be ready to discuss your concerns; your funder may not be able to change the data collection requirements (or they may), but the pushback can be beneficial for them to hear regardless of whether or not it leads to immediate change. And as the saying goes, “closed mouths don’t get fed”.
• Incentivize it – Where possible, programs should offer incentives to participants for participating in data collection – especially when that data collection takes significant time (more than five minutes).
• Give cash – When possible, programs should give cash or cash app payments to participants as incentives. Though gift cards are popular with programs, especially those concerned with “encouraging drug use” by giving folks cash, the reality is that participants routinely sell gift cards for cash values below the card’s face value. This means that instead of denying participants cash, all the program has done is guarantee that a third party is benefiting from the effort to micromanage how participants spend the money they have earned.

Readers may be familiar with some or all of the following terms and concepts. They are defined here for clarity in a harm reduction context.

Terms

• Monitoring – the facts and data about your program and participants’ health.
• Quality Assurance – having standards for, and mechanisms to achieve, the best possible services for participants.
• Quality Control – making sure you meet your quality assurance goals with regard to providing services.
• Evaluation – making sure your services meet your goals and objectives.
• Qualitative Data – descriptive and conceptual findings collected by asking open-ended questions of both people and evidence that are then coded and analyzed.
• Quantitative Data – information that can be counted or measured collected by asking closed-ended questions of both people and evidence that are then analyzed.

Methods

• Needs Assessment – an assessment of the gaps between services provided and services needed. Conducting annual needs assessments are an important practice for harm reduction agencies and allows them to react quickly to a changing drug-use environment.
• Focus Groups– entail calling together a small to medium size group to have a facilitated discussion about a topic without leading questions or collecting much, if any, quantitative data. Focus groups are recorded and analyzed for themes and data.
• Surveys – questionnaires that may be self-administered online or in-person, or administered by staff. Surveys can consist of both quantitative and qualitative questions.
• Interviews – qualitative interviews consist of guided interviews asking open-ended questions about a topic(s) without leading questions or collecting quantitative data. Interviews are recorded and analyzed for themes and data.

Coming of age during an era that vilified people who used drugs, harm reduction arose at intersections of lived experience and research. As an untested set of interventions which undermined core existing approaches like abstinence-only drug policies, harm reduction had almost no funding opportunities in the beginning. One exception were research grants that allowed for pilot projects of harm reduction interventions to be tested and allowed some funding to trickle down to folks doing grassroots work. In turn, much of this research was based on the lived experience of people who use drugs, including the lived experience of people who used drugs who also happened to be academic researchers. This research, which time and again proved that compassionate, person-first harm reduction interventions work to reduce costs and harm, acted as a catalyst for further research proving its legitimacy, and more funding despite intense public resistance. In addition, harm reduction began as a means to reduce disease burden, and many of its early proponents were or became public health experts. Moreover, data collection has allowed harm reduction programs and proponents to not only prove that what they are doing is objectively effective, it allows SSPs to interrogate the efficacy of their programs to ensure that they truly “meet people where they are at”.

This background provides the context for, and motivations of, harm reduction organizations to collect and analyze data. In general there are two types of reasons for SSPs to collect data; external and internal ones.

External reasons for collecting data include: legal, contract, and funder obligations. Legal obligations make up a minority of external reasons for data collection, and include some disease surveillance and/or other regulatory public health concerns. The more common external reasons are contract or funder obligations.

Government contracts often require contractees to track certain data, such as number of syringes given and/or collected, tests performed, referrals made, or other services performed. They may also require agencies to track disease, demographic, or other data. Sometimes agencies decide on how they will track this information but, more often, government agencies will provide SSPs with the data collection tools they are required to use. Private funders also often require certain data to be collected and reported, such as number of participants seen and services provided, but they rarely provide specific tools for data collection.

Internal reasons for data collection include internal reporting such as Board and donor reports and, perhaps more importantly, quality control and assurance. Data collection and analysis allows programs to objectively analyze if interventions and the program are working, and to discover what needs to be changed or updated. This is especially important when working with marginalized communities in which circumstances and needs change rapidly, and around whom there is a great deal of stigma and bias. Finally data collection and analysis can also provide a way of empowering participants by giving them a concrete voice in service provision and discovering their unmet needs.

Managers, especially people who may not have previous management experience, may need further assistance to succeed including:

• Development – Developing managers in harm reduction organizations often means training from within. Start by identifying potential leaders and giving them responsibilities.
• Training – Once you have identified potential managers, it is critical that they receive leadership, delegation, and management training. Look for opportunities both in the harm reduction community and also the larger non-profit world. For smaller organizations: look for scholarships and ask for donor support to create the best possible team to serve your community.
• Supporting – As with all employees, managers must continue to be supported in their evolving roles after their initial training through mentorship, continued training, and coaching.

Harm reduction organizations that hire employees of any kind face the same issues as other employers. Some of the biggest concerns regarding having any kind of employees at an SSP include:

• Training – As with SSP volunteers, the minimum training that any frontline SSP worker needs includes:
  • Universal Precautions and site safety
  • De-escalation
  • Program policies and procedures
  • Participant privacy and confidentiality

  Basic training for SSP workers should also include:

  • Harm Reduction
  • Outreach
  • Behavior Change Models
  • Cultural Competency for SSP Participants
  • Safer Drug Use
  • Safer Sex Work
  • Trauma

• Supporting – In addition to training, it’s equally important for employers to support employee’s sustainability and growth through continuing education and creating a supportive workplace that encourages employees personal and professional growth.
• Taxation, Identity, and Privacy – Obviously it is essential for employers to have accurate information about employees and pay appropriate taxes. It is equally essential, and the law, that employers follow employee privacy laws and regulations regarding this information.
• Discrimination and Harassment – SSPs, like every employer, are barred from discrimination, creating a hostile workplace, and harassment.
• Protected Leave – All employees are entitled to protected leave including family leave and sick time.
• Compensation and Overtime – SSPs should pay the highest possible wages and benefits they can afford. SSP hourly employees are subject to overtime laws.
• Safety – SSPs are subject to federal Occupational Safety and Health Administration (OSHA) standards with regard to bio-waste handling.
• State and Local Law – States and localities impose other employee laws and regulations; please check with your state’s employee development department and city government for more information.