Currently, EVALCORP is partnering with Pinellas County, Florida to conduct a program evaluation for initiatives funded through the First Responders-Comprehensive Addiction and Recovery Act Grants (FR-CARA) program. This project addresses the County’s goals of reducing opioid-related deaths and connecting individuals to effective treatment.
Evaluating Overdose Response Programming
In our work for this project, we are evaluating two key elements:
- the Quick Response Team (QRT), who conducts community outreach and also receives referrals from first responders in the County to engage with individuals who have experienced an overdose (OD) or are at risk of OD and connect them to resources and/or treatment; and
- the training delivered to both community members and first responders on how to recognize an overdose and administer naloxone, a lifesaving medication that can reverse the effects of an overdose.
During the initial phase of our work, EVALCORP reviewed secondary data, developed a logic model and an evaluation plan, and created a quarterly reporting template to ensure that data are collected and reported in alignment with the Substance Abuse and Mental Health Services Administration (SAMHSA) grant requirements.
In the next phase, we worked with Pinellas County and their partners to create data collection tools including tracking logs, training surveys, and focus group and interview protocols. The data we have collected provides insights about the extent to which the program has provided training for first responders on the use of naloxone, increased community’s access to naloxone, and connected overdose individuals to effective treatment.
Providing A Dynamic Approach to Program Evaluation
Throughout our ongoing evaluation work, we have delivered dynamic, real-time feedback to Pinellas County, helping them adjust strategies to better serve the community. For example, we examined whether training is reaching the right individuals—those who are most likely to respond in the event of an overdose. We also tracked key variables like housing status and insurance coverage to better understand how these factors impact a person’s ability to connect to treatment. It’s exciting to see how our insights have enabled our clients to make data-informed decisions that support their community more effectively.
For this program evaluation, we are not creating a report that will sit on a shelf; we’re actively helping our clients improve their services as the project unfolds.
It has been rewarding to see our clients make data-informed decisions and adapt to local circumstances throughout the duration of the project
For example, a major cornerstone of this project has been evaluating the experience of individuals who have engaged with the QRT. We have measured how their experience has been while engaging with the QRT staff and evaluated what could be improved so that QRT can effectively connect individuals to treatment. However, engaging with participants for this evaluation has posed a challenge. Many of the individuals we wanted to reach are constantly on the move, and they often have hesitations about engaging in an evaluation of such a sensitive topic.
To overcome this challenge, we collaborated with QRT staff who are trusted by the individuals in their communities to recruit and coordinate phone interviews. To maximize anonymity and create a safer space for individuals to share their thoughts and experiences, we provided prepaid phones to individuals to conduct the phone interviews. This approach ensured that individuals did not have to share their contact information with us and could feel more comfortable about sharing their stories.
Making Data-Informed Adjustments to Increase Engagement
Another key issue that has arisen during our evaluation process has been the delay between individuals’ initial engagement with the QRT team and the time taken for them to connect with treatment. To address this issue, we developed an important follow-up evaluation component. This gap has been a longstanding challenge for the team, and we believe that further data collection and analysis will offer valuable insights on how to shorten this lag time and improve treatment outcomes.
Through the new evaluation component, we will be able to gather data to give insight into how QRT may adjust their outreach efforts to increase the number of individuals connected to treatment and reduce the lag time between first engagement and connection to treatment. This effort will also involve conducting a literature review of best practices used to connect clients to treatment, including those used by Quick Response Teams (QRTs) across the nation, and analyzing existing program data to identify predictors associated with successful treatment connections. The information gathered from the literature review and data analysis will inform the development of a tool designed to assess predictors of successful treatment connection. The tool will generate a score and provide a protocol with recommended action steps for peer specialists.
EVALCORP will evaluate the effectiveness of the intervention by comparing treatment connection rates between the intervention group and a group receiving the usual engagement practices. As our concluding step, we will produce a report detailing the key findings from the intervention experiment.
Partnering for Community Impact in Pinellas County
Throughout the program’s grant years, EVALCORP will continue to collect and analyze data, assist in the preparation of quarterly performance reports and program success stories, present findings for staff and partners as needed, and contribute to literature on best practices for overdose responses with findings from the evaluation.
We are excited to support this project and are looking forward to seeing how the QRT continues to evolve their efforts to meet the needs of the community. It’s inspiring to witness how our evaluation work can lead to meaningful improvements in services that save lives and make a lasting impact. We are committed to helping Pinellas County first responders make informed, data-driven decisions that truly benefit those in need.