Point in Time Survey (PiTS) Toolkit for Use at Syringe Services Programs

A Point in Time Survey (PiTS) allows a syringe services program (SSP) to ask a standardized set of questions from a portion of their clients, providing a snapshot of the population that the SSP serves. These surveys can be a quick and low-cost way to learn about the characteristics, needs, and service utilization patterns of their clients.

Overview

Often called cross-sectional surveys, PiTS can be used by SSPs to deepen their understanding of their participants and community, inform the public and funders about their reach, and potentially reduce routine data collection. Watch this presentation to learn more about what a point in time survey can help you accomplish, and what steps are involved so you know what to expect.

This toolkit lays out five main phases of PiTS implementation. Every SSP who decides to implement a PiTS can decide to spend more or less time on each of the activities we recommend. The phases are:

• Survey planning, where survey goals and objectives are identified, reviewed, and refined, questions are drafted for consideration, and preparations are taken, including efforts to build staff buy-in and logistics planning.
• Survey design, where a survey protocol is developed; sampling method is selected; and survey questions are refined, piloted, arranged into an appropriate flow. The survey is formatted for paper and programmed onto a digital platform if applicable.
• Survey implementation, where survey interviewers are trained, a quality assurance plan is developed, participants are notified of the upcoming survey, and survey responses are collected.
• Data cleaning and analysis, where data are adjusted for errors, analyzed, and results are reviewed with stakeholders.
• Data dissemination and implementation of findings, where survey results are provided to participants and community stakeholders and findings are incorporated to improve programs and services 

This toolkit includes resources, tools, and examples for each stage of the PiTS implementation process for programs to consider when designing and implementing their own surveys. Each example material is based off of a brief example point in time survey so that learners can easily adapt as they develop their own surveys. Each phase is structured around explanatory slides and/or tutorials to provide context about how to use the materials as well as detail about survey best practices to consider. 

The materials and guidance included in this toolkit have been utilized and vetted by SSPs who actually implemented a PiTS. At the same time, learning collaborative members recognized that live support was key in completing their projects due to the motivation associated with real-time feedback. With that in mind, while we believe that the materials can be independently useful, we also hope that you will utilize our technical assistance team as you go along! You may reach out to us at sharpta@uw.edu to submit a request for technical assistance at any time. 

What is a Point in Time Survey?

SSPs use PiTs in different ways. Some SSPs implement a regular (quarterly, annual, bi-annual) PiTS as part of their overall monitoring and evaluation strategy. SSPs may also do a one-time PiTS as needed. These surveys can be a nimble tool for answering urgent questions or providing information in a timely manner. They can also be used as a method for reducing the routine data collection.

Throughout the toolkit, certain terms will be used to describe various aspects of the survey process. To provide an even footing for those with and without previous survey experience, we have created a glossary of key terms for reference throughout the project.

Is the data from a PiTS representative of the population an SSP serves?

A well-executed PiTS that is considerate of factors that impact participation may be considered to be representative of the population an SSP serves. When planning a PiTS, it’s important to think about how the survey can capture a representative group of your clients. This is also known as sampling. Different sampling approaches including inviting all participants over a defined period of time to complete the survey, or only inviting every 3^rd or 5^th participant. If you provide services both at fixed sites and during mobile outreach, we recommend offering the survey to clients in both settings.

How do you know if who participates in the survey is different from who declines to participate?

It is very difficult to know if the people who participate are different from the people who do not. It is considered best practices to record how many people are offered the survey, how many agree to participate, and how many do not. You may also want to ask those who do not want to participate if they would be ok answering demographic questions (usually age, gender, and race/ethnicity). If the demographics of those who decline are measured you can compare them to those who participate to see if they are different. If so, that is a great opportunity to think more about why and what you can change next time.

Is a PiTS representative of people who inject drugs in the larger community?

A PiTS done at an SSP is most likely not representative of people who inject drugs in the larger community. There may be differences between the people who inject drugs who come to an SSP and the people who do not. For example, some research has found that women and/or people of color may be less likely to attend SSPs.

What percentage of clients does a PiTS capture? Or need to capture to be considered representative?

SSPs that know how many unique clients they serve may be able to calculate an estimate of what proportion of their clients participated in a PiTS. Many SSPs that use PiTSs as part of their larger monitoring and evaluation strategy do not know what percentage of their clients participate, and while this is a limitation, it does not mean the PiTS is not meaningful or useful.

To ensure that a meaningful proportion of clients participate, SSPs should strive to collect surveys from as many clients as possible. To achieve this, SSPs should consider the right time of year to conduct a survey (based on weather and migration), how long to run the survey, and ensuring enough staff or volunteers during data collection so participants can complete the survey quickly.

How many participants should a PiTS aim for? What number of participants would I need to reach statistical significance?

There is not one answer for how many people should participate in a PiTS. It is often dependent on the resources available, namely staff time. While running a PiTS there will be more work for staff, so data collection may be limited by how long an SSP can run the survey without burning out staff. 

Statistical significance is a calculation used in research to try and understand if differences between groups are potentially real, or if the differences between groups are due to chance. For example, a study may be exploring health outcomes of an intervention and find a difference between two groups. The researchers would then test their results for statistical significance to help them understand how confident they should be in their conclusion.

Most PiTS conducted by SSPs do not aim to calculate statistical significance since they are not answering research questions. Typically, SSP PiTS data are used for program evaluation for both internal use and external reporting, which tend to not use statistical tests. If an SSP is interested in calculating statistical significance they would need to work with a statistician to calculate the sample size needed to power the study based on the question they are trying to answer.

If an SSP does an annual PiTS, should the questions be the same every year? Or can they be different?

Many SSPs that conduct a regular (annual, bi-annual) PiTS will have a “core” set of questions they ask in every survey. These core questions are typically based on reporting requirements (e.g., participant demographics) and/or key health and behavior indicators (e.g., syringe sharing, naloxone possession, self-report opioid overdose). SSPs will add other questions based on current topics of interest at the time of each survey (e.g., interest in fentanyl test strips, PrEP knowledge, COVID-19 vaccination).

The core questions are asked in the same way each time so the data are comparable across periods. The other questions may remain for a couple of instances and then rotate off the survey as trends and needs change.

Do I need to deduplicate PiTS data? Or does my PiTS need a unique ID?

A PiTS can, but does not need to, have a unique ID. The major benefit to having a unique ID is that at the end, the data can be deduplicated and an SSP can be reasonably sure that all the answers are from unique individuals. If there is not an incentive offered for completing the survey, the most basic strategy to avoid duplication is asking a person if they’ve already taken the survey. Most people will not want to answer a survey twice if there is no incentive. If there is an incentive offered, it may be useful to include a unique ID in order to de-duplicate data.

Should we offer an incentive for completing the survey? 

Whether to offer an incentive or not is up to each SSP. There are reasons to consider either option. This is important to discuss internally with clients and staff before data collection. The toolkit includes a sample incentive protocol for SSPs that decide to utilize incentives.

If an SSP conducts a PiTS, is it considered research and does it need approval from an Institutional Review Board (IRB)?

A PiTS done by an SSP can be considered part of routine surveillance or program evaluation. If the objective is to use the data to understand clients and inform program activities, it does not require review by an institution review board (IRB). If the SSP wants to conduct a survey to answer a research question and generalize the findings to all SSP clients, this would likely require IRB review. Regardless of whether a survey is reviewed by an IRB, we encourage SSPs to incorporate principles to maximize participant safety and confidentiality. For example, while all clients during the survey period are approached to participate, participation should be optional and not affect the receipt of services. All clients who participate must provide consent.

What it takes to implement a Point in Time Survey

When SSPs are first considering running a PiTS, they often request examples from organizations who have done it before. During the Learning Collaborative we shared the example of Public Health Seattle & King County, which has been running a biannual PiTS for more than 10 years. A description of their program and project is below.

Several members of the Point in Time Survey Learning Collaborative contributed similar descriptions of their survey development process and implementation experience, to provide insight into what it took them to complete the project.