PiTS Toolkit: Implementation Phase

Objectives

In the third phase, syringe services programs aiming to conduct a Point in Time Survey (PiTS) will: 

• Finalize survey protocol, including sampling methods and eligibility criteria
• Train survey interviewers to understand best practices of survey administration, including ways to minimize the impact of trauma and bias in order to maximize accuracy and ensure positive survey experience for respondents
• Collect survey data
• Monitor the quality of survey data as it is collected

Note that there are quite a few materials in this phase. While this phase may seem overwhelming, there are several distinct steps within this phase, so have faith that you don’t need to become familiar with all the materials all at once!

Contents

To support you as you implement your Point in Time Survey, this page contains brief presentations and resources to walk you through survey protocol development, interviewer training and interview guide development, incentive management, and data monitoring and quality assurance. And finally, there are screen recordings demonstrating skills needed to monitor survey data using Excel and REDCap.

Step 1: Decide Your Survey Methods

When you’re done designing your survey, it’s time to firm up your plans for your survey methods, which includes your sampling approach, eligibility criteria, recruitment strategies, data collection time frame, and sample size. Making intentional decisions about your methods will help you finish your survey protocol and plan for your data analysis.

Step 2: Finalize Your Survey Protocol

A survey protocol is essential for a program to document their methods for developing, designing, and implementing their survey. It will inform interviewer training and data monitoring as well as future updates and changes to survey processes. We have included a sample survey protocol for you to adapt according to your program needs.

Step 3: Plan for Incentive Distribution [if applicable]

Programs who choose to compensate participants for completing the survey should have a clear protocol in place for how incentives are dispensed. In general, programs who dispense incentives should consider how they will manage potential duplicate responses. If a program has a large enough incentive budget that having a few duplicate responses is not an issue, then there is no need for a unique ID and they should just be prepared to spend more time cleaning their data (i.e. cross checking the survey responses to make sure any responses that may be the same person are removed).

However, if the incentive budget is tight, the program should consider developing some way of identifying each survey, so that the same person is unlikely to take the survey twice.

While there is no single approach that will work for every program, one way this can be done is to create a unique ID that is tied to the chronological survey number, but is recorded only on the incentive log, not on the survey, so that interviewers can cross check the incentive log for prior survey response. Then, at the end of the implementation period you can destroy the incentive log and have no record of the unique IDs you collected (which could be compromised). Programs that choose to utilize this approach will need to come up with a unique ID format that requires as few personal details a possible. One recent survey used the following elements on a survey of ~500 people and had only one duplicate: 1) first 3 letters of astrological sign, 2) first 2 letters of first name, and 3) day of the month you were born on. The tie breaker if the unique ID already existed was the person’s shoe size.

Programs that do not offer an incentive are unlikely to administer the survey more than one time to a single individual since people are more likely to identify as having already taken it. Some programs may choose to have only certain people who can access or dispense the incentives in order to minimize the potential for lost or duplicate incentives. Below is a sample incentive protocol and incentive log for programs that offer a cash incentive, which should be adapted for your program’s unique needs. It can also be adapted for programs that distribute gift cards instead of cash.

Step 4: Train Interviewers

Every program implementing a Point in Time Survey should train their interviewers to administer the survey in a standard way to reduce survey bias. Below is a brief presentation where you will learn how to set up, and what to include in, an interviewer training.

As described in the presentation, SSPs should create an interviewer guide their interviewers can reference while administering a survey, in the event something unexpected comes up, such as an unusual answer or point of clarification. This example guide can help you create your own based on your own survey flow and questions.

Additionally, as noted in the presentation, we created a template slide deck you can adapt to train your interviewers based on your unique survey, including components on the survey background, consent processes, data privacy and security, good practices for survey administration, “what if” scenarios, survey bias, and time for practice.

For SSPs whose surveys include sensitive questions, interviewers may wish to have training on the concepts of motivational interviewing, as they include important counseling skills that may help interviewers navigate any unexpected participant responses related to their experiences using substances or other life experiences. This list includes some free resources you may provide your interviewers.

Step 5: Collect Data (And Monitor It!)

Once you have selected an implementation time frame, identified a sampling method and eligibility criteria, and trained your interviewers, you are ready to collect data! This entails finalizing a recruitment plan and promoting the survey to eligible respondents.

After you’ve begun collecting data, reviewing your data throughout the implementation period is essential to ensure that any problems with your survey tool or your interview process are addressed in a timely manner so that the remaining surveys are conducted in a consistent way that results in accurate data. The remaining resources and materials in this section will help you build the skills needed to monitor your survey data.

Data monitoring plans can be simple and straightforward, but should be documented in writing so that the person or people responsible for monitoring survey data have clear instructions they can follow. A sample data monitoring plan that can be adapted for your unique program needs is below.

Data Monitoring Tip: File Naming

When you download your dataset from your survey platform, you will want to make sure you save the file in a way that allows you to track versions, meaning that you know the difference between each download. For example, you may download your data partway through the implementation phase to monitor the quality of survey data so far, and you’ll want to be able to easily recognize which file has incomplete data versus complete data. Similarly, you want to differentiate between your raw data and your cleaned data, and data that has been sorted in specific ways for analysis. Watch the brief tutorial below for some good practices about creating file versions of your dataset.

Data Monitoring Tip: Filters and Pivot Tables

In these tutorials we will learn how to filter survey data and create pivot tables in order to track survey data as it comes in using the sample dataset that corresponds to the sample Point in Time Survey provided. This will help you report out on how many surveys have been completed to date, basic respondent characteristics, and identify and survey issues and/or potential errors and address them with the survey team to ensure greater accuracy during the rest of the survey period.

Special Topic: Running Reports in REDCap

During survey design phase, we learned that SSPs implementing Point in Time Surveys should consider the pros and cons of various platforms for entering their survey data. SSPs who choose to utilize REDCap may be new to the software. To support familiarity with the platform, we have recorded an introduction to how to run reports in REDCap. This brief tutorial will show you how!