Participant Quality
The Highest Quality Survey Participants
QQFS’s commitment to providing the highest quality survey participants is underpinned by multiple initiatives designed to ensure that only the right people can access a survey, and highlight our commitment to eradicating respondent fraud and duplicate respondents. We combine industry-leading technologies including Imperium’s RelevantID, and Google’s reCAPTCHA, with our own proprietary innovations, to deliver best-in-class solutions to address these concerns:
Concern
Ensure only the right people are able to participate in a survey:
With the incentives on offer, particularly for physicians but also other healthcare professionals, there is a motivation for bad actors to try and participate in surveys
Solution.
- Panel Verification
The foundation is a high-quality panel and ensuring only the right people are available to be invited, by carefully verifying registrants against available data sources
- Check members prior to every survey
This critical but often overlooked step ensures that once removed from a panel, any bad actors cannot access other surveys even if they had already been invited
Concern
Eliminate duplicate respondents:
For a variety of reasons, some due to the complexity of having multiple fieldwork providers cooperating on a project, some due to innocent but perhaps forgetful valid respondents and some likely by design we see duplicate survey attempts by individuals and sometimes completions
Solution.
- Panel Qualtiy
Keeping duplicates out of large panels is not easy. They can occur innocently when someone registers, perhaps with a new email address and phone number after changing jobs. However, there are people who think they can get more survey opportunities by registering multiple times. Being aware and attentive to both issues is crucial to managing a high quality panel
- RelevantID Device Digital Watermarking and Fingerprinting
Being able to identify the same computer (or other electronic device) attempting to take a survey more than once is a good sign of possible duplicate participation. It is not perfect as some legitimate users will share devices, and some bad actors have learned to use multiple devices. Additionally, the technology used here can easily be confused with ad tracking solutions and result in blocking by some browsers. It is something we will be able to rely upon less over time
- Name / Address Matching
With custom recruited and our own panel members we de-duplicate surveys based on the name and partial address of the participant, either collected in real-time or based on their registered address.This is one more way we seek to minimise the possibility of duplicates appearing in survey data
Concern
Eradicate respondent fraud:
We see continued efforts, largely automatically blocked by our technologies, for people to inappropriately use valid survey links they have been sent
Solution.
- Geo-IP Validation
Completing a survey from a location that doesn’t match the desired target audience is a warning flag. Occasionally it can be legitimate participation whilst away from home, but if it is systematic then we will remove the respondent from the survey and panel
- Bot Detection (reCAPTCHA + RelevantID)
We occasionally see automated attempts to complete surveys using specially designed software. Pre-survey bot detection is an essential tool to catch the less sophisticated and make it more difficult for the more sophisticated attackers. However, it needs to be used in combination with survey data quality checks to defeat the most sophisticated attackers
- RelevantID FraudScore (with QQFS customisation for healthcare)
These checks rely upon fraudsters being either careless or unsophisticated, but it is surprising how many fall into these traps. Combining it with a feedback loop into the panel means the value of these checks is maximised
- AI-based IP Reputation Flagging
This is the cutting edge of antifraud technology and uses real time data collection and feedback to highlight high risk internet users so they can be excluded from surveys
Concern
Ensure high-quality survey data:
Poor quality data can result from inattentive participants. Either they become bored or distracted during a survey or perhaps sometimes come to it that way. Whilst survey design plays a role, panel hygiene is also needed to minimise this issue
Solution.
- Auto-capture survey start / end time
- Automated checks for:
- Speeding
- Flatlining
- Poor open-end responses
- Inconsistent answers
- ‘Red herring’ responses
- Data centralised and incentive payments blocked (Panel Quality)
- Scripted data checks customised for each project
