This guide walks you through adding LLM and bot authenticity checks to a Qualtrics study. For what these checks do and when to use them, see What are authenticity checks?
Before you start
You'll need a Qualtrics plan that supports custom header scripts. This isn't available on Qualtrics free accounts.
Your study should already exist as a draft on Prolific, with its study URL entered on the 'Data collection' page. The authenticity check script is generated from this URL, so it needs to be in place first.
You can set up LLM checks, bot checks, or both, depending on your study. See What are authenticity checks? to learn how each one works.
Watch a video walkthrough
What you'll do
Set up your study on Prolific to record Prolific IDs
Set up Qualtrics to run the checks you want
Confirm the checks are working once data starts coming in
Step 1: Set up your study to record Prolific IDs
In Qualtrics
Go to Qualtrics, add a block to your survey named 'Prolific ID', and create a 'Text entry' question. In the question text, include the following message:
2. Go to 'Survey Flow' and add an 'Embedded Data' element at the beginning of your study. Into the embedded data field, type 'PROLIFIC_PID' and make sure it says 'Value will be set from panel or URL'. Do the same for 'STUDY_ID' and 'SESSION_ID'. Including SESSION_ID matters here - it's what lets us link the authenticity check to the submission in your study.
3. Now go to the survey question that records the participant's Prolific ID. Right-click on the text-entry field and select 'Default Choices'.
4. When the window opens, click the blue drop-down arrow on the right, then select 'Embedded data field'. Either enter 'PROLIFIC_PID' directly, or click the drop-down arrow to the right of the text entry box, select 'PROLIFIC_PID', then click 'Insert'.
5. You should see the string '${e://Field/PROLIFIC_PID}' populate the field. Click 'Save' and you're done with this part.
In Prolific
To record the Prolific ID, use the URL parameters option in Prolific. In the 'Recording Prolific IDs' section, enter your Qualtrics study URL in the 'What is the URL of your study?' box, then select 'URL parameters'. This automatically adds the necessary GET parameters to your URL.
Your 'Study Link' box should now look something like this:
Step 2: Set up Qualtrics to run the checks
On the 'Data collection' page in Prolific, find the 'Authenticity checks' section. Check 'Check for LLM use', 'Check for bots', or both, depending on your study. Your script generates automatically once your study URL is in place.
2. Click 'Copy'.
3. Paste the generated script into the Qualtrics 'Header'. To find it:
Click 'Survey' on the top toolbar
Click 'Look and Feel' from the left side options
Click 'General'
Scroll down to 'Header' and paste your script using one of these methods:
Click in the input field and paste the script directly
Or click 'Edit' to open a modal window, turn on 'Code Mode', then paste the script in the text area
Click 'Save'
4. Press 'Publish' in Qualtrics.
5. Go back to Prolific and click 'Check script'.
If the script is detected, you'll see a green 'Connected' tick next to the button.
If it's not detected, you'll see a red error message. If it says 'Script not found,' double-check that you've added the script to the Header in Qualtrics, then try again.
The script must show as 'Connected' before you can publish the study.
Step 3: Confirm your checks are working
Once your study goes live, you'll see two new columns on the submissions page: 'Auth check: LLM' and 'Auth check: Bots'.
As submissions come in, you'll see a result for each column. For what these results mean and how to act on them, see How to review your authenticity check results.