Using Qualtrics survey logic to eliminate redundancy in simple experiments

I just reviewed a Qualtrics survey for a 3x2 between subjects experiment which repeated 3 times within subjects. The survey designer was fairly inexperienced so it's not surprising that they used 36 blocks with 5-8 questions each and had incredibly complex logic to accomplish this. Unfortunately, there was so much redundancy and complexity that I wasn't able to error-check the survey instrument. I gave the student suggestions for how to go from 36 blocks with complex survey logic to 1 simple block with straightforward logic. My recommendations follow below in case others find it useful.

The stimuli were text-based (no images), so my suggestions here focus on that. 

Between subjects differences: Use Embedded Data. 

Embedded data allows you to assign conditions and specific text to individual participants. The conditions can be randomly assigned by simply using a randomizer in the survey flow. 

Here are the steps:

  1. Edit survey flow
  2. Add randomizer
  3. Within the randomizer, add embedded data. In a 2x2 experiment, you should create 4 blocks of embedded data, one for each condition. I've only included the first two below but you get the idea. Alternatively, you can have two randomizers, one for each factor, and you can set the embedded data separately for each factor. 

That's it for the survey flow. Now when you create the question block, you can insert the variable text you've assigned to each participant using "Piped Text," "Embedded Data Field." 

Repeated measures: Use Loop and Merge.

If you ask the same set of questions three times but only vary a few words in each set of questions, Loop & Merge can help you do that without having to copy all of the questions. The reason you want to avoid duplication where possible is because you may go back later and tweak the question. Then you'll have to tweak it in multiple places and error can enter in. 

Within Loop & Merge, each row is a repetition and each column is a set of text that varies across the repeated measures. So if I wanted to ask the same set of questions about 3 different celebrities, I would delete Field 2, add the 3 names in Field 1 and delete rows 4 and 5. 

Bringing in the dynamic text for Loop & Merge is very similar to how we did it for Embedded Data. 

Use Page Break to pipe in data from a previous survey question.

In this case, the respondents were asked to give the name of a celebrity or person they knew (depending on the condition). If you pipe in data from a previous question, you don't want to do that on the same page that you asked the previous question because it won't refresh. Instead, you should ask it on the following page. That doesn't necessitate a new block. You can simply use "+ Add page break" (see below). 

Again, you would use the piped text tool but this time you would select "Survey Question" to pipe in text from a previous question. 

That's how you can go from 36 blocks with an incredible amount of complexity and redundancy down to one very simple block. I hope it helps. 


Popular posts from this blog

Conflicts between Dan Ariely's statement and Footnote #14 (DataColada #98)

About that metaphor priming paper...

RETRACTED ARTICLE: Why money meanings matter in decisions to donate time and money