How to detect bots and AI agents in online surveys

Signs of a bad response

Look for patterns, not single oddities. Each of these has innocent explanations on its own; together they tell a story.

• —Finishing far faster than the median completion time.
• —The same answer down a whole grid of questions (straight-lining).
• —Failing a clear attention check.
• —Open-ended answers that are generic, off-topic, or suspiciously polished and similar to other respondents' answers.
• —Pasted text in fields people normally type into.
• —Answers that contradict each other, such as an age that doesn't match a stated birth year.
• —Several accounts sharing one device or network.

Stop them at the door

The cheapest fraud to deal with is the fraud that never reaches your survey. Recruit from a panel that verifies people before they see a study: device and network checks, one verified payment account per person, and identity checks when risk or your study calls for them.

Check from inside the survey

In-survey signals catch what verification can't.

• —Timing on each page, and how often people leave the tab.
• —Paste and typing-speed counts on open-ended questions, without recording what anyone wrote.
• —A hidden instruction that only automated agents read, placed so screen readers skip it, then checking whether an answer follows it.
• —Open-ended questions that ask about the person's own experience, which are hard to answer generically.
• —Your survey tool's own bot detection, if it has one.

Be fair to real people

Fast readers, people using assistive technology and people on older devices can trip a single check. Don't reject on one signal. Use attention checks that are clear and fair, combine several signals, give a reason for every rejection, and let participants appeal. Pre-register your exclusion rules so you aren't deciding after seeing the results.

Report what you excluded

Reviewers increasingly ask how a sample was checked. Report your exclusion rules, how many responses each rule removed, and how participants were recruited and verified. A signed provenance report makes this a citation instead of a paragraph.