Two new publications were released in February:
- Detecting careless response patterns in Likert scales using supervised machine learning and Monte Carlo simulations (authored by Artur Pokropek),
- Do You Agree? Do You Strongly Agree? The Effect of the Number of Response Categories on Response Processes and Verification of Substantive Hypotheses (authored by Artur Pokropek, Tomasz Żółtak, & Marek Muszyński).
The first paper appeared in Current Psychology and focuses on detecting respondents who provide careless answers in self-report data. The authors examine the effectiveness of an approach based on simulated data and machine learning algorithms—deep neural networks (DNNs) and support vector machines (SVMs)—in identifying complex patterns of careless responding at the respondent level. The study considers five response patterns: straight-lining, diagonal bouncing, midpoint responding, extreme alternating, and random responding. Simulation results indicate high classification accuracy for both models. Extreme alternating proved the easiest to detect, while random responding was the most difficult; SVM models with an RBF kernel performed best. Applying the method to empirical data collected under attentive and experimentally induced careless responding conditions revealed significantly higher occurrences of diagonal bouncing, midpoint responding, and random responding among inattentive respondents, whereas differences for straight-lining and extreme alternating were not statistically significant.
The second paper was published in the International Journal of Public Opinion Research and investigates how the number and labeling of response categories in survey scales influence respondent behavior, psychometric properties, and substantive conclusions. In a web-based survey experiment (over 2,800 participants), respondents were randomly assigned to scale versions differing in the number of response options and labeling formats. Engagement, response style tendencies, reliability, and convergent validity were assessed using process data (e.g., response times, cursor movements), self-reports, and psychometric modeling. The findings suggest that a greater number of response categories is associated with higher reliability, although the effects are modest and scale-dependent. Scales with more response options required longer completion times and elicited more complex response behavior, yet the number of response categories did not affect substantive conclusions in simple regression models.
Both articles are part of the publication series stemming from the NCN project Understanding response styles in self-report data: consequences, remedies and sources (2019/33/B/HS6/00937).
Those interested in obtaining full-text versions of the articles are kindly invited to contact the authors.
