How to design surveys and experiments for top tier publishing

Being able to design experiments and surveys is a key qualification for anyone who wants to do any type of behavioral research. However, despite a lot of effort and resources invested, often experiments and surveys are designed suboptimally or are outright flawed. This also means that the chances to publish such studies in top-tier journals will be seriously limited. In this four-day PhD (and junior faculty) seminar, we will teach how to design and conduct experiments and surveys that rigorously test theory.

Additional emphasis will be laid on typical challenges when designing, recruiting for, analyzing, and writing up experiments and surveys. This will also include coverage of online studies.

The seminar consists of a mix of extensive prereading of key articles, to-the-point in-class discussion, and actual designing of own research projects within the seminar using Questback or Qualtrics and the Amazon’s MTurk platform.

At the end, participants will be able to confidently design their own behaviorally-oriented studies, and potentially already have a running start with one of them.

• Introduction to the behavioural foundations of research in management and organization disciplines and how they are linked to the survey/experimental paradigm.
• Coverage of  various experimental design types (e.g. lab experiment, scenario experiments, online experiments, between/within-subjects designs, etc.) and survey designs (panel designs, time-lag designs, cross-sectional designs, multi-source, etc.) 
• Typical challenges when designing experiments/surveys (e.g., demand characteristics, sampling bias, measurement, common method/source variance, measurement quality and separation, implicit vs. explicit measures,  single item measures, manipulation checks, outliers, drop-out, translation, etc.) and how to address them optimally.
• Introduction on how to approach the design from the perspective of your theory (e.g., how to design experiments for testing mediation effects), provision of input on typical mistakes in the analysis of experimental and survey data (e.g., not accounting for nested data structures, difference scores, etc.),  and discussions of types mistakes in the write-up (e.g., inferring causality, implying behavior instead of perceptions, etc.).
• Discussion of how you should design multiple study papers to provide a compelling test of your hypothesis (e.g., triangulation, conceptual replication, etc.). 
• Strategies to rule out alternative theoretical explanations of your findings by design or with covariates.
• Outlook on how topics like Big Data, new European privacy laws, and the replication crisis might affect the methods presented in this course.
• First hand insights how to dance the tango with reviewers from two instructors who have published in and reviewed for some of the field’s best journals (Including Niels being Senior AE at The Leadership Quarterly and Chris being AE at the Academy of Management Journal). Additional live input (incl. Q&A session) will be provided by Marc Gruber, current editor in chief of the Academy of Management Journal.

This course is designed for PHD students and other early stage researchers from all disciplines in management (e.g., organizational behaviour, strategy, operations management, etc.) and related fields (e.g. psychology, sociology, etc.).

This is not a statistics course. We therefore do require that you have working knowledge in statistics, particularly, variants of regression-based methods. You will also need a statistics software to analyze your survey/ experimental data. Of course, it is you choice what you will use. But we recommend STATA simply because it is a powerful software, we know this software quite well, and can give feedback on how to use it.

The course will be limited to max. 50 participants.