Discussing Experimental Design Challenges and Field Experiments in Public Administration Research

College students looking for in-depth discussion prompts on the Hawthorne effect, internal validity, generalizability, and public sector field experiments in research methods courses benefit from this detailed assignment guide.

You should post your answers to the individual discussion questions to this forum by 11:59 p.m. on Friday, February 27th. Engaging with these questions early helps build a stronger understanding of research principles. Then, you will need to offer some input with respect to what other students have written.

Individual Discussion Board Questions:

1. What is the “Hawthorne effect,” and how might it impact an experimental design? What are some ways that researchers might control for this effect?
2. What are the eight sources of internal invalidity? How might they affect the results of a study? With respect to your study, which sources, do you think, would be most problematic? Why? What actions will you take to address these issues?

Considering these factors encourages careful planning in your own research proposals.

3. Why is generalizability a major issue when it comes to experimental designs? Is there anything that researchers can do to improve an experiment’s generalizability? If so, what? If not, why not?
4. Hansen and Tummers (2020) examine the notion of field experiments as they apply to public administration. What are some of the considerations that researchers must take into account when designing public-sector field experiments? Discuss. Does our second article of the week, by Menger and Stein (2018), take these considerations into account? Is there anything that Menger and Stein could have done to improve their field experiment?

Reviewing these articles closely reveals practical applications of experimental methods in real-world settings.

The Hawthorne effect refers to changes in participants’ behavior due to their awareness of being observed, and it can skew results by introducing unintended variables into an experimental design. Researchers can control for this by using blind or double-blind procedures where neither participants nor experimenters know the group assignments. Another approach involves incorporating control groups that receive similar attention without the actual intervention. In studies involving human subjects, minimizing direct observation through technology like remote monitoring also reduces this bias. For internal invalidity, sources like history and maturation pose risks by allowing external events or natural changes over time to influence outcomes. Instrumentation issues arise when measurement tools vary, leading to inconsistent data. Selection bias becomes problematic in my study on policy implementation because uneven group assignments could misrepresent effects, so I plan to use random assignment to mitigate it (Hansen and Tummers 2020, https://doi.org/10.1111/puar.13181).

References/Learning Materials

1. Jilke, S., Olsen, A. L., Resh, W. and Siddiki, S. (2019) Using large-scale social media experiments in public administration: assessing charitable consequences of government funding of nonprofits. Journal of Public Administration Research and Theory, 29(4), pp. 627-639. DOI: 10.1093/jopart/muz023.
2. Pérez-Durán, I. (2024) Twenty-five years of accountability research in public administration: authorship, themes, methods, and future trends. International Review of Administrative Sciences. DOI: 10.1177/00208523231211751.
3. Christensen, T. and Lægreid, P. (2025) Future challenges of public administration. Public Organization Review. DOI: 10.1080/15309576.2025.2549297.
4. Menger, A. and Stein, R. M. (2018) Enlisting the public in facilitating election administration: a field experiment. Public Administration Review, 78(6), pp. 892-903. DOI: 10.1111/puar.12833.