Publications and research

COVID-19

Since April 2020, I’ve been part of the Delphi group’s COVID-19 response team. Delphi produces the COVIDcast web site and API, containing real-time data on the pandemic across the United States, and obtains data from unique sources. After time as co-manager of the engineering team, I moved to the the COVID-19 Trends and Impact Survey (CTIS), conducted in collaboration with the University of Maryland and Facebook. I became Principal Investigator (with Robin Mejia) of the United States version of the survey, which ran until June 2022.

My work with Delphi resulted in the following papers, along with several posts on the Delphi blog:

1. Ellingwood, M., Reinhart, A., Do, D. P., & Mejia, R. (2024). Health concerns and government distrust: Unraveling types of COVID-19 vaccine hesitancy in the US before and at universal vaccine eligibility. (Submitted)
2. Reistma, M. B., Rose, S., Reinhart, A., Goldhaber-Fiebert, J. D., & Salomon, J. A. (2024). Bias-adjusted predictions of county-level vaccination coverage from the COVID-19 Trends and Impact Survey. Medical Decision Making, 44(2), 175–188. doi:10.1177/0272989X231218024
3. Reinhart, A. et al. (2021). An open repository of real-time COVID-19 indicators. Proceedings of the National Academy of Sciences, 118(51). doi:10.1073/pnas.2111452118
4. Salomon, J. A. et al. (2021). The US COVID-19 Trends and Impact Survey: Continuous real-time measurement of COVID-19 symptoms, risks, protective behaviors, testing, and vaccination. Proceedings of the National Academy of Sciences, 118(51). doi:10.1073/pnas.2111454118
5. King, W. C., Rubinstein, M., Reinhart, A., & Mejia, R. J. (2021). COVID-19 vaccine hesitancy January-May 2021 among 18-64 year old US adults by employment and occupation. Preventive Medicine Reports, 24. doi:10.1016/j.pmedr.2021.101569
6. King, W. C., Rubinstein, M., Reinhart, A., & Mejia, R. J. (2021). Time trends, factors associated with, and reasons for COVID-19 vaccine hesitancy: A massive online survey of US adults from January–May 2021. PLoS ONE, 16(12). doi:10.1371/journal.pone.0260731
7. Arnold, T., Bien, J., Brooks, L., Colquhoun, S., Farrow, D., Grabman, J., Maynard-Zhang, P., Reinhart, A., & Tibshirani, R. (2021). covidcast: Client for Delphi’s COVIDcast Epidata API. https://cmu-delphi.github.io/covidcast/covidcastR/

Delphi and its collaborators have received several awards for this work, including:

• The 2021 American Statistical Association Statistical Partnerships Among Academe, Industry, and Government (SPAIG) Award
• The 2022 AAPOR Policy Impact Award, for CTIS
• The 2022 AAPOR Warren J. Mitofsky Innovators Award, also for CTIS

Pedagogy

I have an active interest in statistical pedagogy, and in developing new ways to improve student learning, assess understanding of statistical concepts, and better teach the foundations of statistical reasoning. I help lead the Teaching Statistics Group at Carnegie Mellon University’s Department of Statistics & Data Science, and collaborate with researchers in the Department of English to study student writing in statistics. I am an Associate Editor for the Journal of Statistics and Data Science Education and edit the CMU Statistics & Data Science Data Repository, a collection of datasets curated for classroom use.

1. Reinhart, A., Brown, D. W., Markey, B., Laudenbach, M., & Weinberg, G. (2024). Do LLMs write like humans? Variation in grammatical and rhetorical styles. http://arxiv.org/abs/2410.16107 (Submitted)
2. DeLuca, L., Reinhart, A., Weinberg, G., Miller, S., Laudenbach, M., & Brown, D. W. (2024). Developing students’ statistical expertise through writing in the age of AI. (Under revision)
3. Reinhart, A. (2024). The regressinator: A simulation tool for teaching regression assumptions and diagnostics in R. (Under revision)
4. Evans, C., Reinhart, A., Cooley, E., & Cipolli, W. (2024). Learning while learning: Psychology case studies for teaching regression. Journal of Statistics and Data Science Education.
5. Laudenbach, M., Brown, D. W., Guo, Z., Ishizaki, S., Reinhart, A., & Weinberg, G. (2024). Visualizing formative feedback in statistics writing: An exploratory study of student motivation using DocuScope Write & Audit. Assessing Writing, 60. doi:10.1016/j.asw.2024.100830. https://ssrn.com/abstract=4445888
6. Reinhart, A., Evans, C., Luby, A., Orellana, J., Meyer, M., Wieczorek, J., Elliott, P., Burckhardt, P., & Nugent, R. (2022). Think-aloud interviews: A tool for exploring student statistical reasoning. Journal of Statistics and Data Science Education, 30(2), 100–113. doi:10.1080/26939169.2022.2063209. http://arxiv.org/abs/1911.00535
7. Reinhart, A., & Genovese, C. R. (2021). Expanding the scope of statistical computing: Training statisticians to be software engineers. Journal of Statistics and Data Science Education, 29(S1), S7–S15. doi:10.1080/10691898.2020.1845109. http://arxiv.org/abs/1912.13076 (Special issue on Computing in the Statistics and Data Science Curriculum)

Selected presentations include:

1. M Meyer, J Orellana, and A Reinhart (2020). Using Cognitive Task Analysis to Uncover Misconceptions in Statistical Inference Courses, eCOTS 2020.
2. C Evans, A Reinhart, P Burckhardt, R Nugent, and G Weinberg (2020). Exploring how students reason about correlation and causation, eCOTS 2020.
3. P Burckhardt, P W Elliott, C Evans, A Luby, M Meyer, J Orellana, J Wieczorek, R Nugent, and A Reinhart (2019). Writing practical pre- and post-tests for concepts in introductory courses, CMU Eberly Teaching and Learning Summit.
4. A Reinhart, P Burckhardt, P W Elliott, C Evans, K Lin, A Luby, M Meyer, J Orellana, R Yurko, G Weinberg, J Wieczorek, and R Nugent (2019). Using think-aloud interviews to assess student understanding of statistics concepts. Breakout session, USCOTS 2019.
5. P Burckhardt, P W Elliott, C Evans, S Hyun, K Lin, A Luby, C P Makris, M Meyer, J Orellana, R Yurko, G Weinberg, J Wieczorek, R Nugent, and A Reinhart (2018). Developing an assessment for concepts in introductory statistics and data science. CMU Eberly Teaching and Learning Summit. (People’s Choice Award winner)
6. S Hyun, P Burckhardt, P Elliott, C Evans, K Lin, A Luby, C P Makris, J Orellana, A Reinhart, J Wieczorek, R Yurko, G Weinberg, and R Nugent (2018). Identifying misconceptions of introductory data science using a think-aloud protocol, eCOTS 2018.
7. P Burckhardt, P Elliott, S Hyun, K Lin, A Luby, C P Makris, J Orellana, A Reinhart, J Wieczorek, G Weinberg, and R Nugent (2017). Assessment of Student Learning and Misconception Identification in Intro Statistics, CMU Eberly Teaching and Learning Summit.

Point processes

During my PhD thesis I worked with Joel Greenhouse to design statistical models to predict crime by using crime hotspots, spatial features, seasonal factors, and leading indicators (like 311 calls, criminal mischief, and so on). My goal is both to improve crime prediction and to provide inference tools for criminologists to understand factors that lead to crime. I also work on evaluation and diagnostic methods to understand the performance of predictive policing models.

My dissertation work was supported by a National Institute of Justice Graduate Research Fellowship (GRF-STEM).

More recently, I have pursued other applications of point process models, such as to the modeling of wildfires and their associated risks.

1. Reinhart, A. (2018, July). Point process modeling with spatiotemporal covariates for predicting crime (PhD thesis). Carnegie Mellon University. https://doi.org/10.1184/R1/7178903.v1
2. Reinhart, A., & Greenhouse, J. (2018). Self-exciting point processes with spatial covariates: Modeling the dynamics of crime. Journal of the Royal Statistical Society: Series C, 67(5), 1305–1329. doi:10.1111/rssc.12277. http://arxiv.org/abs/1708.03579
3. Reinhart, A. (2018). A Review of Self-Exciting Spatio-Temporal Point Processes and Their Applications. Statistical Science, 33(3), 299–318. doi:10.1214/17-STS629. http://arxiv.org/abs/1708.02647 (With invited discussion)
4. Reinhart, A., & Nagin, D. S. (2017). The Next Step: A Spatiotemporal Statistical Model of the Birth and Death of Crime Hotspots. Jerusalem Review of Legal Studies, 15(1), 55–60. doi:10.1093/jrls/jlx007. https://www.refsmmat.com/files/papers/jrls.pdf

Radiation anomaly detection

As an undergraduate I started a project (supervised by Alex Athey of Applied Research Laboratories) to devise methods to continuously monitor the radiation background in a wide area and detect any sudden changes, such as might be introduced by a dirty bomb or stolen radioactive source. We built a system which uses gamma spectroscopy to compare new measurements to previous observations of the radiation background, making it feasible to monitor a wide area with mobile detectors and rapidly detect changes.

At Carnegie Mellon University, I continued the project under Valérie Ventura and Chad Schafer, proposing a new method based on Kolmogorov–Smirnov tests. James Scott and Wesley Tansey continued the work to devise a new spatial smoother for radiation spectra.

1. Padilla, O. H. M., Athey, A., Reinhart, A., & Scott, J. G. (2019). Sequential nonparametric tests for a change in distribution: An application to detecting radiological anomalies. Journal of the American Statistical Association, 114(526), 514–528. doi:10.1080/01621459.2018.1476245. http://arxiv.org/abs/1612.07867
2. Tansey, W., Athey, A., Reinhart, A., & Scott, J. G. (2017). Multiscale spatial density smoothing: An application to large-scale radiological survey and anomaly detection. Journal of the American Statistical Association, 112(519), 1047–1063. doi:10.1080/01621459.2016.1276461. http://arxiv.org/abs/1507.07271
3. Reinhart, A., Ventura, V., & Athey, A. (2015). Detecting changes in maps of gamma spectra with Kolmogorov–Smirnov tests. Nuclear Instruments and Methods in Physics Research A, 802, 31–37. doi:10.1016/j.nima.2015.09.002. http://arxiv.org/abs/1507.06954
4. Reinhart, A., Athey, A., & Biegalski, S. (2014). Spatially-Aware Temporal Anomaly Mapping of Gamma Spectra. IEEE Transactions on Nuclear Science, 61(3), 1284–1289. doi:10.1109/TNS.2014.2317593. http://arxiv.org/abs/1405.1135
5. Reinhart, A. (2013, April). An Integrated System for Gamma-Ray Spectral Mapping and Anomaly Detection (Undergraduate thesis). University of Texas at Austin. https://hdl.handle.net/2152/20071

Popular press

My book Statistics Done Wrong was published in March 2015 by No Starch Press. Covering common statistical errors present in a wide range of current scientific research—not just simple misuses of t tests but failures of statistical power, unnoticed pseudoreplication, effect size inflation, and much more—it has since been translated into German, both simplified and traditional Chinese, Japanese, and Korean.

Other articles in the popular press:

1. Reinhart, A. (2014). Huff and puff. Significance, 11(4), 28–33. doi:10.1111/j.1740-9713.2014.00765.x. https://www.refsmmat.com/files/papers/huff.pdf

Letters, editorials, and miscellany

1. Response to “Crime Places in Context” (PDF) (May 2016), a letter to the editor regarding statistical errors in an article by Deryol et al. in the Journal of Quantitative Criminology. Their reply.
2. Comment on “Inigo Montoya and Statistical Significance” (April 2016), an invited reply to a column by Joseph Bernstein in Clinical Orthopaedics and Related Research.