Discover
BERTopic (MiniLM embeddings, UMAP, HDBSCAN, c-TF-IDF) under randomized hyperparameter search. The model with the highest cv coherence wins.
50 draws × 24 solutions1,200 planned candidate models
A human-in-the-loop instrument for public health surveillance
A five-stage pipeline that turns large volumes of social media text into a small set of interpretable themes, and those themes into prevalence estimates you can test. Use the visual demonstration to understand every step, then fork the repository or download the Python package for a reproducible, agent-assisted research workflow.
The method
Machines do what machines do well: search a large model space, read every document, and count. People do what only people can: decide what a pattern means. The pipeline keeps the two in their lanes and writes down everything that passes between them.
BERTopic (MiniLM embeddings, UMAP, HDBSCAN, c-TF-IDF) under randomized hyperparameter search. The model with the highest cv coherence wins.
50 draws × 24 solutions1,200 planned candidate models
A language model names each topic thousands of times, independently. The modal label is adopted, and a fragmented vote flags an ambiguous topic.
Votes per topic5,000 · temperature 0.5
Two researchers group topics into themes under the Theme and Topic system, reading against the literature. Deliberately not automated.
ProtocolPrimary + senior, consensus
Boolean and LLM classifiers are piloted against human-labeled documents. The pipeline refuses to scale a classifier below the agreement threshold.
Benchmark · gate250 documents · >85%
Every document carries a label, so prevalence is computable in any slice. Proportional z-tests compare periods under Bonferroni correction.
OutputPrevalence + z-tests
The demonstration
The pipeline was built for, and first applied in, the author's dissertation: a study of what eleven alcohol companies said on Twitter before, during, and after the first pandemic lockdowns, and of whether that messaging moved the public conversation.
| Theme | Pre-pandemic (%) | Peri-pandemic (%) | Post-pandemic (%) |
|---|---|---|---|
| Alcohol Delivery and Isolation Drinking | 9.1* | 17.4† | 11.9*† |
| Restaurant Support | 1.9* | 5.5† | 1.7* |
| Social Media Promotions | 6.6* | 16.5† | 10.6*† |
| Sports | 8.2 | 11.4 | 8.9 |
| Gaming | 1.1 | 2.0 | 1.6 |
* differs from peri-pandemic; † differs from pre-pandemic (p < 0.0033, Bonferroni-corrected).
The preprints
The methods paper describes the instrument. The original studies are available directly from the unchanged dissertation. Existing reflowed PDF and Word editions are also preserved. The methods paper has a separately identified audited revision.
Methods · 2026
A transparent chain from documents to topics, human-reviewed themes, validated labels and prevalence estimates.
Study 1 · 2026
“This study aimed to identify themes used in tweets by alcohol brands early in the pandemic, determine if the prevalence of these themes differed across time, and characterize the sentiment used in the themes.”
Study 2 · 2026
“Early potential exposure to alcohol brand tweets significantly predicted subsequent user-generated content about alcohol delivery and isolation drinking … Conversely, early user-generated content did not significantly predict subsequent potential exposure.”
The full dissertation, Alcohol Marketing on Social Media During the COVID-19 Pandemic: Historical Perspectives, Modern Evidence, and Future Regulation (The University of Texas at Austin, 2025), is available here unchanged.
The instrument
Start with a guided, seven-topic synthetic reconstruction of the dissertation. Recalculate the published prevalence pattern, inspect the source, and download the evidence. Try fixed live OpenAI examples, or explore local discovery with the teaching mock. For your own research corpus, fork the repository or install the Python package.
What it does
The guided study explains each stage and exports its synthetic calculation. Live examples return model identifiers and token usage. The local sandbox exposes the mechanics and a validation gate. The Python workflow retains the complete research audit trail and requires actual researcher review.
When to use the Python pipeline instead
Browser discovery is a distinct approximation with different JavaScript algorithms and preprocessing; it does not establish numerical parity with Python BERTopic. For a study you intend to publish, use the reference implementation in the code section. It provides seeded search, checkpoint guards and validation artifacts. Retain and validate the exact environment used for your analysis.
The data
The dissertation's data cannot be posted openly. In their place is a synthetic demonstration corpus that has the same shape as the original and contains no real posts, accounts, or people.
5,872 posts from 11 fictional brands, Jan 2019 – Jul 2021 (post_id, brand, date, tweet).
250 synthetic examples with generator-assigned labels. These are not human annotations.
The theme and campaign behind every post, for checking your own runs.
Domain stopwords: brand terms and time words (§2.1).
An archived synthetic reference run. Stage 1 picked 20 topics at cv = 0.790, covering all five themes, and Stage 5 run on the ground truth returns Plate II.
How the corpus was built, what it is calibrated to, and what it is not.
Built to mirror the original
The corpus follows the reported total and brand structure; rounded theme prevalence matches the published table. Period sizes are synthetic calibration choices, not recovered historical denominators. Feed the ground-truth labels to Stage 5 and you get back Plate II's prevalence table and its significance pattern. Run the full pipeline and you see how close a classifier comes. The campaigns are the same kinds the dissertation found: a delivery-app promotion, stay-home sipping, a restaurant relief fund, living-room concerts, giveaways, a virtual draft party, home sports challenges, and a charity gaming tournament, all set among ordinary brand chatter.
Released under CC BY 4.0.
The generator is dtm_pipeline/make_demo_data.py, which is seeded and reproduces the files byte for byte.
Access to the dissertation data
The source data are brand tweets recorded by the Prevention Research Lab Twitter Marketing Study, and user tweets from the Computational Media Lab COVID-19 Twitter Dataset. They include account-level information and content governed by the platform's terms and by the agreements under which they were collected, so they are not posted publicly. The original records are not distributed with this tool because of privacy and research-governance constraints. Methodological questions may be directed to: jethomasphd@gmail.com.
The code
A maintained Python pipeline written against the current OpenAI SDK, with documented prompts and search spaces. Maintained changes, including a larger classification output limit, are distinguished from the historical protocol. It adds a seeded search, an enforced benchmark gate, checkpointing for corpus-scale runs, and a test suite. Out of the box it runs on the demonstration corpus.
Start with the reproduction guide. The repository includes an agent contract, installable Python package, synthetic acceptance checks, protected source hashes and explicit researcher review gates.
git clone https://github.com/jethomasphd/TopicLLM.git
cd TopicLLM
python -m pip install ".[dev]"
topicllm --profile quickcheck --output-dir runs/quickcheck
# Then read AGENTS.md and REPRODUCE.md with your agent.
# Install ".[ml,dev]" for discovery and declare a budget before API calls.Start a reproducible workflow Download Python source + data
| Component | Where | What it is |
|---|---|---|
| Reference pipeline | dtm_pipeline/ | Five stage scripts, orchestrator, configuration, demonstration-corpus generator, tests |
| Browser instrument | docs/app/ | Visual synthetic walkthrough, capped live examples, and a local teaching sandbox; research runs use Python |
| Supplement S1 | S1_bertopic_optimization.txt | Archival Stage 1 code, transcribed from the dissertation appendices |
| Supplement S2 | S2_democratic_topic_naming.txt | Archival democratic naming (openai==0.27.0) |
| Supplement S3 | S3_llm_classification.txt | Archival corpus-scale classification (openai==0.27.0) |
Citation
Cite the methods preprint for the pipeline and the software for the code. DOIs will be
added here when the preprints are deposited. The repository also carries a
CITATION.cff
that GitHub reads.
Thomas, J. E., Murthy, D., Pinedo, M., Loukas, A., & Pasch, K. E. (2026). Coherence-optimized topic modeling with democratic large language model interpretation: A human-in-the-loop pipeline for public health surveillance of social media [Preprint].
Thomas, J. E., Murthy, D., Pinedo, M., Loukas, A., & Pasch, K. E. (2026). Alcohol marketing on Twitter during the COVID-19 pandemic: Theme identification, prevalence comparison, and sentiment analysis [Preprint].
Thomas, J. E., Murthy, D., Pinedo, M., Loukas, A., & Pasch, K. E. (2026). Alcohol information dynamics during crisis: A longitudinal analysis of bidirectional influence between alcohol industry marketing and user-generated content on Twitter during the COVID-19 pandemic [Preprint].
Thomas, J. E. (2026). TopicLLM: Coherence-optimized topic modeling with democratic large language model interpretation [Computer software]. https://github.com/jethomasphd/TopicLLM
Thomas, J. E. (2025). Alcohol marketing on social media during the COVID-19 pandemic: Historical perspectives, modern evidence, and future regulation [Doctoral dissertation, The University of Texas at Austin].@misc{thomas2026topicllm_methods,
author = {Thomas, Jacob E. and Murthy, Dhiraj and Pinedo, Miguel and Loukas, Alexandra and Pasch, Keryn E.},
title = {Coherence-Optimized Topic Modeling with Democratic Large Language Model Interpretation: A Human-in-the-Loop Pipeline for Public Health Surveillance of Social Media},
year = {2026},
note = {Preprint},
url = {https://topicllm.dev/}
}
@misc{thomas2026alcohol_marketing,
author = {Thomas, Jacob E. and Murthy, Dhiraj and Pinedo, Miguel and Loukas, Alexandra and Pasch, Keryn E.},
title = {Alcohol Marketing on Twitter During the COVID-19 Pandemic: Theme Identification, Prevalence Comparison, and Sentiment Analysis},
year = {2026},
note = {Preprint}
}
@misc{thomas2026information_dynamics,
author = {Thomas, Jacob E. and Murthy, Dhiraj and Pinedo, Miguel and Loukas, Alexandra and Pasch, Keryn E.},
title = {Alcohol Information Dynamics During Crisis: A Longitudinal Analysis of Bidirectional Influence Between Alcohol Industry Marketing and User-Generated Content on Twitter During the COVID-19 Pandemic},
year = {2026},
note = {Preprint}
}
@software{thomas2026topicllm,
author = {Thomas, Jacob E.},
title = {TopicLLM: Coherence-Optimized Topic Modeling with Democratic Large Language Model Interpretation},
year = {2026},
url = {https://github.com/jethomasphd/TopicLLM},
license = {MIT}
}
@phdthesis{thomas2025dissertation,
author = {Thomas, Jacob E.},
title = {Alcohol Marketing on Social Media During the COVID-19 Pandemic: Historical Perspectives, Modern Evidence, and Future Regulation},
school = {The University of Texas at Austin},
year = {2025}
}