NIST/SEMATECH e-Handbook of Statistical Methods (Tidyverse Edition)

I’ve used the NIST/SEMATECH e-Handbook of Statistical Methods for nearly 20 years. First, as an applications scientist working for a semiconductor capital equipment company and later as an educator teaching a graduate level applied statistics course. A guiding principle of the book is to make the examples and case studies accessible with most allowing the reader to work out the problems by hand. While the book promotes the use of NIST’s Dataplot software, it was straight forward to adapt the case studies to R. Overall, this online book is a valuable resource.

My biggest complaint? The format—which uses dedicated HTML pages for each subsection of each chapter. This makes the book hard to navigate when you know what you’re looking for and nearly impossible to browse when you don’t.

Over the last year, I, like many people have been experimenting with LLMs and was curious if I could leverage an LLM to convert this e-Handbook into a Quarto book. After a month of work (okay, five weeks—which was more than I expected), I have a working Quarto book on GitHub. What I refer to as the “Tidyverse Edition.”

My goal was to stay faithful to the original text and only update the in-text examples, graphics, and case studies with reproducible R code that leans heavily on the tools of the Tidyverse (dplyr, ggplot2, etc.). That said, I did try to correct minor typos or inconsistencies when found and document more serious issues (which were few) with call-out notes where appropriate.

The original e-Handbook was released into the public domain and this effort follows their convention.

Was this a worthwhile project? Yes, I think so. Over the last several years there have been several instances where, during a semester, the NIST website was unavailable due to a government shutdown. While an archived version of the e-Handbook is available for download, this is less than idea for most students. Additionally, there seems to be a risk that this type of government work may be removed from public view if it’s deemed unnecessary or no longer considered a government responsibility.

Was this project appropriate use of an LLM? Definitely. I estimate that manually editing the original document, even with tools like pandoc, the project would have become a part-time endeavor taking many months, or years instead of a full-time effort over five weeks.

Finally, I hope others might be inspired to use this repository as a launching point for their own projects. Several of the chapters could easily become their own dedicated books and there are opportunities to build new case studies from other industries.