Can AI-assisted workflows really help?

October 30, 2026 RayJamesHooblerdata-visualization, python, plotnine, ggplot2, ai, artificial-intelligence, technology, llmLeave a comment

I consider myself proficient in data analysis and enjoy the challenge of producing a good visualization. I’ve experimented with AI agents over the last several years now and have had mixed results. At times, it felt like the agent was getting in my way. For a recent project, I wanted to pull data from the SEC’s Electronic Data Gathering, Analysis, and Retrieval (EDGAR) system. Working with API calls is an area where I struggle, so I was hopeful this might be a good use case.

The goal of the project was to produce a simple graphic showing gross margins of selected U.S. semiconductor companies across three sectors that could be easily reproduced, updated with future data, or used to produce similar visualizations with a different or updated company list.

Overall, I was happy with the final result (Figure below).

Line graph showing gross margin trends for Fabless companies, IDM (Integrated Device Manufacturers), and WFE (Wafer Fab Equipment suppliers) from 2010 to 2030, with notable companies indicated in each category.

At some point in the past, I would have manually pulled each company’s 10-K (annual) filing and dutifully entered the needed values into a spreadsheet as Excel was the tool I, like many, used and abused. Here, I pull data from data.sec.gov/api/xbrl/companyfacts/ using the API https://data.sec.gov/api/xbrl/companyfacts/CIK##########.json. This will provide all the company concepts data for a company returned in a single API call.

Code is available on GitHub

The basic steps:
– Get the SEC’s company identifier (CIK)
– Use the CIK in the above API call (produces a JSON file)
– Extract revenue, gross profit, and cost tags from the JSON file
– Calculate the gross margin as (revenue – cost)/revenue; some entries did not report a gross profit tag, so I compute it from the revenue and cost
– Combine each company’s data into a final data frame

The final data frame becomes the input for the visualization (created with Plotnine).

Where was the agent most helpful?

The agent was most helpful in creating functions that pulled tags for revenue, gross profit, and cost. For these companies, there were three different tags for revenue and two for cost. I’ve done similar tasks before and it has always been painful and having the AI agent write these functions was very helpful. The generated code was well-structured and reasonably commented. I’m sure my initial effort would have been less clean.

So where did I contribute? The Plotnine visualization. Plotnine is a Python implementation of R’s ggplot grammar of graphics. I’ve been working with ggplot for nearly a decade and with Plotnine since roughly 2023 when the tool seemed to reach a point of maturity and stability. I now have my favorite tool (which is built on Matplotlib functionality) in Python. This is where, frankly, I don’t need (or want) much help.

It took me a couple hours to pull everything together—without help, this would have been a good day’s worth of work.

Additional Info

“AI agent” == “Claude code”

Company names and trading symbols (ordered by revenue)

Name	Symbol
NVIDIA Corporation	NVDA
Broadcom Inc.	AVGO
Intel Corporation	INTC
QUALCOMM Incorporated	QCOM
Advanced Micro Devices, Inc.	AMD
Micron Technology, Inc.	MU
Applied Materials, Inc.	AMAT
Lam Research Corporation	LRCX
KLA Corporation	KLAC

Companies fall into three sectors: fabless, integrated device manufacturers (IDM), and wafer fab equipment (WFE) suppliers.

Color Sources:
– NVIDIA, logo.dev (https://www.logo.dev/search/brands/nvidia.com)
– AMD, Brandpalettes (https://brandpalettes.com/amd-colors/)
– Broadcom, Designpieces
(https://www.designpieces.com/palette/broadcom-color-palette-hex-and-rgb/)
– Intel, Designpieces
(https://www.designpieces.com/palette/intel-color-palette-hex-and-rgb/)
– Micron, logo.dev (https://www.logo.dev/search/brands/micron.com)
– Applied Materials, logo.dev (https://www.logo.dev/search/brands/appliedmaterials.com)
– Lam Research, Brandpalettes (https://brandpalettes.com/lam-research-colors/)
– KLA, Brand Color Code (https://www.brandcolorcode.com/kla-corporation)
– Qualcomm (https://www.designpieces.com/palette/qualcomm-color-palette-hex-and-rgb/)

Don’t worry. . . you’ll be wrong

October 28, 2026 RayJamesHooblerdata-visualizationLeave a comment

As a forecasting case study, I’ve been collecting data on silicon wafer shipments since 2019 where each October the remaining year and three future years are presented in a press release.

I don’t remember many details from the meeting where I was first asked to prepare a product forecast—I just remember the statement above. Over time I’ve became more comfortable in navigating both bottom-up and top-down forecasts and enjoy the data curation of the former and modeling of the latter.

When I talk with others (both colleagues and students), I always remind them not to worry; that they too will be wrong, but that it’s also good to remember the quote by statistician George Box, “Essentially, all models are wrong, but some are useful.” If the discussion continues for any length of time, we’ll undoubtedly fall into a discussion that addresses the usefulness of a forecast, i.e., forecasting accuracy. We know we’ll be wrong, but “how wrong” can we be and still be useful?

The global silicon wafer shipments and forecasts have been published for over a decade, and as an industry, we are fortunate this information is publicly available. I assume the forecast is a hybrid forecast using bottom-up, statistical, and judgmental methods.

But how good are the forecasts? What’s a reasonable error?

At first the results don’t look promising (Table 1).

Horizon	MAPE	RMSPE	n
0	0.84	1.20	7
1	7.77	9.59	6
2	18.36	19.73	5
3	20.36	22.37	4

Here

$MAPE = \frac{1}{n} \sum_{t = 1}^{n} \frac{∣ y_{t} - {\hat{y}}_{t} ∣}{y_{t}} \times 100$

and

$RMSPE = \sqrt{\frac{1}{n} \sum_{t = 1}^{n} {(\frac{y_{t} - {\hat{y}}_{t}}{y_{t}})}^{2}} \times 100$ n is the number of years available for the calculation.

So one year out, we’re looking at an error of 7–10%.

More interesting to me is how the forecast has gone through an inflection during this time (Table 2).

A table showing SEMI forecast error by forecast horizon, indicating the percentage difference from actual shipments for the years 2019 to 2025. Positive values indicate over-forecasting and negative values indicate under-forecasting. — Table 2: Forecast error by year, 2019-2025

Data from the original press release shows a downward trend ending in 2024 with a recovery starting last year.; this demonstrates the difficulty in capturing the magnitude and length of downturns and recoveries as errors move from negative to positive between 2022 and 2023.

Anyone who has worked in the semiconductor industry knows the industry is notoriously volatile, with forecasts being sensitive to memory pricing and capacity lags. Anyone (or organization) that’s willing to publish their forecast should be commended as having a general direction for the industry is valuable. Of course, details underlying the top-level forecast are available to further help professionals refine their own work. However, but even the top-level forecast can be used as a baseline value for independent analysis.

If you’re interested in discussing data visualization, technical education, or how we can build better analytical skills for these kinds of business contexts, I’d love to connect.

Silicon Slopes Tech Summit 2019 (Day 2)

February 2, 2019February 2, 2019 RayJamesHooblerLeave a comment

I am glad I can get out of the office and learn what companies are doing in Utah and Day 2 of the SSTS 2019 offered some more reminders of what is on the mind of tech leaders. In no particular order, here are several items I noted:

Companies are looking for individuals with domain knowledge who can reach across disciplines.
Culture eats strategy (i.e., it doesn’t matter how great your corporate strategy is if the culture of your company is terrible.)
Good companies promote a learning culture.
As a leader, get your hands dirty.
Make hiring decisions based on core competencies even if the person looks different than you.
How do customers increase revenue when they adopt your product or service?
Start by measuring something.

Unwritten rules?

October 22, 2018November 12, 2018 RayJamesHooblerLeave a comment

Working with graduate students in higher education is a satisfying experience, and it is rewarding to watch their personal growth as they progress through their program of study—once a student is in our program, I see it as my job to provide guidance and make sure they don’t get stuck. It is upsetting to the student, but also to me when they get stuck or leave the program because they fail to follow a documented process. Even though there may be a breakdown in the process, individuals are responsible for becoming knowledgeable about the requirements that impact them. But what about undocumented procedures or “rules?” I struggled this week to resolve a complicated issue and part of the problem—I figured out at the end of the process—was that I did not have all the information. There were unwritten rules.

At the height of my frustration, I thought this was a problem only found in academics; however, after giving myself time to reflect, I realize it is a problem in industry settings as well.

So my question is why? Why do we have unwritten rules?

Of course, it could be that there are no rules, just guidelines that are intended to be flexible based on the situation. This distinction is an important point. Guidelines provide flexibility for both parties in a negotiation which may not be the case with rules. Until it is written down, I would argue any process is just a guideline and negotiation is an option.

Where the rules are documented, there may be unwritten exceptions—situations that warrant deviation from the normal process. Unwritten exceptions recognize that managers (or administrators) encounter problems that deserve consideration based on the merits of the case. But why are they unwritten?

I think this is an issue of trust. Do you trust people not to abuse the exceptions? In organizations that work with high levels of trust, a defined process that evaluates exceptions remove the possibility of arbitrary application or favoritism.

Documenting exceptions provides a level of transparency and levels the playing field for those not inclined to challenge the status quo. Individuals who are reluctant to ask the question “why?” effectively lose out on options that are available to those willing to test the system. I doubt this the intent, but it can be the result.

As managers, we have a choice on how we set up the rules. I would argue that we should write them down and strive for a transparent and fair process.

Ray James Hoobler

management