AI

  • Fascinating read from the team at fly.io on their experiences of building out a GPU based infrastructure in their cloud offering. The short version is they have not found any demand for an inference product across their customer base. The larger story is that they see developers trying to use LLMs, but its all back to the providers. This is interesting but I think it reflects much more on their customer base, which is very developer focused, rather than broader market demand, which is, growing. Cloudflare, for example, indicated they are going to increase spend on AI Inference in their Q4FY24 earnings call on Feb 6th.

  • Gartner published a lovely note on Emerging Patterns for Building LLM-Based AI Agents. Subscriber only, but if you have a Gartner seat spend an hour on it. Its easily the best summary of the various patterns I have seen.

General Interest

  • The Go Compiler just got a whole lot more WebAssembly capabilities with a new go:wasmexport directive. Posts on the go.dev site and a bit more on Googles blog. I mentioned my past WebAssembly research at Gartner recently. In the few client inquiries I did get on WebAssembly, Go featured a lot. But that reflected the audience, who were already pretty advanced for Gartner. Personal take, but this is very cool to see.

  • Google released really interesting research on Designing Sustainable AI, in which they introduce a new metric Compute Carbon Intensity (CCI). I haven’t read the entire academic paper yet, its in my weekend reading stack, but I have stuck it into notebooklm, and included the summary that Google generated on their own research below (it also created a really nice briefing doc, but doesn’t export markdown). Now this is also interesting on a more macro level, various companies are looking at more efficient approaches to AI. Groq is one that grabs my eye, not least because its founder is one of the original TPU designers. But there are a whole bunch of interesting companies working in this space right now. That NVidia moat is not as insurmountable as people think in the medium term.

  • Broadcom and TSMC are rumoured to be in talks on how to carve up Intel. If this goes through it is a sesmic moment for the semiconductor industry.

Data

  • Adaptavist, an agile consultancy, have an interesting report on the aftermath of the Crowdstrike outage last July. Its focused on how software development practices will change, which is, to put it politely, a bit of a stretch from an outage caused by an end point security product. That said there are some nice data points on the lack of incident response plans prior to Crowdstrike (84%) and on investments into hiring in QA and other areas.

This study introduces a comprehensive life-cycle assessment (LCA) of greenhouse gas (GHG) emissions from Google’s Tensor Processing Units (TPUs), covering everything from raw material extraction to disposal and energy consumption. A key contribution is the new metric, compute carbon intensity (CCI), which normalizes emissions by computational performance. The research demonstrates that CCI improves significantly with each TPU generation, particularly from TPU v4i to TPU v6e (Trillium), driven by hardware design and manufacturing efficiency. The study also highlights the importance of both embodied and operational emissions, showing that operational emissions can be substantially mitigated by carbon-free energy (CFE) procurement. It also advocates for more accurate carbon accounting methods, like 24/7 CFE matching, to accelerate grid decarbonization and promote sustainable AI hardware development.