Issue #394 - The ML Engineer 🤖

Stanford Study on Local LLMs, Snorkel's Senior SWE-Bench, Google DeepMind's Tabular Foundation Model, Performance Per Dollar Improving, CVEs Spike After Mythos Release + more 🚀

Jul 05, 2026

Thank you for being part of over 70,000+ ML professionals and enthusiasts who receive weekly articles & tutorials on Machine Learning & MLOps 🤖 You can join the newsletter https://bit.ly/state-of-ml-2025 ⭐

If you like the content please support the newsletter by sharing with your friends via ✉️ Email, 🐦 Twitter, 💼 Linkedin and 📕 Facebook!

This week in ML Engineering:

Stanford Study on Local LLMs
Snorkel’s Senior SWE-Bench
Google DeepMind’s Tabular Foundation Model
Performance Per Dollar Improving
CVEs Spike After Mythos Release
Open Source ML Frameworks
Awesome AI Guidelines to check out this week
+ more 🚀

Stanford Study on Local LLMs

A study from Stanford University showed that 71.3% of chatgpt queries could be accurately answered by a local model: Stanford researchers have published a large empirical study that explores whether local LLM inference can take on part of today’s cloud-served workload. The paper defines intelligence per watt as task accuracy divided by power consumption, and evaluates 20+ local language models across 8 accelerators on more than 1M single-turn chat and reasoning queries. The main result is that local models can correctly handle 88.7% of the studied queries when routed to the best local model, although coverage is much stronger for chat and knowledge-style tasks than for harder technical reasoning. The longitudinal results are also relevant for ML platform teams as it seems 2023-2025 local-query coverage increased from 23.2% to 71.3%, while intelligence per watt improved by 5.3x. Cloud accelerators still retain a clear per-query efficiency advantage on identical models, but hybrid local-cloud routing changes the system-level tradeoff, as just with an 80%-accurate router, the simulated deployment reduces energy by 64.3%, compute by 61.8%, and cost by 59.0% against a batched cloud baseline. This is super promising, as it shows that there’s so much potential in local models that is really untapped - there’s really a lot to come from this space.

Snorkel’s Senior SWE-Bench

SnorkelAI has dropped a new SWE-Bench for LLMs that attempts to capture Senior Engineering type tasks that extend the scope towards under-specified feature requests, runtime bug investigation, behavioral verification, and code-quality assessment: It is interesting to see the rise of SWE benchmarks attempting different schools of thought to bring balanced reviews of new models; I like that this one is based on real PRs, uses a mix of pre-written verifiers, an adaptive agents to ensure a useful distinction between code that passes tests and code that fits the surrounding codebase. In the reported results, GPT-5.5 has the highest basic solve rate at 55.0%, while Claude Opus 4.8 has the highest tasteful solve rate at 24.0%; GPT-5.5 is also more efficient, averaging 36.3K output tokens and 89 agent steps per task, compared with 117.1K tokens and 131 steps for Claude Opus 4.8. For production ML practitioners building or evaluating coding-agent workflows it is worth noting that pass rates are an incomplete signal now, as teams should also measure root-cause correctness, design quality, abstraction fit, and even codebase-pattern alignment.

Google DeepMind’s Tabular Foundation Model

Google Research has now entered the Tabular Foundation Model space! They have released TabFM, a tabular foundation model for classification and regression, which seems to mostly use learnings from the first-mover startups in the space (but still awesome): This model basically treats each table as an in-context learning task (instead of fitting a dataset-specific model), and this is useful because TabFM instead combines row and column attention, row compression and an ICL Transformer, with pre-training on hundreds of millions of synthetic datasets generated from structural causal models. Google reports strong TabArena results across 51 classification and regression datasets, including a zero-shot setting that runs in a single forward pass and an ensemble variant with cross features, SVD features, NNLS blending and calibration. For production ML practitioners, the most immediate value is in fast baselines and prototyping clearly. However, unfortunately it seems that the public weights are non-commercial (boo!), classification is limited to 10 classes, memory scales with the number of context rows, and (obviously) domain-specific validation remains necessary before use in critical systems.

Performance Per Dollar Improving

Performance per dollar in local LLMs is getting faster and cheaper! Wafer shows how they served GLM5.2 on AMD MI355X at 2626 tok/s/node and 213 tok/s single stream at over 2x lower cost than Blackwell. This is an interesting case study that showcases how much opportunity there is on local model inference efficiency; 2.4 RPS on a 20k input / 1k output workload with a 60% cache-hit rate, and 213 tok/s single-stream on a 10k input / 1.5k output test is impressive. The result is relevant for production ML teams because the gains came mostly from framework and configuration work rather than new custom kernels: Wafer quantized bf16 GLM-5.2 to MXFP4 with AMD Quark, selected sglang after testing vLLM and ATOM, fixed ROCm-specific speculative decoding issues, enabled FP8 KV cache, and tuned MoE kernel selection for GLM’s fp4 shapes. Vercel has also made GLM 5.2 Fast via Wafer available through AI Gateway, with its own benchmarks reporting higher throughput than other serverless GLM-5.2 providers across small-context, large-context, and tool-call scenarios.

CVEs Spike After Mythos Release

Disclosure of serious cyber vulnerabilities spiked around the release of Claude Mythos Preview: is it time to rethink public vulnerability reporting/mgmt? Epoch AI reported that public disclosure of high/critical severity CVEs have been spiking right after Anthropic announced Claude Mythos Preview. In June 2026 the 21 major organizations tracked by Epoch published around 1.5k high/critical severity CVEs which is more than 3.5 times the previous monthly record before Mythos Preview was announced. Anthropic’s related Project Glasswing update claims that roughly 50 partners have found more than 10k high/critical severity vulnerabilities, while also stating that the limiting factor has shifted from discovery to verification, disclosure, patch development, and deployment. OpenAI’s Daybreak initiative describes a similar defensive workflow, focused on finding, validating, and fixing vulnerabilities before attackers can use them. It seems that the dawn of agentic cybersecurity has come, and we’ll be likely seeing quite a few shifts in mindset, approach and criticality to security across every domain and industry.

Upcoming MLOps Events

The MLOps ecosystem continues to grow at break-neck speeds, making it ever harder for us as practitioners to stay up to date with relevant developments. A fantsatic way to keep on-top of relevant resources is through the great community and events that the MLOps and Production ML ecosystem offers. This is the reason why we have started curating a list of upcoming events in the space, which are outlined below.

Events we are speaking at this year:

Signals Conference - September @ Berlin
World Summit AI Europe - September @ Amsterdam

Other relevant events:

KubeCon Europe - March @ Amsterdam
PyData Berlin - April @ Frankfurt
Databricks Summit - June @ San Francisco
World Developer Congress - July @ Berlin
EuroPython 2026 - July @ Prague
EuroSciPy 2026 - July @ Krakow
AI Infra Summit 2026 - Sept @ California
Code.Talks 2026 - Nov @ Hamburg
MLOps World 2026 - Nov @ Austin

Open Source MLOps Tools

Check out the fast-growing ecosystem of production ML tools & frameworks at the github repository which has reached over 20,000 ⭐ github stars. We are currently looking for more libraries to add - if you know of any that are not listed, please let us know or feel free to add a PR. Here’s a few featured open source libraries that we maintain:

SARC - Provides wrappers for popular agentic frameworks to enable guardrails and constraints that are enforced through the flow.
KAOS - K8s Agent Orchestration Service for managing the KAOS in large-scale distributed agentic systems.
Kompute - Blazing fast, lightweight and mobile phone-enabled GPU compute framework optimized for advanced data processing usecases.
Production ML Tools - A curated list of tools to deploy, monitor and optimize machine learning systems at scale.
AI Policy List - A mature list that maps the ecosystem of artificial intelligence guidelines, principles, codes of ethics, standards, regulation and beyond.
Agentic Systems Tools - A new list that aims to map the emerging ecosystem of agentic systems with tools and frameworks for scaling this domain

Please do support some of our open source projects by sharing, contributing or adding a star ⭐

About us

The Institute for Ethical AI & Machine Learning is a European research centre that carries out world-class research into responsible machine learning.

Check out our website

The Machine Learning Engineer

Ready for more?