GoMLX and Hugot: Expanding the Horizons of Machine Learning in Go

“Machine learning should be available in every language, just like databases are. Go deserves a first-class ML—and we’re building it.” – Jan Pfeifer

For years, machine learning in production has often meant one language for research (Python) and another for deployment (Go, C++, or Java). The interface between them has always been painful. Exported models, awkward microservice wrappers, oversized containers, brittle dependencies—it’s a familiar story for anyone trying to bridge the gap between Python’s research-first ecosystem and Go’s production-first environment.

Two projects are rewriting that story: GoMLX and Hugot. Together, they are expanding the native Go ecosystem for machine learning in a way that doesn’t just imitate Python’s approach, but leverages the added benefits of Go’s clarity, type safety, and production-ready simplicity.

GoMLX: The Foundation

At its core, GoMLX is a lower-level, general-purpose machine learning framework for Go. Think of it as Go’s equivalent of PyTorch—a powerful backend that handles the heavy lifting of forward and backward propagation, training, and optimization.

Jan is the creator of GoMLX and a veteran of machine learning since the late ’90s. His experience at both Yahoo! and as a research engineer at Google has given him hands-on experience with some of the most challenging deployment patterns for machine learning. Jan described his motivation for moving away from Python:

“TensorFlow’s APIs were a constant source of frustration, and Python’s lack of typing and "there's more than one way to do it" philosophy, while empowering for writing quick code, complicates maintenance. Its performance bottlenecks often required switching languages even for input pre-processing.

With GoMLX I aim to build a framework that is: (1) readable, predictable and easy to reason about; (2) Extensible and easy to experiment and implement new research in ML; (3) Reliable and robust for production use. “

GoMLX leverages XLA (the same backend used by TensorFlow and JAX), and recent work is bringing support for dynamic shapes via StableHLO, meaning developers won’t be locked into rigid input dimensions for text or sparse datasets.

While still maturing, GoMLX is already delivering competitive performance: shared backends mean speed parity with Python frameworks, and future Go updates (like SIMD) promise even more gains.

Hugot: The Abstraction Layer

If GoMLX is PyTorch, then Hugot makes using machine learning models as easy as the widely used Huggingface Transformers. Hugot sits one layer above, offering pipelines for embeddings, text generation, image classification, and other ML tasks that can be integrated into go applications with minimal boilerplate.

“Our mission with Hugot is simple: we want engineers to write three lines of Go and get a working LLM pipeline—no REST wrappers, no giant containers, no juggling Python in production.” - Riccardo Pinosio

Hugot supports multiple backends—XLA, ONNX Runtime, and a pure Go backend. That last piece is crucial: by removing C dependencies, Hugot can cut container sizes from tens of gigabytes down to something manageable, while also offering simpler deployments and better integration into Go-native codebases.

Why This Matters

Both Hugot and GoMLX share a vision: eliminate the microservice bottleneck for machine learning.

Python-first approaches often push teams into awkward architectures where Go backends communicate with Python ML services via REST or gRPC. That works at a small scale, but once you’re aiming for tens of thousands of inferences per second, the overhead becomes prohibitive. It also adds considerable complexity to deployments.

By bringing ML inference directly into Go applications, these projects unlock:

• Simplicity: Fewer dependencies, containers, languages and toolsets to maintain and support.

• Developer Velocity: Go’s type safety means faster debugging and fewer runtime surprises.

• Portability: it runs everywhere Go runs, including in applications for the Web, using WebAssembly (WASM).

“A Go backend is performant enough for a broad range of ML applications, and it keeps improving. With the upcoming SIMD support in Go, it should broaden even further.” -Jan Pfeifer

Of course, one could remove the dependency on Python by relying on cloud services such as OpenAI. However, this doesn't suit everyone for a variety of reasons:

1. If sensitive data is processed this can require sending the data to external model providers.

2. You lose control on the exact model version that is deployed.

3. You need to manage the associated deployments and cost overheads.

Hence a need for a self-hosted, but intuitive framework to manage and run models inside your applications.

Building Together

The parallels with the Python ecosystem are striking: GoMLX as the PyTorch equivalent, Hugot as the Transformers equivalent. But in Go, these projects aren’t just cloning—they’re reimagining what ML in production could look like if designed for deployment from day one.

At Knights Analytics, we believe so strongly in this vision that we’re excited to share some news: Jan is joining Knights Analytics as an advisor! His deep experience building GoMLX and his long history in machine learning align perfectly with our mission to make Hugot the go-to library for LLMs and modern ML in Go.

“The real challenge isn’t just building libraries—it’s creating critical mass. Adoption, contributions, and real-world use cases are what will turn GoMLX and Hugot into the foundation of Go’s ML ecosystem.” – Rob Keevil

Key Features and Recent Developments

Both GoMLX and Hugot are evolving at a rapid pace, with features that directly address long-standing frustrations in deploying ML to production:

• Pure Go Backend (SimpleGo): GoMLX now ships with an experimental pure Go backend, affectionately called simplego. 

  
  

This is a huge step forward. By removing C dependencies, deployments are simplified, container sizes shrink, and developers can avoid C compilation nightmares. While currently about 5x slower than XLA on average models, it performs competitively on smaller workloads and offers a frictionless path for teams who value simplicity and maintainability over squeezing every drop of performance. 

“GoMLX uses XLA as a backend engine, but just recently I’ve been implementing a pure Go backend (simplego), so there is no more need for C for many use cases.” – Jan Pfeifer

• SIMD Acceleration in Go: The performance story doesn’t stop there. Upcoming support for SIMD in Go is expected to bring 3–10x performance improvements to the pure Go backend. For many more use cases, the pure Go path will be fast enough to make it the default choice.

• Generative pipelines in Hugot: Hugot has recently released a Text Generation pipeline—the equivalent of Hugging Face’s popular TextGenerationPipeline. This will unlock a wide range of generative use cases, from building conversational chatbots to running large-scale content generation pipelines in Go. 

  
  

In addition, new pipeline types have been released such as CrossEncoders and ImageClassification,  further expanding Hugot’s reach into ranking, search, and sentence-pair classification scenarios.

“ Until now, we’ve focused on embeddings and classification tasks. With text generation, Hugot becomes a real gateway to LLM-powered applications in Go.” – Rob Keevil

• StableHLO and Quantization: Work is also underway to use StableHLO as an intermediary representation that can be directly fed to XLA’s PJRT. It removes more C/C++ dependencies, it’s more debuggable (there are more performance optimization tools available) and adds support for polymorphic shapes which could allow for more versatile exported models. On the efficiency front, quantization support for 4-bit models is planned, opening the door to dramatically smaller, faster models that are easier to deploy at scale.

  
  

Final Thoughts

Python may have been the first mover in machine learning, but Go offers something Python can’t: a direct path from idea to scalable and reliable production.

With GoMLX and Hugot, the Go ML ecosystem is no longer a dream—it’s here, it’s fast, and it’s growing.

“Machine learning should be available in every language, just like databases are. Go deserves a first-class ML—and we’re building it.” – Jan Pfeifer

Despite the promising developments, in both GoMLX and Hugot the challenge in scaling the frameworks will be driven by more adoption to scale up development. A larger community can help accelerate both libraries with contributions or feedback to help refine the frameworks – so reach out if you would like to help grow and contribute to this ongoing work!

Hugot can be found here: https://github.com/knights-analytics/hugot

GoMLX can be found here: https://github.com/gomlx/gomlx