Connect what you already run.
Launch vLLM or SGLang with a registry address. Servers register themselves and send heartbeats, using your existing model arguments.
Explore model servers ↗
Your cluster changes.
Your inference keeps going.
Lightweight coordination for model servers, CPU workers, and container environments.
Connect your cluster. Host your stack. Keep doing research.
pip install literegistryPlaces using literegistry
From your terminal to the whole cluster
Watch your cluster come to life.
Watch your cluster take shape.0 / 7 services connected
Less infrastructure. More research.
Let your services find their way.
Research clusters are always in motion. literegistry connects inference, execution tools, and stateful container environments through one service registry and gateway.
One gateway. Your whole cluster.
Send requests through an OpenAI-compatible API. Load balancing and automatic failover route traffic to available model replicas.
Meet the gateway ↗Built for nodes that come and go.
Use Redis, SQLite, or a shared filesystem. Monitor your services from the CLI as your deployment grows and changes.
Choose your backend ↗Turn CPU nodes into environments.
Run rootless Podman replicas across your CPU cluster. Create container sessions from your own images and execute commands through the gateway.
Explore environment hosting ↗A session that stays with you.
Session affinity sends commands to the replica that owns your container. Files and installed tools stay available across commands for the life of that session.
See the session lifecycle ↗Give your agents a tool cluster.
Register Python execution, restricted terminal pipelines, search, and cache workers alongside your models. Scale services across nodes behind a shared gateway.
Explore execution services ↗Your CPU cluster. Your environment stack.
Bring an image.
Make room for possibility.
Host stateful environments for agent workflows, coding tasks, and research tools with rootless Podman. Package the languages, dependencies, and utilities your workload needs into a compatible container image.
Start hosting environments ↗Use images with Bash and sleep available. Container sessions preserve state until they are closed or cleaned up.
ONE ENVIRONMENT · MULTIPLE COMMANDS
01
Create
Choose a container image. The gateway selects an available Podman replica.
POST /affinity/handshake02
Work
Run commands with the same affinity ID. Keep files and tools in the same environment.
POST /affinity/podman03
Close
Release the container and its routing binding when the task is done.
POST /affinity/closeCapacity-aware placement
New sessions route around workers reporting full capacity. Existing sessions stay pinned to their owning replica.
Resource budgets
Set memory, process, and session limits to keep workloads within the capacity you assign.
Automatic housekeeping
Configure idle-session cleanup and image pruning to reclaim resources across your worker pool.
From zero to connected
A small addition.
A connected cluster.
Install literegistry, then connect model servers or environment workers through a shared registry. Keep your familiar tools and workflow.
- Install with
pip install literegistry - Start a registry and launch your services
- Route requests through the gateway
Your application
Clients & agent workflows
literegistry gateway
Shared registry · service discovery
vLLM / SGLang
Inference
Inference
CPU workers
Code & tools
Code & tools
Podman
Environments
Environments
Familiar tools. Lighter coordination.
Fits into your workflow.
Coordinate GPU inference and CPU execution from the same cluster. Use local or Apptainer runtimes for model services and rootless Podman for stateful environments, alongside your Python and Slurm workflows.
Redis · SQLite · Filesystem
vLLM · SGLang · Podman · Python & terminal workers
Behind the project
Why we built literegistry.
The story behind lightweight service discovery for research clusters.
Let’s connect
Building something?
We’d love to hear.
Questions about your deployment, ideas for the project, or just want to say hello? Get in touch.
contact@literegistry.com ↗For bugs and feature requests, open a GitHub issue.
For the research community
Cite literegistry
If you use literegistry in your research, please cite the software.
@software{literegistry2025,
title={literegistry: Lightweight Service Discovery for Distributed Model Inference},
author={Faria, Gonçalo and Smith, Noah},
year={2025},
url={https://github.com/goncalorafaria/literegistry}
}