DocsBlogSpecAbout
fabric ·nu.llm

One chat wire. Every provider.

Declare a ChatRef on a Service, bind a preset, call it like any other Ref. Supports Ollama, OpenAI, OpenRouter, Groq, Cerebras, xAI, and vLLM out of the box.

Read the referenceSee the code

See it.

A Ref that talks to a chat/completions endpoint. Sync or async, one shot per call, message list or a plain prompt=.

Chat as a Ref, providers as presets.

ChatRef.method(...) declares an endpoint on the Service. Defaults set once at declaration; per-call kwargs override them.

Swap nu.llm.ollama(...) for openai, openrouter, groq, or your own vllm server. The call site does not change.

bot.py
python · 11 locpy
import nu
class Model(nu.Service):
chat = nu.llm.ChatRef.method(temperature=0.7)
# ollama running locally, one preset call
app = nu.With(
nu.llm.ollama(Model, host="localhost", model="qwen2.5:7b-instruct"),
body=nu.print(nu.dict(Model.chat(prompt="haiku about rust"))["text"]),
)
nu.run(app)

What you can do with it.

One wire covers most of the model market. Design against the Ref, choose the backend later.

one wire

Every provider that speaks OpenAI-compat.

Ollama, OpenAI, OpenRouter, Groq, Cerebras, xAI, vLLM. Preset fills base_url + api_key + model. Same ChatRef on top.

swap in place

Change providers without touching call sites.

Prototype on Ollama, ship on OpenRouter, benchmark on Groq — one line at bind time. The Service and its calls stay identical.

sync + async

Async when it matters.

LLM calls are network-bound. Prefer nu.arun so parallel prompts, streaming, and UI ticks all keep going while the model thinks.

Combines well with.

nu.llm gets you text. Other fabrics hold the surrounding state, ship it to a UI, or fan it across machines.

nu.kv

Durable state fabric.

Persist prompts, responses, embeddings, conversation history — same Refs, plain Python reads and writes.

nu.ui

Reactive web UI.

Wire a chat Ref to a text block and a button. Type a prompt in the browser, get the answer streamed back.

Try Nu.

One command gets you the wheel with every fabric. Then follow the movies tutorial to build a real app in an afternoon.

01 Install

pip install "nustack-py[all]"

02 Run the demo

nu demo movies

03 Build your app

Browse examples

Like what you see?

The project is young. Star it, join the room, watch what we ship next.

Star on GitHubnustackdev/nuJoin DiscordTalk to the team, share what you build.Follow updatesShip notes and small demos on X.