Inference

One-shot generation

pip install -e '.[train]'
tidelm generate \
  "Explain binary search in exactly three bullets." \
  --max-new-tokens 128 \
  --metrics

This currently runs the audited upstream base because no TideLM adapter is stable. After promotion:

tidelm download adapter --destination models/tidelm
tidelm generate \
  --adapter models/tidelm \
  "Return JSON with one key named status."

Chat

tidelm chat --adapter models/tidelm

Type /quit to exit. The CLI is intentionally local and does not upload prompts.

Python

from pathlib import Path
from tidelm.inference import generate_response, load_model

model, tokenizer = load_model(
    adapter=Path("models/tidelm"),
    device="cpu",
    dtype="bfloat16",
)
text, metrics = generate_response(
    model,
    tokenizer,
    "Explain a failing unit test in two concise steps.",
    generation={"max_new_tokens": 128},
)
print(text)
print(metrics)

Thinking mode

SmolLM3 supports hybrid reasoning. TideLM disables thinking by default for compact deterministic diagnostics. Opt in with --think. Do not expose hidden reasoning text as a correctness guarantee.

Memory and quantization

The pinned upstream BF16 payload is 6.15 GB; runtime memory is higher. For small machines, use an upstream SmolLM3 quantized variant and a compatible PEFT/llama.cpp adapter conversion. Quantized quality and speed must be evaluated separately; TideLM does not call an untested quantization a release artifact.

Safety

Verify SHA-256 checksums. Load only SafeTensors. Keep remote code disabled. Model output can be false or unsafe, and generated code must run in a disposable sandbox—not merely the evaluation harness's lightweight subprocess limits.