$ cat projects/ai-framework-rust.md
NeuralFrame — AI Framework in Rust
A Rust workspace for LLM apps: HTTP server, five providers with failover, HNSW vector search, agents.
// why: Python AI stacks force a perf vs safety trade — Rust doesn't.
$ cat projects/ai-framework-rust.md▌
TL;DR
- One workspace: HTTP server, LLM layer, vector search, agents and cache
- Five providers: OpenAI, Anthropic, Gemini, Ollama, Groq with failover
- Python-friendly: PyO3 bindings, Criterion benchmarks, cargo-audit CI
// what does it do?
A 9-crate Rust workspace for LLM applications: radix-tree HTTP routing, five LLM providers (OpenAI, Anthropic, Gemini, Ollama, Groq) with automatic failover, a from-scratch HNSW vector index with write-ahead-log persistence, four memory backends, an agent runtime, semantic caching, JWT / API-key auth and PyO3 Python bindings.
// why this project?
Existing Python AI frameworks give up either performance or memory safety. In Rust you don't have to pick one.
// what broke / learned
Currently extending it — async vector storage was the rabbit hole.
// results
- About 13.6K lines of Rust across nine crates (core, llm, memory, agents, vector, prompt, cache, auth, py). The vector crate implements HNSW itself (no external index library) with metadata filters and WAL + snapshot recovery
- memory can be in-memory, SQLite, Redis or Postgres. Criterion benchmarks cover vector operations, and GitHub Actions runs CI plus a cargo-audit security workflow. PyO3 bindings let existing Python teams adopt it incrementally instead of rewriting everything at once.
Tech stack