https://docs.asrafuli.com/docs/projects/Recent content in Projects on My Random DocsHugoen-ushttps://docs.asrafuli.com/docs/projects/llm-gateway/Mon, 01 Jan 0001 00:00:00 +0000https://docs.asrafuli.com/docs/projects/llm-gateway/<h1 id="llm-gateway">LLM Gateway<a class="anchor" href="#llm-gateway">#</a></h1> <h2 id="purpose">Purpose<a class="anchor" href="#purpose">#</a></h2> <p>Build a gateway layer that sits in front of one or more large language model APIs, providing a unified interface for routing, authentication, rate limiting, logging, and cost control.</p> <h2 id="status">Status<a class="anchor" href="#status">#</a></h2> <blockquote class='book-hint '> <p>Active</p> </blockquote><h2 id="goals">Goals<a class="anchor" href="#goals">#</a></h2> <ul> <li>Provide a single endpoint that abstracts multiple LLM providers (OpenAI, Anthropic, etc.)</li> <li>Enforce API key management and per-consumer authentication</li> <li>Track usage, costs, and latency per consumer and model</li> <li>Support request routing rules (e.g. route by model name, fallback on error)</li> <li>Enable rate limiting and budget caps per consumer</li> </ul> <h2 id="scope">Scope<a class="anchor" href="#scope">#</a></h2> <h3 id="included">Included<a class="anchor" href="#included">#</a></h3> <ul> <li>HTTP proxy layer that accepts OpenAI-compatible requests</li> <li>Provider adapters for at least Anthropic and OpenAI</li> <li>Request and response logging with token counts</li> <li>Per-consumer API key issuance and validation</li> <li>Rate limiting and monthly spend caps</li> <li>Admin interface or config file for managing consumers and rules</li> </ul> <h3 id="not-included">Not included<a class="anchor" href="#not-included">#</a></h3> <ul> <li>Fine-tuning or model hosting</li> <li>Training data management</li> <li>End-user chat UI</li> <li>Multi-region deployment (deferred)</li> </ul> <h2 id="tasks">Tasks<a class="anchor" href="#tasks">#</a></h2> <ul> <li><input disabled="" type="checkbox"> Define the API contract (request/response schema)</li> <li><input disabled="" type="checkbox"> Stand up a basic HTTP proxy that forwards to one provider</li> <li><input checked="" disabled="" type="checkbox"> Add provider adapters (Anthropic, OpenAI)</li> <li><input disabled="" type="checkbox"> Implement consumer key management</li> <li><input checked="" disabled="" type="checkbox"> Add request logging with token and cost tracking</li> <li><input disabled="" type="checkbox"> Add rate limiting per consumer</li> <li><input disabled="" type="checkbox"> Add budget cap enforcement</li> <li><input disabled="" type="checkbox"> Write admin config or dashboard</li> <li><input disabled="" type="checkbox"> Document the setup and deployment steps</li> <li><input disabled="" type="checkbox"> Test with real traffic</li> </ul> <h2 id="links-and-files">Links and files<a class="anchor" href="#links-and-files">#</a></h2> <ul> <li>Repository: <a href="https://github.com/asrafuli/gato">https://github.com/asrafuli/gato</a></li> <li>Docs:</li> <li>Notes:</li> <li>Related workflow:</li> <li>Related how-to:</li> </ul> <h2 id="decisions">Decisions<a class="anchor" href="#decisions">#</a></h2> <table> <thead> <tr> <th>Date</th> <th>Decision</th> <th>Reason</th> </tr> </thead> <tbody> <tr> <td>2026-05-10</td> <td>Start with OpenAI-compatible request schema</td> <td>Maximises compatibility with existing tooling</td> </tr> </tbody> </table> <h2 id="outcome">Outcome<a class="anchor" href="#outcome">#</a></h2> <ul> <li>A running gateway service that proxies LLM requests</li> <li>Per-consumer usage reports</li> <li>Documentation covering setup, configuration, and adding new providers</li> </ul> <h2 id="notes">Notes<a class="anchor" href="#notes">#</a></h2> <p>Capture rough notes, lessons, and things to remember.</p>https://docs.asrafuli.com/docs/projects/project-template/Mon, 01 Jan 0001 00:00:00 +0000https://docs.asrafuli.com/docs/projects/project-template/<h1 id="project-template">Project Template<a class="anchor" href="#project-template">#</a></h1> <p>Use this page as a starting point for new projects.</p> <h2 id="purpose">Purpose<a class="anchor" href="#purpose">#</a></h2> <p>Describe why this project exists.</p> <p>Example:</p> <blockquote class='book-hint '> <p>Build a repeatable documentation system for personal tools, workflows, and technical notes.</p> </blockquote><h2 id="status">Status<a class="anchor" href="#status">#</a></h2> <p>Choose one:</p> <ul> <li>Planned</li> <li>Active</li> <li>Paused</li> <li>Completed</li> <li>Archived</li> </ul> <p>Current status:</p> <blockquote class='book-hint '> <p>Active</p> </blockquote><h2 id="goals">Goals<a class="anchor" href="#goals">#</a></h2> <p>List the outcomes that matter most.</p> <ul> <li>Goal 1</li> <li>Goal 2</li> <li>Goal 3</li> </ul> <h2 id="scope">Scope<a class="anchor" href="#scope">#</a></h2> <h3 id="included">Included<a class="anchor" href="#included">#</a></h3> <ul> <li>What this project will cover</li> <li>What files, tools, or systems are involved</li> <li>What success looks like</li> </ul> <h3 id="not-included">Not included<a class="anchor" href="#not-included">#</a></h3> <ul> <li>Work that belongs somewhere else</li> <li>Ideas to save for later</li> <li>Things that are intentionally out of scope</li> </ul> <h2 id="tasks">Tasks<a class="anchor" href="#tasks">#</a></h2> <p>Use this as the working checklist.</p>