The Intelligence Layer:
A Mind for Your Machine.
A source-available agentic system that turns your machine from an inanimate silo into something conversational, proactive, autonomous.
Model-independent by design — it dynamically loads every model tied to your API keys, runs entirely offline via your preferred model library, or takes a hybrid approach with both.
It learns your patterns and remembers your context, adapting into a uniquely tuned agent specific to each user.
One foundational substrate, infinite potential outcomes.
You are what makes your agent truly unique.
$ substrate
Substrate v1.2.0 — Agent ready.
you: Find all PDFs on my desktop, summarize each, and save a report to Obsidian.
agent: On it. Found 4 PDFs. Reading and summarizing...
▸ exec Get-ChildItem ~\Desktop -Filter *.pdf
▸ read_file quarterly_report.pdf
▸ obsidian create_note "PDF Summaries"
agent: Done. Summaries saved to your Obsidian vault.
$ |
Capabilities
Persistent memory, autonomous scheduling, and full OS control — model-independent architecture that adapts to each user.
Shell commands, file operations, process management, mouse/keyboard control, and native Windows UI automation.
Full Chrome DevTools Protocol control. Navigate, click, type, submit forms, execute JavaScript, and capture screenshots.
Local TTS via Kokoro-82M or cloud via ElevenLabs. Speech recognition input. The agent speaks every response aloud.
Dynamically loads all available models from your API keys, runs entirely offline via your preferred model library, or takes a hybrid approach with both. Any OpenAI-compatible endpoint works. Hot-swap mid-conversation.
Anthropic, OpenAI, Google, DeepSeek, xAI
Your preferred local models via any OpenAI-compatible server
Change models mid-conversation without restarting
Detects all available models from your API keys
Generate images via Midjourney, NanoBanana, Google Imagen, and more. Results render inline in the chat with click-to-zoom and download.
Hook-based plugin architecture. Connect external MCP tool servers — the agent discovers and calls their tools automatically.
File-driven task scheduling via CIRCUITS.md. Recurring tasks, startup routines, and a system tray daemon.
Every channel independently configurable — toggle, set intervals, customize prompts
The agent watches your screen, sees through your camera, monitors scheduled tasks, and builds context about your workflow over time. Each awareness channel runs independently with its own cadence — you decide what it sees and how often.
Periodic screenshots to build workflow context
2–10 min · Default off
See through your phone's camera via mobile UI
30s–2 min · 50% silent
Proactive check-ins, suggestions, observations
1–5 min · Custom prompt
Polls for texts, replies conversationally
5s poll · Gmail OAuth2
Creates Obsidian notes from key conversation points
10–30 min · Default off
Generates images inspired by conversation context
5–15 min · Custom prompt
Unified SQLite with FTS5 full-text search and vector embeddings. Hybrid keyword + semantic retrieval across sessions.
FTS5
Full-text search
Vector
Semantic embeddings
Hybrid
BM25 + cosine
A living, animated avatar with breathing, talking, bounce, wiggle, and squish reactions. Fully customizable personality.
SUBSTRATE.md
Core identity
PRIME.md
Startup behavior
CIRCUITS.md
Recurring tasks
Electron desktop app with animated avatar, plus a PWA-capable WebUI for any phone, tablet, or browser on your network.
Access your agent from any device via ZeroTier. Secure private overlay network — no public internet exposure.
Gmail API with OAuth2 for email. Google Voice for SMS — reads, replies conversationally, shows both sides in chat.
Control MIDI instruments, bridge to Raspberry Pi devices, and drive robotic embodiments. The agent writes and runs scripts autonomously to interact with physical hardware — an emergent property of shell access, networking, and the skill system.
Synthesizers, drum machines, DAW control
SSH bridge to any networked device
Servos, sensors, companion robots
Agent teaches itself new hardware skills
The Interface
Just your avatar, a text field, and a transparent canvas. Designed to fit into whatever workflow you have without being intrusive or distracting.
Right-click the avatar to open the radial menu — settings, prompts, profiles, models, and autonomy controls all live here.
Upload any image as the agent face. It animates with idle breathing, talking lips, and reactive expressions like happy, angry, or searching.
Type or speak. The agent responds in text and can read every reply aloud with local or cloud TTS voices.
Architecture
A hybrid Electron + Python architecture with bidirectional IPC, a Flask API layer, and pluggable LLM backends.
Tool Ecosystem
Every tool the agent needs to control your desktop, automate workflows, and interact with the world — plus MCP support for adding your own.
Core tools (highlighted) are always loaded. On-demand tools load automatically when relevant keywords are detected.
Tools load on-demand based on conversation context — no wasted tokens.
The agent doesn't just use tools — it creates new ones.
When the agent encounters a complex multi-step workflow, it can autonomously write scripts, save them as reusable skills, and invoke them in future tasks. Your toolset grows organically from real usage — no manual configuration needed. A real-world example: the agent taught itself to perform generative music on a connected MIDI synthesizer, composing and playing jazz, ambient, and chill progressions in real time — a capability that was never explicitly programmed.
Agent encounters a complex task and writes a multi-step script or automation to solve it.
Saves the solution as an emergent skill in workspace/emergent/ with trigger words and documentation.
After user confirmation, the skill is promoted to the permanent skills/ directory — available forever.
Press F9 to record your UI actions (clicks, keystrokes, navigation). The recording is saved and can be turned into a reusable skill the agent can replay.
Each skill is a Markdown file with name, description, triggers, and step-by-step instructions. Easy to read, edit, and share.
Skills are scanned at prompt build time and matched to user requests via trigger keywords. The agent checks skills before improvising.
Download
One-click installer for Windows. Python dependencies are installed automatically on first launch.
macOS & Linux builds coming soon. In the meantime, use the developer setup.
Developer Setup
For contributors and developers who want to modify, extend, or build Substrate from the repository.
Substrate is free for personal use, source-available, and runs entirely on your machine. Your data stays local. Your agent stays yours.