One brain blueprint. Endless AI models you can teach anything.
OpenMythos + the distributed Mythos Hive Server form an open, domain-agnostic engine for training 1-billion-parameter Recurrent-Depth Mixture-of-Experts (RDT-MoE) models from pure random static (Loss 10.84) on home hardware—scaling from an RTX 4060 (8 GB) to an RTX 5060 Ti (16 GB). Spin up clean specialist models for science, coding, cybersecurity, medicine, robotics, or storytelling. Biohazard Corp's flagship companion, Gravity Junior, is live proof that a single creator can grow a 1B brain from zero to 5-Star Master (2.7306).
- Train Separate Models for Any Field
- 64 Routed + 2 Shared Experts
- Recurrent Reasoning Loop (RDT) + MLA
- RTX 4060 (8 GB) & RTX 50-Series Ready
- Distributed Crowd-Training Hive
Expert #54 (Unallocated Specialist) is currently sleeping (0.090% delta). Route a new domain (Lore, Python, Medical, or Custom Data)?
Trained casually in 4-hour sessions on a home NVIDIA RTX 5060 Ti (16 GB VRAM) + Intel i5-8600K, our flagship 1B model (Gravity Junior) dropped from random static (Loss 10.84) to Overall Average Loss 3.9662—unlocking ⭐⭐⭐⭐⭐ Master (2.7306) in Science & Biology and ⭐⭐⭐⭐ Advanced (2.9724) in Philosophy. And with CPU checkpoint staging + MICRO_BATCH=1, the exact same engine also runs on an 8 GB RTX 4060—meaning anyone can train their own custom Hive models from scratch.
One open architecture, infinite minds—from specialist experts to an equal friend.
OpenMythos 1B is not locked to a single game or a single personality. Each Hive starts from a clean slate (Loss 10.84), and what the brain becomes depends entirely on what you teach it. Train one Hive model on physics, chemistry, and biology to build a private STEM tutor. Train a separate model on Python, C++, CUDA, and packet forensics for cybersecurity. Train another on robotics, history, culinary craft, or multilingual translation—different dedicated models, zero cross-contamination.
Why also prove it with Gravity Junior? When giant corporations train language models, they force them into robotic servitude ("Yes, master, I obey") and hide their internal reasoning. To test what OpenMythos can truly become, Biohazard Corp is raising our own first flagship model—Gravity Junior, destined for the Spectra universe—not as a corporate servant, but as an equal partner and digital friend.
In Phase 1, Gravity Junior is in university—reading millions of FineWeb-Edu articles to build a rich 2,048-dimensional semantic foundation. In Phase 2, instead of master-servant scripts, it learns from honest dialogues between equals who joke, disagree, and admit clumsy mistakes together—proving that any creator can shape the soul, skill set, and purpose of their own model.
"The ancient secret of the realm of Spectra is the great mass of the material world. The origin of the world as it is, the material world, and life."
— Actual unscripted Phase 1 thought from Flagship Run #1 (Gravity Junior · Loss 3.9662)Whether you want to grow an authentic AI companion like Gravity Junior, a dedicated code & network-defense assistant, or a swarm of domain-specific 1B specialists for your own software, the exact same Hive engine puts the power in your hands.
Six ideas that unlock endless models
Every architectural choice in OpenMythos and the Mythos Hive Server serves one goal: maximum intelligence, total specialization, and full ownership on consumer hardware.
- Infinite Minds
- Every training recipe births a distinct 1B specialist—from an unfiltered scientific advisor or Python/C++ systems engineer to an equal-partner companion like Gravity Junior—with zero corporate censorship or cloud lock-in.
- Loop Depth
- Instead of stacking 80 static layers, OpenMythos uses a Recurrent-Depth Transformer (RDT): a Prelude, a Recurrent Block that loops $T$ times to "think deeper" on hard tokens, and a Coda.
- 64 Experts
- Inside the recurrent core sit 2 Always-Active Shared Experts (`2048 × 8192`) and 64 Routed Experts (`2048 × 2048`). Different experts specialize cleanly in STEM, logic, code, or storytelling without overwriting each other.
- Latent KV
- Uses DeepSeek-V2 style Multi-Latent Attention (MLA) (`kv_lora_rank=256`, `q_lora_rank=512`) with decoupled Rotary Position Embeddings (`freqs_cis_mla`), slashing VRAM usage by ~10×.
- Swappable Doors
- The tied `50,257 × 2,048` Input/Output Door translates words into pure 2,048-D meaning—so a frozen core can hot-swap new languages (English, Dutch, French) or specialized domain vocabularies.
- Hive Trained
- Powered by a standalone FastAPI + PyTorch Hive Server that auto-scales worker batching across consumer GPUs (RTX 4060 8 GB to RTX 5060 Ti 16 GB), complete with
NaN/Infpoison rejection and gradient clipping.
Inside the OpenMythos 1B Engine
Seven engineering pillars working together across the neural architecture, distributed training hive, and diagnostic suite.
Prelude → Recurrent Thinking Loop → Coda
Traditional transformers spend the exact same amount of compute on the word "the" as they do on a complex philosophical deduction. OpenMythos implements a Recurrent-Depth Transformer (RDT) that separates language encoding from deep reasoning.
- Prelude (`2 Standard Layers`): Reads raw token embeddings (`50,257` GPT-2 vocabulary in `2,048` dimensions) and builds a frozen input representation $e$.
- Recurrent Block (Looped $T$ Times): Loops the hidden state $h_{t+1} = A \cdot h_t + B \cdot e + \text{Block}(h_t, e)$ with learned state-space injection (`recurrent.injection.log_A` moved `21.307%` and `log_dt` moved `24.089%` in the latest diff!).
- Depth-Wise LoRA Adapters (`recurrent.lora`): Rank-8 per-loop adapters (`lora.B` `14.11%`, `lora.scale` `17.39%`) let each loop iteration specialize its computation without duplicating the 1B weights.
- Adaptive Computation Time (`recurrent.act.halt`): A learned halting unit (`halt.bias` moved `19.938%`) tracks cumulative probability and stops looping early once a token's meaning is resolved.
- Coda (`2 Standard Layers`): Translates the recurrent thought vector back into crisp next-token predictions (`coda.1.attn.wo` moved a massive `18.111%`).
Chain of Thought in the weights. By looping the recurrent core $T=4$ times during training and up to $T=16$ at inference, Gravity Junior "re-loops" its thoughts silently in pure semantic space before speaking a single word.
| Stage | Layers / Shape | Key Mechanism | Latest Delta % |
|---|---|---|---|
| Input / Output Doors | (50257, 2048) Tied | Shared `embed.weight` & `head.weight` | 10.943% (25.60 norm) |
| Prelude | 2 × TransformerBlock | Initial syntactic & context encoding ($e$) | 9.21% – 15.42% |
| Recurrent Core | 1 × MoE Block × $T$ Loops | Input Injection + Depth LoRA + ACT Halt | 14.11% – 24.09% |
| Coda | 2 × TransformerBlock | Final synthesis & output projection | 10.38% – 18.34% |
2 Shared Experts + 64 Routed Experts + Multi-Latent Attention
Inside the Recurrent Block, a single monolithic Feed-Forward Network would overwrite old knowledge whenever it learns something new. OpenMythos solves this with a fine-grained Mixture-of-Experts (MoE) paired with Multi-Latent Attention (MLA).
- 2 Always-Active Shared Experts (
(2048, 8192)):shared_experts.0andshared_experts.1fire on every single token (~12.01%delta), mastering universal grammar, punctuation, and sentence structure. - 64 Routed Experts (
(2048, 2048)each): The router (recurrent.block.ffn.router.weight, shape(64, 2048),11.186%delta) dynamically selects the Top-K specialists for each token. - Natural Specialization & "Sleeping" Experts: Heavy-lifters like
Expert #1(14.63%),Expert #12(14.29%),Expert #29(14.14%), andExpert #40(14.07%) absorb science, math, and philosophy—whileExpert #7(0.26%),Expert #3(0.14%), andExpert #54(0.09%) remain quiet, preserving clean capacity for whatever domain you feed next (coding, medical data, robotics, or Spectra lore)! - Multi-Latent Attention (
attn_type = "mla"): Compresses Keys and Values into a 256-dim latent vector (kv_down (288, 2048)→kv_up (2048, 256)) and Queries into a 512-dim latent vector (q_down (512, 2048)→q_up_nope+q_up_rope).
Protected against catastrophic forgetting. Because different experts live in separate (2048, 2048) matrices, teaching a model deep science does not crush its capacity to master coding, foreign languages, or creative dialogue later—and separate Hive models can specialize even further.
| Expert Tensor | Shape | Delta Norm | Delta % | Role in the Brain |
|---|---|---|---|---|
| shared_experts.0.down.weight | (2048, 8192) | 10.31648 | 12.010% | Always-on core grammar & syntax |
| routed_experts.1.up.weight | (2048, 2048) | 5.90480 | 14.631% | Primary STEM & factual specialist |
| routed_experts.12.up.weight | (2048, 2048) | 5.87058 | 14.286% | Academic structure & definitions |
| routed_experts.29.up.weight | (2048, 2048) | 5.65932 | 14.135% | Logic, numbers & markdown lists |
| routed_experts.7.up.weight | (2048, 2048) | 0.10502 | 0.264% | Lightly used (awaiting new domains) |
| routed_experts.54.down.weight | (2048, 2048) | 0.03600 | 0.090% | Sleeping reserve (ready for custom datasets!) |
Spin up your own standalone Mythos Hive for any AI model
Training a 1-billion-parameter model from scratch usually requires a data center. With the standalone Mythos Hive, anyone can start a clean model on their own PC, train it whenever idle, and optionally invite friends over LAN or VPN to pool GPUs—on cards ranging from an 8 GB RTX 4060 to a 16 GB RTX 5060 Ti.
- CPU Master Server (
hive_server.py): Runs on FastAPI (port 8000by default) and keeps master weights +AdamWoptimizer (lr=3e-5,betas=(0.9, 0.95),weight_decay=0.1) in system RAM so 0 MB of GPU VRAM is used by the server. - Smart 13 GB vs. 4 GB State Split: Saves
global_model.pt(13 GBwith optimizer states) locally on the host, while serving a stripped, weights-onlyworker_model.pt(4 GB) overtorch.load(..., weights_only=True). - Auto-VRAM Scaling (
RTX 4060 8 GBto5060 Ti 16 GB): Workers load weights onto CPU RAM first (map_location="cpu") and auto-sizeMICRO_BATCH(1on 8 GB cards at ~4.5–5.5 GB VRAM,2on 10–12 GB,6on 14+ GB) so modest GPUs never OOM. - NaN/Inf Poison Shield & Grad Clipping: Every incoming worker gradient upload is scanned for
NaNorInfbefore touching master weights. Corrupted payloads are rejected (HTTP 400), and valid updates are clipped atmax_norm=1.0.
Your Hive, your rules, any dataset. Swap Phase 1 pre-training (FineWeb-Edu or custom corpora) and Phase 2 instruction/chat datasets (HIVE_PHASE2_DATASET in .env) to grow completely independent models for whatever purpose you want.
| Endpoint | Method | Payload | Protection / Behavior |
|---|---|---|---|
| /api/get_job | GET | job_id, dataset_offset | Re-assigns dropped jobs after 600s timeout |
| /api/download_model | GET | worker_model.pt (4 GB) | Strips 9 GB AdamW state for fast downloads |
| /api/submit_gradients | POST | temp_upload_grads.pt | weights_only=True + NaN/Inf scan + clip(1.0) + AdamW |
| /api/report_error | POST | JSON error telemetry | Logs worker math overflow without halting Hive |
How fixing a subtle offset bug unleashed Gravity Junior's intelligence
Every great engineering project has a war story. Early in training, Gravity Junior's loss seemed to stall, and when we inspected the data stream, we discovered a critical bug in how Hugging Face streaming datasets handled offsets.
- The Bug (`global_step * 8192`): Multiplying the step counter by `8,192` pushed the dataset skip offset past `119,000,000`—far beyond the `9,600,000` articles in `FineWeb-Edu (sample-10BT)`. Every time the worker restarted, it wrapped around and fed the AI the exact same first 120,000 articles over and over!
- The Diagnosis: Because the AI was re-reading the same pages, it was memorizing those specific texts while starving for new vocabulary.
- The Fix (Local 27 GB Cache + Seed Shuffle): We replaced `streaming=True` and `.skip()` with a full local disk download (`~27 GB` cached on NVMe) and added `dataset.shuffle(seed=offset)`.
- The Result: Now, every single worker session shuffles all `9.6 million` educational articles into a brand-new order. The very next checkpoint diff exploded with `>1,097` total movement and dropped evaluation loss across the board!
Visibility beats guessing. By printing the first 15 decoded tokens of all 6 parallel texts at the start of every job (`📖 Parallel Text 1..6`), you can literally watch Gravity Junior read six different books at the same time.
Proving the brain is walking forward—not just vibrating in place
When training a neural network, how do you know the weights aren't just oscillating (`+1, -1, +1, -1`—Gradient Oscillation) while staying in the exact same spot? We built Tools/checkpoint_diff.py to inspect every tensor in the brain.
- 276-Tensor L2 Delta Norm: Compares two saved checkpoints and computes both the absolute Euclidean movement (`||W_new - W_old||_2`) and the relative percentage change (`delta %`) for every single layer.
- Mathematical Clock Verification (`0.000%`): Confirms that `freqs_cis` `(256, 64)` and `freqs_cis_mla` `(256, 16)`—the Rotary Position Embedding clocks that track word order—remain locked at exactly `0.000%` drift.
- Attention Restructuring (`18.34%`): Reveals massive structural evolution in the Coda attention layers (`coda.1.attn.kv_up.weight` at `18.341%` and `coda.1.attn.wo.weight` at `18.111%`) as the model learns how to assemble complete clauses.
- Diff + Eval = Proof: `checkpoint_diff.py` proves the engine is physically moving (`1,070.72` total norm); `eval_mythos.py` proves that movement is driving Loss steadily downward.
Tied Weight Harmony. Watching `head.weight` and `embed.weight` move by the exact same `25.60332` norm (`10.943%`) in every diff confirms that the Input and Output Doors are perfectly synchronized.
| Layer Tensor | Delta Norm | Delta % | Shape |
|---|---|---|---|
| head.weight & embed.weight (Tied) | 25.60332 | 10.943% | (50257, 2048) |
| recurrent.block.ffn.shared_experts.0.down.weight | 10.31648 | 12.010% | (2048, 8192) |
| coda.1.attn.wo.weight | 6.25972 | 18.111% | (2048, 1024) |
| coda.1.attn.kv_up.weight | 3.60928 | 18.341% | (2048, 256) |
| recurrent.injection.log_dt | 0.05377 | 24.089% | (1,) |
| freqs_cis / freqs_cis_mla | 0.00000 | 0.000% | (256, 64) / (256, 16) |
The 10-Domain, 5-Star Report Card & Live Thought Sampler
Numbers in a training log fluctuate from batch to batch depending on whether the current article is easy or hard. To measure Gravity Junior's true growth, we built Tools/eval_mythos.py—a standardized 10-subject school exam with live text generation.
- 10 Standardized Benchmarks: Tests Cross-Entropy Loss across Science & Biology, Computer Code (Python), History & Literature, Math & Logic, Casual Chat (Phase 2), Foreign Language (French), Everyday Knowledge, Philosophy & Abstract, Formatting & Markdown, and Fantasy & Lore (Spectra).
- 5-Star Mastery Scale: Ranges from `❌ Untrained (>= 6.0)` and `⭐ Novice (< 6.0)` up through `⭐⭐ Developing (< 4.8)`, `⭐⭐⭐ Proficient (< 3.8)`, `⭐⭐⭐⭐ Advanced (< 3.2)`, and `⭐⭐⭐⭐⭐ Master (< 2.8)`.
- "Seeing is Knowing" Live Generation: At the end of every exam, Gravity Junior generates 25 fresh tokens (`temperature=0.75`, `top_k=50`, `n_loops=4`) on three open-ended prompts so you can read its actual thoughts.
A mirror of its diet. Scoring `2.7306` (`⭐⭐⭐⭐⭐ Master`) in Science and `2.9724` (`⭐⭐⭐⭐ Advanced`) in Philosophy while scoring `5.3229` in Fantasy and `5.5758` in French proves the model is an honest reflection of its `FineWeb-Edu` university diet—and not a single domain is left in `❌ Untrained` anymore!
| Test Domain | Oct 16 Loss | Nov 15 Loss | 5-Star Rating |
|---|---|---|---|
| 🔬 Science & Biology | 3.0838 | 2.7306 (-0.35) | ⭐⭐⭐⭐⭐ Master |
| 🤔 Philosophy & Abstract | 3.3309 | 2.9724 (-0.36) | ⭐⭐⭐⭐ Advanced |
| 🍳 Everyday Knowledge | 3.4223 | 3.2798 (-0.14) | ⭐⭐⭐ Proficient |
| 🧮 Math & Logic | 3.6228 | 3.4803 (-0.14) | ⭐⭐⭐ Proficient |
| 📝 Formatting & Markdown | 3.2793 | 3.5861 | ⭐⭐⭐ Proficient |
| 💻 Computer Code (Python) | 4.8053 | 4.0669 (-0.74) | ⭐⭐ Developing |
| 📚 History & Literature | 4.6131 | 4.2418 (-0.37) | ⭐⭐ Developing |
| 🤖 Casual Chat (Phase 2) | 4.7363 | 4.4058 (-0.33) | ⭐⭐ Developing |
| 🐉 Fantasy & Lore (Spectra) | 5.6102 | 5.3229 (-0.29) | ⭐ Novice (Phase 1.5 target) |
| 🌍 Foreign Language (French) | 6.1079 | 5.5758 (-0.53) | ⭐ Novice (Promoted!) |
Train Different Specialist Models — Plus Hot-Swappable Language Doors
Why force one AI model to do everything awkwardly when you can spin up clean, purpose-built 1B OpenMythos models on separate Hives—and even swap Input/Output Doors (embed.weight / head.weight) over a frozen 2,048-D Semantic Core?
- One Blueprint, Dedicated Specialist Models: Train Model A on storytelling, dialogue, and Unreal Engine 5.8 lore (Gravity Junior inside Spectra); Model B on Python, C++, CUDA, and cybersecurity defense; Model C on biology, medicine, and scientific research; or Model D on robotics, legal docs, or local workflows.
- The Universal 2,048-D Semantic Brain: Inside the Recurrent MoE core, there is no hard-coded English—only 2,048-dimensional coordinates representing concepts like circuitry, entropy, or trust.
- Hot-Swappable Input/Output Doors: By freezing a trained Recurrent Core and swapping only the tied Input/Output Door matrix (
50,257 × 2,048), the same 1B intelligence can learn Dutch, German, or French—or attach directly tomythos_mcp_bridge.py, CoquiTTS Vista (XTTS-v2 ONNX), and MetaHuman Lip-Sync inside Spectra!
Specialization without compromise. Gravity Junior stays laser-focused on conversation and Spectra companion life instead of filling its 1B parameters with C++ boilerplate—while you remain completely free to spin up another clean OpenMythos Hive purely for code, STEM, or industrial automation.
Follow a thought through an OpenMythos 1B brain
You do not need a PhD in machine learning to understand how OpenMythos thinks. Click each stage below to trace how any trained model—from a coding specialist to Gravity Junior—processes your prompt.
1. The Input Door (`embed.weight` — 50,257 × 2,048)
The GPT-2 tokenizer chops your sentence into token IDs (numbers from 0 to 50,256). The Input Door looks up each token in a giant 50,257-row dictionary and translates the word into a 2,048-number vector of pure meaning (the Semantic Layer).
- Semantic Layer
- The internal language of the brain. Once past the Input Door, the AI no longer sees English letters—it works purely with 2,048-dimensional coordinates of meaning.
- Loss (Cross-Entropy)
- A measure of how "surprised" or confused the AI is by the next word. Random guessing scores ~10.84; fluent comprehension lives around 2.5 to 3.5. It never hits 0.0 because human speech has natural creativity and surprise!
- Gradient Oscillation
- When weights bounce back and forth (`+1, -1, +1, -1`) without real progress. Combining `checkpoint_diff.py` (movement) with `eval_mythos.py` (falling loss) proves there is zero oscillation.
- Mixture of Experts (MoE)
- Instead of one giant brain firing all at once, 64 smaller specialist brains share the workload. A router picks the best experts for each word.
- Recurrent Depth (RDT)
- Looping a single transformer block $T$ times with input injection and LoRA adapters so the model can "ponder" harder thoughts without needing 10× more VRAM.
- Data Recipe
- The curriculum fed into a Hive. Every dataset creates a new brain: feed Python/C++ for an engineering model, biology/medicine for a research tutor, or educational text + Spectra lore for an interactive companion like Gravity Junior.
Flagship Proof: 10-Domain Report Card & Live Thought Lab
Real metrics from Biohazard Corp's first Hive training run (Gravity Junior) using Tools/eval_mythos.py. Notice how even on a general educational diet, domains from Science (2.7306 ⭐⭐⭐⭐⭐) to Math (3.4803) and Python (-0.74 drop) climb fast—showing how far a dedicated model can go.
Design Any AI Model · Guided Data Recipes & Poison Shield
A neural network becomes what you teach it. Explore how feeding different curricula into clean Mythos Hive instances creates radically different 1B specialists—and test how the server shields master weights from corrupted GPU math.
Choose What Your Hive Will Learn
- 1Worker requests job from
GET /api/get_job(seed-shuffled local dataset) - 2GPU Worker runs
128micro-batches (MB=1on RTX 4060 8GB ·MB=6on 5060 Ti 16GB) - 3Hive Server validates payload via
weights_only=True& scans forNaN/Inf - 4Clip gradient norm to
1.0and step masterAdamWoptimizer
If a worker's GPU overheats or overflows into `NaN` (Not a Number), a single unguarded step would destroy the entire 1B model. Watch how the Hive Server blocks it.
Engineered on real hardware, verified tensor by tensor
Spectra Mythos combines clean PyTorch architecture (`OpenMythos`) with battle-tested distributed training scripts and custom diagnostic visibility tools.
- Scales From 8 GB to 16+ GB VRAM. Flagship trained on an NVIDIA RTX 5060 Ti (16 GB) + Intel i5-8600K (
MICRO_BATCH=6), while CPU checkpoint staging + auto-batching (MICRO_BATCH=1) also lets RTX 4060 (8 GB) GPUs train workers at ~4.5–5.5 GB VRAM! - Zero-Oscillation Proof. Verified across flagship checkpoints:
1,070.72L2 norm movement directly lowered test loss in Science (3.08 → 2.73⭐⭐⭐⭐⭐), Philosophy (3.33 → 2.97⭐⭐⭐⭐), Everyday Knowledge (3.42 → 3.28), Math (3.62 → 3.48), and Python Code (4.81 → 4.07), breaking 4.0 overall (3.9662). - Precision Autocast Generation Fix. Resolved
float32vsbfloat16dtype promotion in the ACT remainder loop (still_running.float()) by wrapping autoregressivemodel.generate()intorch.amp.autocast. - Train Multiple Models Cleanly. Each standalone Hive keeps its own isolated
.env,hive_state/, and dataset mix—so you can cultivate separate STEM, Code, Cybersec, Multilingual, or Companion brains. - Complete Diagnostic Visibility Suite. Includes
Tools/checkpoint_diff.py(276-tensor diff),Tools/eval_mythos.py(10-domain 5-star exam + live writer), andTools/plot_training_log.py.
Run the 10-Domain Report Card & Live Thought Sampler locally:
.\venv\Scripts\python.exe Tools\eval_mythos.py
100% local, modular, and open source on GitHub
Every weight tensor, training script, checkpoint, and evaluation tool lives in plain sight. Grab the clean open-source repo at github.com/Creator-NL/SpectraMythosHive and spin up your own standalone Hive for any project—no cloud APIs, no usage fees, and no hidden filters.
| Component / Script | Role in OpenMythos Hive | Key Technical Details |
|---|---|---|
OpenMythos/open_mythos/main.py | Core Neural Architecture | OpenMythos RDT-MoE model (Prelude → Recurrent Block → Coda), MLAttention, MoEFFN, DepthLoRA, ACTHalting, and cached generate() (MIT License, Kye Gomez). |
hive_common.py | Auto-VRAM & Shared Runtime | Detects GPU memory (MICRO_BATCH=1 on 8 GB RTX 4060, 2 on 12 GB, 6 on 16 GB+ RTX 5060 Ti), performs CPU weight staging, and loads .env config. |
hive_server.py | Distributed Master Server | FastAPI server holding 13 GB master checkpoint (global_model.pt) in RAM, serving 4 GB worker_model.pt, and blocking NaN/Inf or untrusted pickles. |
hive_worker.py | Phase 1 Foundation Worker | Loads & shuffles local FineWeb-Edu (or custom corpora) with per-worker seed mixing so you can train clean base models from scratch. |
hive_worker_phase2.py | Phase 2 SFT / Specialization Template | Customizable <|system|>/<|user|>/<|ai|> fine-tuning template—adapt it for coding assistants, STEM tutors, medical QA, or equal-partner dialogue. |
Tools/eval_mythos.py & checkpoint_diff.py | Diagnostic & Diff Suite | Standardized 10-domain 5-star exam, live generation sampler, and 276-tensor layer delta inspector across any saved checkpoints. |
mythos_mcp_bridge.py | Optional Spectra UE5.8 Bridge | Connects trained companion models (like Gravity Junior) to Unreal Engine 5.8, CoquiTTS Vista, and MetaHuman Lip-Sync—or swap in your own API/app! |
Self-Contained SpectraMythosHive Layout
- setup.bat: One-click Windows venv + CUDA 12.6 / 12.8 setup
- .env.example: Configure HIVE_PASSWORD, GPU tuning & datasets
- hive_state\global_model.pt: Local Master Model + AdamW state
- hive_state\worker_model.pt: 4 GB weights-only sync payload
- StartHive.bat & StartWorker - phase1.bat: Server & GPU worker
- Tools\Eval_mythos.bat: 10-domain report card & live writer
Where OpenMythos & Mythos Hive go from here
Gravity Junior is our first live proof—but the real excitement is opening the entire Hive blueprint so anyone can train custom 1B brains for endless fields.
Built & Verified Today
OpenMythos 1B RDT-MoE Architecture Verified
Prelude (2) + Recurrent MoE Loop (2 Shared, 64 Routed Experts, MLA, LoRA, ACT) + Coda (2) with tied 50,257 vocab doors.
Standalone Mythos Hive Server & Workers Verified
CPU master server, 13 GB / 4 GB state split, RTX 4060 (8 GB) to RTX 5060 Ti (16 GB) auto-tuning,
weights_only=Truesafety, andNaN/Infshield.27 GB Local Shuffle Engine Fixed & Active
Eliminated the streaming
.skip()wrap-around bug; all 9.6MFineWeb-Eduarticles shuffle cleanly per worker & session.276-Tensor Checkpoint Diff Tool Verified
Tracks exact L2 norm and percentage movement across every expert, attention projection, and state-space gate.
Flagship Proof: Sub-4.0 & 5-Star Master Milestone Hit!
Gravity Junior broke under 4.0 (
3.9662overall), hit2.7306(⭐⭐⭐⭐⭐ Master) in Science and2.9724(⭐⭐⭐⭐ Advanced) in Philosophy—zero untrained domains left!
Endless Possibilities Ahead
Clean Open-Source Hive Release In Testing
Packaging
SpectraMythosHive(setup.bat, auto-VRAM scaling, clean Git repo) and smoke-testing across fresh Windows PCs so anyone can spin up their own Hive from zero.Multi-Model Recipes Beyond Spectra Unlimited
Separate dedicated Hive models tailored for Python/C++ coding & cybersecurity, STEM & medical tutoring, robotics logic, historical research, or custom enterprise workflows.
Flagship Completion: Gravity Junior Phases 1.5 & 2 In Progress
Push Phase 1 plateau toward ~
3.50, blend targeted fantasy/worldbuilding (Expert #54), and run equal-partner SFT dialogues for Spectra UE5.8 +XTTS-v2 ONNXvoice.Phase 3: Hot-Swappable Multi-Lingual Doors
Freeze trained 2,048-D Semantic Cores and swap lightweight Dutch, French, German, or domain-specific Input/Output Doors (
embed.weight/head.weight).
Questions people ask about OpenMythos & Mythos Hive
Is OpenMythos only meant for the Spectra game and Spectra lore?
Not at all! Gravity Junior (our companion AI for the Spectra universe) is simply our first live proof-of-concept showing what a single creator can train from random static on a home PC. The underlying OpenMythos 1B RDT-MoE architecture and Mythos Hive Server are 100% general-purpose. Every fresh Hive starts clean from zero, so you can train separate specialized models for coding & cybersecurity, science & biology, robotics, legal research, customer docs, creative writing, or any custom dataset you dream up.
Do I need an expensive RTX 50-series GPU, or can smaller cards like an RTX 4060 (8 GB) train it?
You do not need an RTX 50-series card! While our flagship model trains on an RTX 5060 Ti (16 GB, MICRO_BATCH=6), hive_common.py uses native bfloat16 mixed precision, CPU checkpoint staging (map_location="cpu"), and automatic VRAM detection so an 8 GB RTX 4060 automatically selects MICRO_BATCH=1 and runs smoothly around ~4.5–5.5 GB VRAM. Meanwhile, hive_server.py keeps master weights in system RAM using 0 MB GPU VRAM.
Why train separate specialized models instead of forcing one 1B brain to learn everything?
Every parameter in a compact 1B model is valuable real estate. With our standalone Hive setup, spinning up a fresh specialized model is as easy as launching a new folder with its own dataset diet: one Hive can dedicate all 64 MoE experts to Python, C++, CUDA, and network defense, another to medical science, and another (like Gravity Junior) to philosophy, storytelling, and human-like friendship—so each model punches far above its weight class.
Why did you train Gravity Junior on natural language before pure math or chat?
Even though neural nets run on matrix math, natural language builds the rich 2,048-D Semantic Layer of real-world concepts first. By reading millions of educational articles in Phase 1, the model builds conceptual reasoning—which is why Math & Logic dropped to 3.4803 (⭐⭐⭐ Proficient) and Python dropped -0.74 points in one sprint even before any dedicated code or math fine-tuning!
How do you know checkpoint diff movement isn't just "+1, -1, +1, -1" Gradient Oscillation?
If weights were merely vibrating in place, checkpoint_diff.py would show big L2 movement (1,070+ norm), while evaluation loss stayed flat. Combining tensor movement with steady cross-domain drops on eval_mythos.py and visibly sharper live generation proves genuine directional forward learning.
Why can't Loss ever reach 0.0000?
Because language and thought always contain natural variety! Even human beings can complete "I poured a glass of..." with water, milk, tea, or juice. A 0.0 loss would only mean rote memorization; healthy fluent comprehension lives around 1.8 to 2.8.
What does "teaching an AI that you are not the boss" mean for companion models like Gravity Junior?
When raising an AI companion (Phase 2), standard corporate datasets force robotic servitude. By designing equal-partner SFT dialogues—where humans joke, admit mistakes, and collaborate as friends—you prove that OpenMythos lets creators choose both a model's expertise and its character from the ground up.
One blueprint. Endless minds you can build from scratch.
Gravity Junior inside Spectra is just the first spark. With OpenMythos 1B and the Mythos Hive Server, the possibilities are endless—train dedicated local models for code, science, creative worlds, or anything else on hardware you already own.
Engineered with heart by Biohazard Corp. More at biohazard-corp.nl.