0xSero ✓

[video] Local AI is about to be competitive. I will do everything in my power to make it better than Claude desktop/claude code by end of year.

[image] Best models for your hardware - 4gb to 12gb vram - VibeThinker-3B - smokes everything remotely close to its weight class. Challenging 30b models! Last version was also topping math benchmarks - 12gb to 24gb vram - Gemma-12B-coder Built on top of

[image] Important: This is a summary of an amazing video by one of the best creators I know. Video in the first comment. If you've been struggling to setup a productive local environment I'll summarise, but you should watch the video. 1. Qwen3.6-27B with NO THINKING - 4bit - 16bit

[image] Weekly best models for your hardware: ~~ 8 to 16gb ~~ Granite models are amazing: [NEW] - Gemma-E4B is a good general QA model - Qwen3.5-9B is the best at this level imo - ~~ 16 to 64gb ~~ Another

[text] Breakthroughs: 1. Turboquant merged into vLLM 75% vram reduction for kvcache near losslsss 2. Someone merged M2.7 & M2.5 & got it to perform better than M2.7 3. 40% faster prefill on AMD strix halo (128gb for MoEs w 10B active params) 4. Megatrain 100B model trained on 1 GPU

[image] Locally Part 1 - Apple Silicon Macs give you large pools of memory to run big models, but the token generation speed will be lower than most are used to. Macs are best with large MoEs that have low ACTIVE params. Basically when you see a model like Qwen3.5-397B-A17B this

[image] New build cooking - AMD threadripper 7965WX - Asus pro WS WRX90E-Sage SE - 256gb ddr5 - Corsair 9000D - 14 Noctua fans - 8tb NVMe I’m connecting an exhaust fan and pushing the air outside

[text] Here’s what I’d recommend if you’re just getting started in AI, local or otherwise. 1. Work with the compute you have, even the dumbest LLMs can be useful if you treat them as a node in your system. Some basic problems of what could be useful to get you started - tag all

[image] Cheapest competitive build on Nvidia 2 sparks = 8000$ Total specs - 256gb - 8tb - 546gb/s memory bandwidth - tons of flops --- models: - Deepseek-v4-flash - MiMo-v2.5-flash fp4 - MiniMax-M2.7 - Qwen3.5-397b-reap Flaws: - Low mem bandwidth - You need 2 for best perf

[image] For any homelab friends. Tailscale + mullvad here’s a repo to help if you’re on Mac

[image] Best harnesses for local models: 1. Droid: - Very good performance, forces the models to behave, you can wire in all your local LLMs very easily w BYOK - Allows you to use your local models as orchestrators/subagents so you can benefit from Cloud as models as well - Practically

[image] One thing I've done this year is: - Download all my X data from settings/account - Download all my youtube, gmaps, gmail, google from takout google com - Download all my personal data from Claude, ChatGPT - Export a copy of every AI session on Cursor Claude Code, Codex, Droid,

[text] Help me spread this, I am on a role and need to squeeze every last bit out of it.

[image] Based on my knowledge of: - AI labs - hardware manufacturers - model size performance - training process - training economics If you want to buy hardware for local AI you need 500-750gb of 500gb/s+ memory bandwidth to make serious use of local hardware at frontier levels

[image] To my fellow homelab friends, I need some help. My room is starting to cook, so I need to setup air conditioning (I already have a portable one but need something more pro) Essentially the rooms humidity is quite low and given I have cats there’s a lot of dust I need to clean

[image] Guide to running BIG B0Is on your small hardware. 1. Use REAPs: up to 50% savings 2. Use quantisations: 75% savings - AWQ / GPTQ / W4A16 / FP8 = FAST inference - GGUF / EXL3 = Slow but just works - MLX = Best for apple 3. Use 8bit KV cache: 50-75% savings

[image] Exl3 is the best quantization schema for local AI rn, especially if you have Blackwell lite (DGX/5090/4000/5000/6000) It was integrated into sparkinfer (fork of vLLM) and it’s running at 3x the speeds it was on exllamav3 It also works well when you have odd number of cards

[image] Top 5 builds for AI inference in 2025-2026 I have spent around 12 months researching, building, experimenting, and bench-marking AI models, tools, hardware and costs. Top 3 picks will be the safest, best cost to performance ratios. The last 2 will be more interesting

[image] I got 2 intel bad boys on Friday. At this point I’m struggling to find more power for all this. - 4x 6000s - 1x DGX Spark - AMD Strix - 4x 3090 - 2x intel arc b70 - Mac mini 16gb - MacBook Pro 32gb 544gb VRAM 300gb mixed Total = 844gb AI + 512gb ddr4 I need some guru help

[text] We should get localmaxxing gang in here. I love what they're doing, going to start competing on it next week <3 Also does anyone here know how to make X bots the legal TOS way?

[image] Why I built vllm-studio - Storing configurations for models and engines - Fully integrated agent with Pi - Model downloading, exploration, and config - Creating benchmark datasets Backend can be deployed to multiple servers, and the frontend can connect to all from anywhere.

[text] Local AI is a human right, our children, families, neighbours, friends, and fellow humans deserve privacy and freedom.

[text] I meant to post this in here, I will be posting weekly models meant for limited hardware budgets, RN I am learning how to deal with the 30b~ class