Most people build a local LLM workstation around the fastest processor they can justify, then discover that the model they wanted does not fit in memory.

The verdict up front: prove the workflow on your current computer, then choose either a tower with 24GB to 32GB of GPU memory or a 128GB Ryzen AI Max workstation. I run a quantized Qwen3.8 build locally from an external drive, while keeping a smaller cloud subscription for requests that earn more compute.

1. How much memory does a local LLM need?

This is the biggest lever. Quantization compresses model weights, but context, retrieval, and the agent runtime still need their share. Treat 24GB of addressable memory as the entry point for a serious local workstation, 32GB as useful breathing room, and 48GB or more as the practical target when an agent shares the machine with normal office software.

Qwen3.8 is the workload behind this guide. Both the original weights and quantized builds are available for local hosting. I keep the quantized version on an external drive. The exact quantization still matters because model size, context, and runtime overhead decide whether a build that loads remains responsive once the agent begins working.

2. Which NVIDIA GPU fits local AI?

An expandable tower keeps the model on the GPU and leaves system memory for the agent and documents. The RTX 3090 and RTX 4090 both provide 24GB of GDDR6X. The older 3090 is the value-minded route when its history and cooling are trustworthy. The 4090 is faster, but NVIDIA specifies 450W graphics power and an 850W system recommendation.

The RTX 5090 raises capacity to 32GB of GDDR7. That extra memory matters more than its benchmark lead, but the trade-off arrives on the same specification sheet: 575W graphics power and a 1000W system recommendation.

3. Can a compact workstation run larger models?

The Framework Desktop with Ryzen AI Max+ 395 carries 128GB of unified memory, with up to 96GB graphics-addressable. Its 4.5-liter enclosure measures 96.8 by 205.5 by 226.1mm. The memory is soldered, so the capacity you buy is permanent.

DwarfStar 4 makes this machine more interesting. Its ROCm backend explicitly supports 128GB Strix Halo systems such as the Framework Desktop. Setup still requires Linux, ROCm, a large GPU-visible memory aperture, and the correct quantization. This is a supported experiment, not an appliance experience.

4. Use local AI as the default lane

My daily process is deliberately uneven. A quantized Qwen3.8 build and Hermes Agent handle private documents, drafts, summaries, retrieval, and short tool sequences on the workstation. A larger cloud model handles unfamiliar problems, long planning chains, heavy multimodal jobs, and work where a weak answer costs more than the request.

That division is the point. Local inference buys privacy, predictable availability, and repeated work without metering every prompt. Cloud compute remains the specialist I call when the task earns it.

Budget by tier

Starter: test Ollama or LM Studio on the computer already owned.

Professional: use a verified 24GB or 32GB GPU, adequate power, 64GB of system memory, and fast NVMe storage.

Studio: choose 128GB unified memory when larger models or DwarfStar are paid workloads rather than future wishes.

Who should skip most of this

This is hard to justify when local AI is occasional or every important file already lives in shared cloud tools. A hosted model will be simpler.

It is also the wrong project if maintaining runtimes, quantizations, and agent permissions sounds like a second job. Local ownership moves responsibility onto your desk with the model.

Build around the exact Qwen3.8 build first, memory second, and speed third. The workstation earns its space when the cloud becomes a deliberate escalation instead of the default.

Sources

More field guides

Research the next purchase

Choose the room or problem you are working on next.

Browse all guides