gavinj.net

From 24GB Blackwell workstation cards to one 128GB DGX Spark, then two Sparks and a QSFP cable: running DeepSeek-V4-Flash-Vision-Exp locally, understanding vLLM, and planning the next step with a 200Gb/s switch.

A DGX Spark with 128GB of unified memory let me explore much larger local models. Then I discovered Heretic variants, and a hardware experiment became a rather uncomfortable lesson in how AI refusals work, how easily they can be changed, and what that means for the rest of us.

I started using ShellGPT today with a locally hosted Gemma 4 26B quantized model through Ollama on my RTX PRO 4000 Blackwell workstation. Asking questions in the terminal was fine, but the real value showed up when local AI met the Unix pipeline.

I wanted to know whether a sub-$3k Australian workstation GPU could run useful modern local LLMs. The short answer is yes, if you pick the right model, the right quant, and build around the 24GB VRAM limit instead of pretending it is a tiny cloud.

Ubuntu 26.04 LTS is not trying to turn the desktop into a chatbot. It is making it easier to run AI tools locally, see what hardware is being used, and choose which models belong on your machine.