One GPU, a Whole AI Stack: Benchmarking the Radeon AI PRO R9700

The hardware

• GPU: AMD Radeon AI PRO R9700 (RDNA 4, gfx1201, 32 GB VRAM, 300 W)
• Second GPU: Intel Arc A750. It drives the monitor and runs the small helper jobs.
• CPU and RAM: Intel i7-13700K, 64 GB RAM
• OS: Kubuntu 26.10, ROCm 10 for the AMD card, Vulkan for the Arc

The software stack

The models I compared

The hard part is fitting a good LLM and leaving room for image generation on one card. I tested three approaches:

1. Swift 27B on vLLM. Swift 1.5, a fine-tune of Qwen3.8-27B, quantized to MXFP4, with a DFlash2 speculative-decoding drafter. vLLM gets 66% of the VRAM: 64k of context, and enough room left for ComfyUI.
2. Flash-Next on Strata. Qwen3.8-Flash-Next is a mixture-of-experts model, here an IQ2_XS quant. Strata keeps the experts in system RAM and caches the hot ones in VRAM. I set it to always leave 11 GB of VRAM free for ComfyUI.
3. Swift Flash-Next on Strata. The Swift fine-tune of Flash-Next, same quant and same settings.

For reference I also ran two models on plain llama.cpp.

How I benchmarked

A small script talked to each server’s OpenAI-compatible API, with streaming on and ComfyUI loaded on the same card the whole time:

• Single-user decode: four task types (chat, code, prose, JSON) at temperature 0 and 1, best of two runs, 300–700 output tokens each.
• Time to first token on a short prompt.
• Long-prompt processing at roughly 3k, 12k and 46k input tokens.
• Concurrency: 1, 4 and 8 simultaneous requests.
• Reasoning test: six classic trick questions with thinking on, checked for correctness.

Decode speed, one user

Tokens per second:

Swift on vLLM wins clearly on code and JSON, where the speculative drafter guesses well. On free-form prose the three are close.

Latency and long prompts

Several users at once

Total tokens per second across all requests:

This is where vLLM really pulls ahead: it batches requests, so four users get almost three times the total throughput. Strata stays at about 106 tok/s however many users there are.

Tuning vLLM for several users

The Swift build shipped with settings for one user. I changed two:

• Speculative tokens: 15 to 7
• Max batched tokens: 2048 to 4096

Single-user code generation dropped by about 25%, and throughput with 4 to 8 users rose 53–84%. All the Swift numbers above are with the tuned settings.

Reasoning with thinking on

The Swift fine-tune didn’t think noticeably less on easy questions, and both got everything right.

Why the Swift fine-tune was slower on Strata

Swift Flash-Next was 10–25% slower than the original everywhere. Strata’s expert-cache profile is built for the original model, so with the fine-tune the cache hit rate started at about 35% and only climbed to about 63%. More expert lookups fall through to system RAM.

Reference: llama.cpp

How this compares with other R9700 owners

My numbers line up with what other single-R9700 users have published. The one gap is Swift with 8 users, where published runs reach about 480 tok/s against my 339. They give vLLM 98% of the VRAM; I give it 66% so image generation can run alongside.

Claude Code: Anthropic’s servers against my local model

Strata offers an Anthropic-compatible API, so I set up claude-local: the regular Claude Code app with its own config folder, pointed at Flash-Next on Strata. I gave it and regular Claude Code (Opus 5.5) the same tasks. Each task had hidden checks the agents never saw.

Round 1: five everyday tasks

Task 4 had a trap: read the spec literally and the reason “llama.cpp is loaded…” gets cut to “llama”. Both got it right. The one difference in depth: on the code question, only Opus also spotted a subtler timing race in the script.

Round 2: one harder task

A broken repository with five separate problems:

• Python 2 print syntax
• a version mismatch
• a data file path that only worked from the repo folder
• a helper script that couldn’t import the package
• a test command that breaks on Python 3.14, to be fixed without editing the tests

Both found all five problems. The difference was in how they fixed a circular import. Opus removed the unnecessary import and warned that outside code using it would break. The local model moved one line so the loop no longer fails, which works but is fragile. Tests don’t catch that kind of quality gap.

Takeaway

• Correctness: the local model matched Opus on every task.
• Speed: it was 1.7 to 2.8 times slower, and the gap grew on harder work.
• Cleanliness: its fixes were a little less clean, and it read code less deeply.
• Cost: free and fully private, against about $1.20 of usage for the Opus runs.

These were well-defined tasks with a clear “done” test. Long, open-ended system work, where nothing tells the agent it’s wrong, is where I’d still use Opus.

Images: the prompt improver makes the difference

ComfyUI runs Z-Image Turbo for text-to-image and FLUX.3 Klein for text-to-image and editing. Both are hooked into Open WebUI, so you can ask for an image in a chat.

The biggest quality jump came from rewriting the prompt first. A short request becomes a detailed scene description, and the results look clearly more realistic and lived-in. The rewriting runs in a chain:

1. First choice: a small vision model on the Arc A750, using Black Forest Labs’ official prompt-upsampling instructions. About 38 tok/s: roughly 10 s for text-to-image and 14 s for edits.
2. Fallback: the main LLM on the R9700.
3. Last resort: the raw prompt.

Timings and fixes:

• Z-Image Turbo: about 15 s plain. About 40–44 s with the prompt improver plus a detail pass (1.25× upscale, a light second sampling pass at denoise 0.3, a touch of film grain).
• FLUX.3 resolution: running at about 2 megapixels (1440×1440) instead of 1 fixed most eye defects on small faces.
• Prompt rules: a few pose and clutter rules in the improver’s instructions stopped awkward half-kneeling legs and overly busy scenes.

Video, and a ROCm bug

LTX-2.5 (22B) does image-to-video but needs the whole card, so a video mode unloads the LLMs first. The LLM proxy refuses to restart Strata while a video renders. I learned that the hard way, when a chat request reloaded the LLM mid-render and the video job ran out of memory.

The nastiest problem was videos coming out as pure noise or black frames. It turned out to be a ROCm 10.0 bug with dynamic VRAM on gfx1201, fixed in ROCm 10.1. Upgrading ComfyUI’s PyTorch to the ROCm 10.1 build fixed it: three back-to-back video runs came out clean.

Voice assistant

Home Assistant uses the box for speech-to-text, the LLM and text-to-speech, in English and Serbian:

• Speech-to-text: whisper large-v3 on the Arc.
• LLM: Flash-Next on Strata with thinking off. It replies in 0.4–0.5 s when warm and about 1.5 s cold.
• Device control: 9 out of 9 commands correct in both languages.
• State questions (“is a light on?”): the model guessed at first. After an instruction to never guess and always check the live state, it got 12 out of 12.

Switching costs

Which one do I use?

Flash-Next on Strata is my daily driver. It’s slower than Swift on vLLM, but it leaves 11 GB for image generation, has vision, reasons well, and runs the voice assistant and the local coding agent. Swift on vLLM stays installed for raw speed or many users at once.

Lessons

1. 32 GB is enough for a full stack if you plan the VRAM. Reserve memory for image generation explicitly, and don’t let the LLM take it all.
2. A cheap second GPU is worth it. Moving speech-to-text and the prompt improver to the Arc freed the main card.
3. MoE with the experts in RAM is a real option. Slower than a dense model fully in VRAM, but a much bigger model in a fraction of the VRAM.
4. Tune for how you’ll actually use it. Fewer speculative tokens cost a little single-user speed and gained a lot with several users.
5. A local coding agent is genuinely useful. On well-defined tasks it matched a frontier model on correctness at about half the speed, for free.
6. On a new GPU architecture, suspect the driver stack. The “corrupted cache” was a ROCm bug, and updating fixed what no workaround could.