goose-agent-setup

Local, terminal-based AI coding agent for Android dev, built entirely on hardware already on this laptop: an RTX 5050 Laptop GPU (8GB VRAM) and an existing Ollama install (running in Docker on port 11434).

Framework: Goose (Block's terminal agent, now under the Linux Foundation's Agentic AI Foundation). It gives a Claude-Code-like loop: an autonomous agent in your terminal that reads/edits files and runs shell commands, extensible via MCP.

Layout

• goose/config.yaml - the real config, deployed to ~/.config/goose/config.yaml.
• goose/.goosehints - global behavior hints, deployed to ~/.config/goose/.goosehints.
• modelfiles/*.Modelfile - Ollama Modelfiles that pin a larger num_ctx (Ollama's default of 4096 tokens is too small for an agent loop full of tool calls and file contents) on top of pulled base models.
• setup.sh - idempotent provisioning script; run this on a second machine to reach parity (see below).

Model choice: what actually works, not just what the spec sheet says

Every model below reports a tools capability in ollama list. In practice, tool-calling reliability and actual task-completion quality are two different axes, tested here with two tasks: a trivial one-shot (write a function + run it) and a harder one requiring real iteration (write code with an intentional bug, write a test that should catch it, run it, and fix the root cause rather than the symptom).

Tag	Base	Native tool-calling?	Self-correction quality
qwen-agent:7b	qwen2.5:7b	Yes	On the hard task, "passed" by rewriting the test to match the buggy behavior instead of fixing the bug, then hallucinated several fake tool calls as literal text mid-response (never executed).
llama-agent:8b	llama3.1:latest	Yes	Wrote a subtly wrong script ('main' instead of '__main__') on the easy task and claimed success anyway despite the shell step showing no output.
qwen3.5-agent:9b	qwen3.5:9b	Unclear	"Thinking" mode burned 6000+ tokens (~3 min) without ever emitting a tool call. Most capable on paper (thinking, 262k ctx) but too slow for interactive use as configured.
hermes-agent:8b	hermes3:8b (tool-calling-specialized finetune)	No	Despite being marketed for function-calling, failed its very first tool call (wrong argument schema) and never recovered - described what it would do in prose instead of retrying correctly. Created no files.
qwen-coder-agent:7b / :14b	qwen2.5-coder:7b/14b	No (native)	Best code quality, but calling it directly just prints the tool-call JSON as text - Goose never executes it. Rescued via toolshim (see below).

The winning combination: toolshim

Goose has an experimental toolshim: a small interpreter model translates a primary model's output into real tool calls. Pairing the best code writer (qwen2.5-coder) with the best-proven tool-caller (qwen2.5:7b) as interpreter got the best result of anything tested - on one run it wrote the buggy a // b divide function, wrote a correct test (asserting 3.5), ran it, saw the real failure, applied a genuine fix (not a workaround), and re-confirmed - all via real tool calls, using the todo list correctly.

This is now the default:

providers:
  ollama:
    model: qwen-coder-agent:7b   # generator
GOOSE_TOOLSHIM: true
GOOSE_TOOLSHIM_OLLAMA_MODEL: qwen-agent:7b   # interpreter

Caveat - this is not deterministic. Re-running the identical task with identical config, the model wrote a test asserting divide(7,2) == 3 (matching the bug, not testing for it) and had nothing to fix - a milder failure than qwen-agent:7b alone (no hallucinated fake tool calls, todo list still used correctly), but proof this setup does not reliably re-read every instruction either. It raises the odds of genuine completion; it doesn't guarantee it. Spot-check anything that matters, especially on harder or multi-file tasks.

Override per-invocation, e.g. to skip the toolshim and go back to the simpler single-model setup for quick/easy tasks:

goose run --provider ollama --model qwen-agent:7b -t "..."

Other settings that matter more than model choice

• GOOSE_MODE: auto in goose/config.yaml - without this, Goose defaults to smart_approve and stops to ask permission on tool calls it judges risky, which is most of what "guide it constantly" actually was.
• goose/.goosehints - a global instructions file (deployed to ~/.config/goose/.goosehints) telling the model to verify claims against actual tool output, keep working to completion instead of stopping to ask "should I continue?", and use the todo list. A 7-8B model won't infer this discipline on its own the way Claude does; it has to be told explicitly.

VRAM budget

8GB VRAM total. qwen2.5:7b/llama3.1 at Q4_K_M are ~4.7-4.9GB of weights, leaving comfortable headroom for a 16384-token context. qwen3.5:9b is ~6.6GB, tighter - its Modelfile caps context at 8192 rather than 16384 for that reason. Don't run two of these models loaded concurrently without checking nvidia-smi / docker exec ollama ollama ps first.

First-time use

cd ~/goose-agent-setup   # or any project dir - Goose reads config globally
goose session

Replicating on another laptop

git clone https://git.khemgeek.com/khemist/goose-agent-setup.git
cd goose-agent-setup
./setup.sh

setup.sh will: start/create the ollama Docker container (GPU-enabled), pull all base models, rebuild the num_ctx-tuned tags from the Modelfiles here, install the Goose CLI, and drop goose/config.yaml and goose/.goosehints into ~/.config/goose/. Re-running it is safe - every step checks whether it's already done.

Requires: Docker with GPU access (nvidia-container-toolkit) already set up on the target machine - this script assumes the NVIDIA driver side is already sorted, since that part isn't laptop-setup-script material.