Yesterday’s AI Unconference in Sydney ran as open, participant-led discussions rather than talks. These notes cover three of them: agentic software development, security, and local LLMs.
Session 1: Agentic software development
The main question was where humans still sit in the loop, and how to remove errors before an agent or a person has to review anything.
Review is shifting from correctness to speed. One participant has moved to roughly 80% speed, 20% correctness on pull request reviews. An agent does the first scan, and the prompt is refined each time a colleague pushes back on a false positive. Two review agents once argued with each other; the fix was a rule to back off after two explanations. Spec and PRD changes are still read by a human, using a generated HTML walkthrough with one tab per requirement, a diagram, and opinions from simulated product, technical and security reviewers.
Eliminate classes of error early. The group favoured shifting checks left so review agents have less to find:
- Hooks at the end of an agent’s turn are the only reliable enforcement. Rules in a prompt get followed until they are inconvenient.
- Enforcing cyclomatic complexity (lizard, gocyclo) cut token use by about 25%. Agents faced with a tangled function tend to wrap it in another large function rather than edit it.
- Linting SQL migrations with squawk flags irreversible changes for human sign-off.
- getlgtm.dev uses a fast model to judge whether a PR is finished and safe before it consumes expensive CI minutes, which matters for Mac app projects.
Handoffs between agents. Several people are still the copy-paste middleman between a coding agent and a review agent. Approaches discussed: an event-driven server where the task, not the session, accumulates context and status changes trigger the next agent; named Claude Code sessions that can read or message each other; nodeterm for linking context across Claude, Codex and other harnesses over a Unix socket; and pushing all review into CI with Claude Code actions, CodeRabbit and getlgtm.dev. A layered “survivability onion” of checks was the shared mental model.
Running out of RAM, not tokens. More than about eight sandboxed agents on a laptop exhausts memory, so some have moved sessions to the cloud. Few people still edit code in an IDE.
OpenRouter caution. Some upstream providers serve over-quantised open-weight models. Symptoms appear a few tasks into a long session as hallucinated personas and garbled output. Tracing the upstream provider, allowlisting providers, and capping repeated identical tool calls were the suggested defences.
What it costs. Personal spend ranged from a basic ChatGPT and Claude subscription to a $140/month Max plan plus about $70 on OpenRouter. One engineering director reported $2,000 in a typical month, and $7,000 during a full rebuild. Of that, $1,000 went on investigating a memory leak that now saves $20,000 a month on AWS. His company budgets $1,500 per engineer per month; light users subsidise heavy ones. Cheaper models often cost more overall because of retries, and there was some agreement that people default to the newest frontier model as a hedge against failure.
Session 2: Security
The consensus was that AI has sharply raised the volume of security findings, and that putting an LLM inside an application changes what that application fundamentally is.
Open source is flooded with reports. Security reports to the Apache Software Foundation went from about 20 a month at the start of the year to 400 or 500 a month now. Apache Airflow built triage skills that became a separate project, Apache Magpie. Pointing an agent at the backlog is not enough; it needs documents defining what is and is not a security issue in that project.
LLMs as weird machines. From language-theoretic security: software contains latent “weird machines” that crafted input can program, and LLMs with tools are now finding them. An LLM in an app makes the app effectively Turing complete, so there is no general way to decide whether an incoming message is safe. Suggested responses:
- Route all model traffic through a gateway that logs everything and watches sessions for developing anomalies.
- Have the model synthesise a program that can be compiled and checked, then run that, rather than trusting the agent’s actions directly. One participant does this for ledger computations over millions of rows.
- Watch regex in particular. Python’s engine is powerful and models write alarming patterns.
Sandboxing and permissions. Granted agent permissions do not expire, so they need manual cleanup. Deleting sandboxes after use limits what persists. Magpie runs with no environment credentials, sandboxed file and network access, restricted tools, and human approval for any external write. Views differed on containers versus VMs; native options include macOS containers and WSL on Windows, with agents confined to dedicated partitions or an external SSD. One team wraps git in a substitute the agent cannot tell apart from the real thing.
Monitoring. CrowdStrike flagged and killed processes during a run with 23 sub-agents. One participant is building a monitor that labels agent conversations from three angles (Bayesian anomaly, epistemic quality, game theory) and distils the frontier models’ labelling into a fast, small model. His finding: distilled models lose some capabilities entirely, game theory in particular, while the largest frontier models (Fable and Astra) and Sonnet came closest to his own judgement. Point-to-point monitoring works; many agents interacting through a shared board remains unsolved.
The great decompiling. Models now decompile binaries well, including a 31-year-old file one participant was working on. Any code shipped to a client becomes an attack surface, which may push more logic server-side and back into secure enclaves.
Are we creating more vulnerabilities? Models trained on bad code generate a lot of new code, and human review was already inconsistent. A line from The Reverse Centaur’s Guide to Life After AI was quoted: that AI is the asbestos we will be removing from software for decades. One counterview was to treat software as disposable and keep the spec as the durable asset, ideally a spec that a cheaper non-frontier model can build from.
Session 3: Local LLMs
The room agreed local models are good enough for many tasks, provided you match the model to the task and pay attention to memory, not just parameter count.
The KV cache is the real constraint. Every layer keeps a record of every token, and on consumer hardware the cache is often larger than the loaded model. Ways to shrink it:
- Replace most attention layers with linear layers (RWKV-style), which reuse a few fixed slots. Nemotron is mostly linear with only a few attention layers. RWKV-7 attention holds up to about 32k tokens; a couple of standard attention layers cover the rest.
- Quantise the cache to 8, 4 or even 2 bits.
- Use sparse attention that loads only the most relevant entries, or DeepSeek-style low-rank compression.
What people are running.
| Setup | Model | Speed / use |
|---|---|---|
| MacBook Air | Bonsai, a ternary distillation of Qwen 27B | Usable for many tasks |
| PC with an 8 GB GPU | Bonsai, fitted to the GPU by Claude | 30 to 40 tokens/s |
| Old work laptop, 24 GB | Qwen 27B | 9 to 10 tokens/s, run overnight |
| M5Stack module (AX630C, 4 GB) | Qwen 1.5B | Translation across 100+ languages |
| Phone | RWKV 1.5B | On-device chat |
The night shift. Slow inference matters less when nobody is paying per token. One writer ran a dedicated home Mac over seven nights, with the model leaving itself a progress file each morning, and turned about 30 years of writing into a 200,000-word synopsis. Another uses a harness called Juma that breaks tasks into verifiable subtasks, retries failures, and leaves a reviewable list by morning. About one task in ten still needs a frontier model.
Routing by task. A common pattern is to plan and write specs with a frontier model and execute locally. One participant pairs Nemotron with the Hermes harness for most work, including coding, and calls Qwen 27B for planning. Keeping a catalogue of which model handles which task well makes routing simpler.
Give small models tools. Local models are universally bad at maths, so hand calculations to Unix bc, which is deterministic, arbitrary-precision and programmable. An offline Wikipedia lookup lets a model sense-check itself. Qwen needs only a one-line example per tool in the system prompt.
Hardware. RAM matters most. AMD APUs (Strix) work well; Intel integrated graphics do not. Cheap mini PCs that take whatever DDR4 or DDR5 you have spare are a practical route. At the other end, Meta’s open-weight Spark needs around eight 100 GB-class GPUs, which is not local for most people.
Common threads
- Deterministic checks beat instructions. Hooks, linters, complexity limits, compiled-and-checked programs and bc all came up as ways to stop trusting the model on things a tool can verify.
- Frontier for thinking, cheaper models for doing. Specs and planning on the largest models; execution, triage and labelling on distilled or local ones, once you know where they fall short.
- The spec is the asset. Several people treated code as replaceable and the spec, the review rules and the security definitions as what is worth maintaining.
- Memory is the bottleneck. Whether it is sandboxed agents on a laptop or a KV cache on a GPU, RAM ran out before anything else.
