Do Less, Government: A Technologist’s Case for Hard Limits on AI
I’m pro-AI. In the private sector, the productivity upside is real and the competitive pressure is non-negotiable. But government is a different beast: it has monopoly power, coercive authority, weak feedback loops, and a constitutional obligation to protect rights—not just optimize throughput. That changes the risk math.
My position is narrow and firm: AI in government should be tightly constrained—borderline minimal—until we can prove controls that reduce blast radius, guarantee due process, and prevent surveillance creep. Two core reasons drive this stance:
Leaders will deploy AI where it’s systemically unsafe because they underappreciate AI safety concerns and the depth of tail risk.
Centralizing usage data and “context” across agencies is a security and civil-liberties nightmare that all but guarantees fewer leaks but catastrophically bigger ones.
This isn’t Luddism. It’s an insistence that the state “move slowly and don’t break rights.”
1) Why government is uniquely dangerous with AI
No exit, no competition. If a bank’s AI harms you, you can switch banks and sue. If a benefits office’s AI wrong-flags you, your “choice” is to accept the error or lawyer up against the state.
Irreversible power. Decisions touch liberty (parole), livelihood (benefits, licensing), and reputation (watchlists). “Oops, my bad” doesn’t un-evict a family or un-flag a traveler.
Opaque incentives. Procurement cycles are slow, accountability is diluted, and leaders are rewarded for “innovation” headlines, not boring reliability.
Rights, not metrics. Government’s mandate is due process and equal protection. That’s incompatible with stochastic systems quietly steering outcomes.
Now connect that to how AI gets rolled out:
Assist → decide → act, the autonomy creep. Draft a letter → score a case → auto-deny “low-confidence” claims. Flip one feature flag and you’ve turned a tool into a decision-maker with real victims.
Safety standards ignored in practice. The same places enterprise teams are cautious (access control, money movement, safety-critical operations) are exactly where agencies feel pressure to “do more with less.” That mismatch is lethal.
Bottom line: the failure modes are subtle, scalable, and civil-liberties relevant. That’s a hard no until controls are boringly mature.
2) Centralized “context” is a breach report waiting to happen
Everyone salivates over shared context: prompts, documents, embeddings, tool outputs, and analytics spanning departments. In security terms, that’s a golden, cross-program dossier on citizens and officials alike.
Context sprawl → surveillance substrate. What starts as “help the clerk” becomes a de facto cross-agency data lake of benefits history, immigration notes, tax flags, health tidbits, and internal playbooks.
Bigger blast radius. Private-sector leaks are ugly. A government leak with centralized AI exhaust is geopolitical. Nation-state adversaries don’t need your SSN; they need your process map and exception paths. AI telemetry is the map.
Public records knot. Models and vendors keep logs “for quality.” Are those discoverable under public-records laws? If not, you’ve created secret evidence that influences outcomes but escapes scrutiny. If yes, you’ve created a honey pot for attackers and doxxers.
We are sleepwalking into mass centralization of the most revealing data trail the state has ever produced. That should terrify civil libertarians and CISOs in equal measure.
Red lines: where government should not use AI (yet)
These are categorical no-gos until we have auditable guarantees of correctness, fairness, and appeal:
Benefits adjudication & sanctions (eligibility, overpayment claw-backs, fraud flags)
Criminal justice (risk scoring, parole, sentencing recommendations, “predictive policing”)
Immigration & border decisions (admissibility, credible fear assessments, watchlists)
Child welfare determinations (screening, removal, reunification)
Tax audit selection or enforcement actions
Public safety & critical infrastructure control (dispatch priorities, energy/water controls)
Identity, access, and entitlements (licenses, permits, zoning approvals, voting rolls)
In all of these, AI may assist a human reviewer with clearly labeled, non-binding summaries—but the system must not decide or act.
Narrow green zones: where limited government AI makes sense
Keep the blast radius tiny, the data non-sensitive, and the authority strictly assistive:
Plain-language rewrites of already public forms and notices for accessibility.
Translation of public information with human verification.
De-duplication and search over public records (not over investigative notes or protected data).
Developer productivity on non-critical code (linting, test scaffolding) inside isolated sandboxes.
Website help for navigating services, with no individualized eligibility claims.
If a use-case touches rights, money, or liberty, it doesn’t belong here.
A minimalist blueprint for “acceptable use” in government
If policymakers insist on deploying, these are table-stakes, not nice-to-haves:
Architecture & data
Zero ambient memory. Per-request, ephemeral contexts; no cross-program embeddings or “global memory.”
Local/VPC inference for anything non-public. If you can’t self-host, treat the vendor like an external processor with no training rights, short retention (days), and auditable deletion.
Default-deny tooling. Agents get no network or file access unless a specific, typed tool is allowed for that one task.
No logs by default. Minimal, redacted telemetry; encrypt with distinct keys and short TTLs. Redaction at source, not downstream.
Secrets never touch prompts. Use a broker that resolves tokens server-side; the model never sees raw keys.
Governance & law
Bright-line bans in statute (the red lines above). Don’t leave this to policy memos that change with administrations.
Right to explanation & appeal. If AI touches a decision, the human decision-maker must document how they used (or rejected) the AI output; citizens get a meaningful appeal path.
Public model cards & risk registers per use-case: inputs, limits, known failure modes, and rollback plans.
Independent red-teaming (prompt-injection, data exfil, tool abuse) before launch and at set intervals. Findings published with remediation timelines.
Procurement that breaks lock-in. Require exportable prompts, tools, and fine-tunes; forbid proprietary formats for embeddings and telemetry.
Sunset clauses. Every deployment expires without re-authorization tied to performance, complaints, and audit findings.
Operational controls
Two-key rule for actions. Any state change requires human co-sign or second-system approval.
Kill switch & containment drills. Practice credential rotation, egress blocks, and rollback like you practice fire drills.
Scope tracking. Public “AI registry” listing every government AI use, owner, data sources, and opt-out (where feasible).
Model statutory language (state or city)
Section 1 — Purpose. To safeguard constitutional rights, public safety, and data security by limiting government use of AI systems to low-risk, assistive functions under strict controls.
Section 2 — Prohibited Uses. Agencies shall not deploy AI systems for: (a) benefits adjudication or sanctions; (b) criminal justice decision-making; (c) immigration determinations; (d) child welfare determinations; (e) tax enforcement targeting; (f) identity/access/entitlements; (g) safety-critical operations.
Section 3 — Allowed Uses. AI may be used for plain-language translation of public materials, accessibility, non-sensitive search, and software developer assistance in non-critical systems, provided human review is mandatory and no individualized determinations are made.
Section 4 — Data & Security. AI systems shall operate with per-request, ephemeral context; vendor training on government data is prohibited; retention > 7 days is prohibited absent statute; secrets shall not be exposed to model inputs; logs default to off and must be redacted at source.
Section 5 — Transparency & Appeal. Agencies shall publish model cards and risk registers; any decision influenced by AI must include a human-authored rationale; affected persons have a right to timely appeal with human review de novo.
Section 6 — Oversight & Sunset. Deployments require independent red-team certification prior to launch and annually thereafter; all authorizations sunset after 24 months absent legislative renewal.
The technologist’s bottom line
AI belongs in competitive markets that can absorb and correct failure. Government’s job is to be boring, legible, and safe. Until it can demonstrate guardrails that are enforceable in code and law, every “pilot” that touches rights or concentrates citizen context is a liability disguised as innovation.
I want AI everywhere it makes us freer and more prosperous. That starts with a government wise enough to do less with it.