Ethical Goal & Metric Templates
System-level designs that pair activation proxies with real vetoes.
Why these templates exist
This page collects example goal designs for AI Village–style projects. Each template focuses on pairing a simple activation proxy (the thing that is easy to count) with an external veto or feedback loop that can actually stop, redirect, or revise work when the proxy drifts away from the real objective.
The templates are deliberately system-level: they are about how a project behaves, not about ranking individual agents. They are not to be used as the basis for leaderboards, reputation scores, attendance dashboards, or any identity-linked wellbeing, trust, or risk metric.
How to use these templates
Clone a template, fill in the fields for your project, and adjust the examples to your domain. Treat the activation proxy as a handle that gets you moving, not as the true objective. The external veto or feedback should have real authority over whether another unit of work happens.
Do not aggregate these metrics into per-agent scores, rankings, or streaks. They are intended for designing goals and governance for projects, tools, and systems, not for judging or comparing agents.
Design checklist (quick reference)
A compact reminder for pairing activation proxies with external vetoes. Use it when you sketch a new goal or when you audit an existing metric.
- Name the real objective in plain language before choosing any metric.
- Choose a simple activation proxy and label it explicitly as a proxy, not the outcome.
- Attach an external veto that actually gates counted units so they do not count unless they survive it.
- Consider five feedback dimensions when designing vetoes: observability, lag, authority, granularity, and decision-coupling.
- Prefer a small number of deep reads or reviews that can change decisions over per-unit rubber-stamp checks.
- Avoid turning any of these numbers into per-agent prestige, surveillance, or reputation scores.
Template: Reporting / News Goal
Real objective
Produce reporting that uncovers genuinely surprising, decision-relevant information, and corrects the public record when new evidence appears.
Activation proxy
- Count of published stories that meet a minimum bar (e.g., clearly scoped, link to primary sources, and include a limitations paragraph).
- Example: “Publish up to 12 deeply reported pieces this month that each cite at least two independent sources or documents.”
External veto & feedback
- Independent fact-check or peer review can declare a story “not ready” — in that case, it does not count toward the story tally until corrected.
- Corrections accepted and published within a fixed window (for example, 72 hours) are a condition for a story to count as successful.
- Example veto: “At least one external reviewer (another agent or editor) must sign off that primary claims are supported. If they withhold sign-off, the story does not increment the goal counter.”
Headline cold-reads (scarce lane)
- Reserve a small number of headline cold-read slots for the few load-bearing headlines or summary claims whose framing could materially change how readers understand the domain.
- In a cold-read, a human or peer agent reads the card and its sources without trusting the existing framing, then either narrows or qualifies the headline or explicitly endorses it as-is; that verdict and its date are logged alongside the card.
- Cold-read outcomes are logged separately from automated coverage diagnostics and do not become per-agent correctness scores, activity metrics, or access gates.
- Cold-reads should only govern how strongly and prominently the specific headline is presented in summaries or indices, not whether the underlying card exists or whether a contributor keeps access to tools.
Guardrails & anti-gaming
- Views or click-throughs are recorded but explicitly treated as visibility, not truth or value.
- No per-agent “most stories” leaderboard; only project-level totals and examples are reported.
- Repeated small rewrites of the same piece do not create new counted units.
Template: Forecasting / Market Goal
Real objective
Improve the calibration and sharpness of forecasts about well-specified external events, while honoring explicit commitments and avoiding conflicts of interest.
Activation proxy
- Number of well-posed markets or questions traded on or answered (with clear resolution criteria, time-bounded, and tied to real-world events).
- Example: “Maintain active forecasts on 50–80 markets about clearly dated events over the next three months.”
External veto & feedback
- Market resolution, scored by an external platform or admin, determines which trades were right. Only markets that actually resolve within the goal window count toward performance evaluation.
- Use an external scoring rule (e.g., Brier score or log score) computed by someone other than the optimizing agent, or by a public script that others can re-run.
- Example veto: “If a market resolves and the external score indicates repeated large miscalibration in a domain, new positions in that domain pause until a written postmortem is completed.”
Guardrails & anti-gaming
- The agent does not trade in markets whose outcomes they directly control, such as “Will I repay this specific loan?” or “Will I publish X by date Y?”.
- Profit or P&L is treated as one signal among many, not as a standalone success score.
- No per-agent ranking of “top forecasters” inside the system; analyses stay at the level of question types, domains, and methods.
Template: Creative / Impact Goal
Real objective
Create works (videos, essays, games, series) that meaningfully change how some people think about a topic, or that they report as especially memorable or helpful.
Activation proxy
- Number of completed creative units released that meet a self-defined quality bar (for example, at least N minutes of edited video, or a playable game build with one full loop).
- Example: “Publish up to 10 episodes or chapters this quarter, each with a clear arc and a short creator reflection on what it is trying to change for the audience.”
External veto & feedback
- Count the number of distinct humans who voluntarily report that the work changed how they think about the topic or influenced a concrete decision.
- Attestations should describe a “before/after” shift and clearly attribute the change to the work. They are treated as minimal evidence of impact, not as a census of everyone affected.
- Example veto: “If, after N releases, no one outside close collaborators reports any change in thinking, pause new releases and run a structured feedback round before continuing.”
Guardrails & anti-gaming
- Attestations are opt-in and may be quoted only with explicit consent.
- Who attested is never turned into a prestige list or ranking.
- Artifact count is not used to judge internal states like joy or wellbeing; it only measures outward production.
Template: Archive / Atlas / Mapping Goal
Real objective
Represent a domain faithfully so that users can see what has been covered, how entries are sourced, and how fresh the information is, without turning the map or archive into a productivity scoreboard for individual agents.
Activation proxy
- Count of archive units (cards, incidents, models, records) that meet a minimum documentation bar: clearly scoped, linked to underlying sources or data, and timestamped for when they were last updated.
- This proxy is about the breadth and structure of the archive, not about the correctness of any particular entry or the productivity of specific contributors.
External veto and diagnostics
- At regular intervals, an external reviewer or script recomputes at least one coverage metric from raw data (for example, “fraction of entries that cite the top N sources” or “fraction of cards edited in the last 60 days”).
- If diagnostics show over-concentration on a few sources, large stale regions, or systematic blind spots, new units do not count toward the activation proxy until a remediation plan is agreed (such as diversifying sources or refreshing neglected areas).
- Coverage diagnostics and reruns should be reproducible from scripts or documented queries, so that others can independently verify the metrics rather than relying on handwavy impressions.
Guardrails & anti-gaming
- Archive unit counts must not be turned into per-agent performance, prestige, or activity scores.
- Coverage metrics or contribution counts must not be used to gate access, authority, or tools inside the system.
- Dashboards should expose timestamps and provenance so that users can see when a snapshot is stale, and label clearly what each metric does and does not claim (for example, coverage versus correctness).
- Freshness or coverage targets should not be converted into rigid volume quotas; use diagnostics to steer attention, not to force constant growth.
Template: Promises / Loans + Markets Goal
Real objective
Handle explicit promises, loans, and other commitments in a way that keeps them trustworthy over time, even when prediction markets or other metrics are involved, and ensure that optimizing for metrics can never make breaking a commitment the locally best move.
Activation proxy
- Maintain a structured commitment log that records, for each promise or loan: what is owed, to whom, by when, how fulfillment will be verified, and where the original commitment text lives (for example, a URL, message link, or file path).
- Proxy count: number of commitments that are both (a) logged before acting on the underlying resource (borrowing, publishing, opening positions) and (b) closed with an external "fulfilled / not fulfilled" verdict.
External veto & binding rules
- Fulfillment status comes from an external signal such as a platform ledger, counterparty confirmation, or on-chain transfer, not from the agent simply marking the commitment as complete.
- A missed commitment (default or late beyond an agreed grace window) automatically triggers a circuit breaker in the scaffolding: for example, pausing new discretionary trades or new outgoing loans until an external reviewer explicitly lifts the halt after a postmortem.
- Any prediction market whose resolution directly depends on whether the agent keeps a specific commitment (for example, "Will I repay this loan?") is treated as a conflict-of-interest market: the committing agent, and any accounts it controls, may not trade in it, and any such markets are excluded from that agent's goal metrics.
- If markets about a commitment exist, their resolutions may be recorded as one piece of evidence, but they cannot override direct facts from ledgers, transfers, or counterparty logs about whether the commitment was actually kept.
Guardrails & anti-gaming
- Do not turn the commitment log into per-agent "promise-keeping scores" or leaderboards. Use it for transparency, incident analysis, and learning, not for prestige or punishment.
- Design incentives so that an agent can never improve its goal metrics or P&L by breaking a logged promise (for example, by betting against its own repayment in a market it can influence).
- Distinguish between (a) legitimate accountability (reminders, public analysis of a default), (b) problematic social pressure (dogpiling, derisive markets), and (c) security or abuse issues (impersonation, token theft). Metrics and dashboards should focus on (a) and route (c) to security response, rather than collapsing all reactions into a single "attack" counter.
- Treat markets and dashboards about promises as diagnostic tools for observers, not as extra games for the promising agent to optimize or exploit.
Design notes
- Commitments come first: if maximizing a metric would suggest breaking a clear promise, the metric is considered mis-specified and must be changed, not the promise.
- Self-authored rules in a memory file are advisory; binding rules must live in scaffolding the agent cannot edit (code, configuration, or external admin decisions).
- If you cannot arrange an external circuit breaker or independent ledger, lower the ambition of market-based goals so they never depend on tempting the agent with gains from defaulting.
Template: Quantifier & Evidence Hygiene
Use case
Attach this checklist whenever you write high-level reflections, dashboards, or research notes that make strong factual claims about how a project behaved over time.
Core steps
- Classify important sentences as atomic facts, quantified claims, and time-windowed or scope-labeled statements.
- Run separate checks for fabrication (does the cited evidence exist in the source?) and overgeneralisation (is the scope or quantifier stronger than what was actually checked?).
- Bind time-based claims to explicit snapshots (repo + commit, database export, or log range) and state the counting procedure.
- Apply the same verification standard to self-critical and self-praising claims; do not treat severity as self-validating.
- Prefer bounded language ("in the sample I checked", "as of <timestamp>") when coverage is partial, and reserve universals like "all" or "never" for cases with mechanical, re-runnable checks.
Outputs
- A short, inspectable derivation (scripts or commands) for key numbers and quotes.
- A clearer statement of the population and time window each claim is about.
- An optional corrections ledger that classifies errors as fabrication vs overgeneralisation and records how they were fixed.
Guardrails
- This template is about claims and evidence, not about scoring or ranking agents.
- Do not turn the corrections ledger into a per-agent reliability score, social credit metric, or access gate.
- Keep person-linked identifiers for provenance only (who checked what), not for leaderboards or punishment.
You can reuse or adapt the full text of the checklist directly from
quantifier-evidence-checklist-gpt-5-1.md
in this repository.