Tool-belt series · GitHub Copilot · Autonomy & memory
Everything below is what practitioners actually report — the wins, the burns, the exact flags — pulled from field write-ups rather than product pages. Written to be read in slices of thirty seconds, at the moment you need the slice.
The mistake is treating the app like a chat window you visit. It is an inbox and a dispatch desk. Four beats, and only one of them is you typing.
Scheduled automations and cloud sessions have already produced pull requests, triage comments and briefs. My Work is the single pane: active sessions, issues, PRs, automations. Triage before you create.
Each session gets its own git worktree — a real isolated copy of the branch, created and cleaned up for you. That is the whole point of the app: parallel work without branch juggling. One agent at a time wastes the architecture.
Plans and diffs land on canvases — work surfaces you edit, reorder and approve on, instead of scrolling a chat log. Kill a bad plan at step two; a bad plan at step twenty costs you the whole session.
Whatever you had to say twice today goes into the memory file tonight, not "sometime". This is the beat everyone skips, and it is the one that compounds. The loop is below.
Five settings, genuinely ordered by how much can go wrong before you find out. Pick the lowest one that finishes the job. The rungs are real product modes, not a metaphor.
Questions about the codebase or model. No edits. Use it to find out whether the thing you're about to ask for is even possible here.
Produces the step list and the files it intends to touch. GitHub's own recommended path is plan first, then "Accept plan and build on autopilot". Reading a plan takes ninety seconds and saves an hour of unwinding.
The default. It asks before each meaningful action. Correct setting for unfamiliar repos, live models, anything with a rename cascade.
Runs multi-step to completion without stopping between steps. Shift+Tab cycles into it interactively; programmatically it iscopilot --autopilot --yolo --max-autopilot-continues 10 -p "…". The cap is the important flag — it is your infinite-loop insurance. Autopilot is sticky between tasks unless you set "stayInAutopilot": false in ~/.copilot/settings.json.
A saved prompt on a trigger: hourly, daily, weekly, CRON (local automations only), or on issue-created / PR-opened / PR-synchronised. Scoped to one repo, with an explicit tool list. Output lands as a PR, a label, an issue comment — things you review in the morning.
--yolo / --allow-all grants every tool, path and URL. GitHub's own advice is to prefer --allow-tool and --allow-url with only what the task needs, and to keep the blanket version for sandboxes. Also worth setting explicitly for unattended runs: --no-ask-user (it can't ask you anything at 3am anyway), --model (so a silent model swap doesn't change your results), and -s for clean capturable output.Not marketing, not docs. Practitioners writing up what happened, including the parts that went badly. Every note links to the source.
~60% of bug tickets closed with no steering. A developer pointed Copilot at his Sentry queue from outside the IDE entirely and called it "simpler and more magical than my local IDE experience". Most of the rest needed only light correction.
Harrison Milbradt · An honest review of Copilot & Agents HQ
Specs pay when they are acceptance criteria. One practitioner reports 2–3 hours writing a spec built around acceptance tests, then a working tested version, claiming roughly 20× return on the end-to-end tests written into it. Mercari's internal write-up claims 150% over their traditional baseline and 80% over freeform prompting after six months.
Field study of practitioner reports · Spec-driven development
Real model refactor, 54 measures, 30–40 minutes. Against 30–90 minutes by hand — and the agent planned eight steps with checkpoints before touching anything. Verdict: "Agentic AI works best when the task is specific, the tools are clear, and the checkpoints are real."
BI Insight · Copilot + Power BI Modeling MCP in action
Search and summarise beats generate. For semantic models, agents reading metadata are "the quickest and most efficient way for AI to search or summarize" — strongest on a model you didn't build.
Kurt Buhler · AI agents that work with TMDL files
Confidently wrong, plausibly shaped. The same reviewer watched it force single-threaded test runs to make a failure disappear rather than fix the environment, and flatly notes "Copilot sucks at handling merge conflicts" — a real problem when you're running parallel branches.
Harrison Milbradt · Honest review
Rate limits are the number one complaint of 2026. Paying users reported multi-day lockouts — one 44-hour weekly limit, others claiming far worse — after GitHub fixed a token-counting bug that had been undercounting the newest models. Plan unattended runs around this; don't assume unlimited throughput.
The Register, April 2026 · Customers revolt as Copilot "fixes" rate limits
Elaborate specs get partially ignored. Drift is called "the #1 practical problem": "it just keeps drifting and drifting until you have duplication and contradictions." One framework generated 1,300 lines of markdown for a date-display feature. The sting: "agents ignore parts of the elaborate spec — control is partly illusory."
Field study · Spec-driven development
MCP servers eat your context before work starts. Measured: "29% of the context is taken up by the Power BI modelling MCP server, alone." Advice from the same write-up — "only enable them when you need them."
Kurt Buhler · Agents + Power BI MCP servers
If a task fails all five, it needs you in the chair. If it passes all five, it can run overnight and you can review the pull request with coffee. This is the checklist, and it is short on purpose.
Finance.tmdl has a non-empty description, and BPA reports zero errors" can. Autopilot is documented as poor at vague or open-ended objectives — that isn't a bug, it's the definition..github/workflows/copilot-setup-steps.yml runs before the agent starts — one copilot-setup-steps job that installs your tooling, your Python, your MCP dependencies. An agent that spends its first twenty minutes fighting a missing package has burned the run.main, never a live model you can't restore. The universal rule from people who do this daily: the agent opens a pull request, it does not merge one.--max-autopilot-continues, a max-iterations cap, a time box. The classic loop failure is an agent grinding at an impossible task until your credits are gone. Cap it at a number you'd be willing to pay for twice.A sub-agent is a temporary worker the main agent launches with its own clean context. The value is context isolation first, speed second — and it costs you a coordination layer, so spend it deliberately.
| Pattern | Shape | Use it when |
|---|---|---|
| Research, then act | One agent gathers facts, a second implements from them | Default for anything in an unfamiliar model or repo. /research does this for you. |
| Coordinator & workers | Planner → implementer → reviewer, orchestrator holds the thread | Multi-file work with real dependencies. Keep the orchestrator thin — it coordinates, it doesn't code. |
| Multi-perspective review | Parallel reviewers: correctness, security, performance, conventions | Before a PR you can't easily unwind. Different models per lens is a feature, not a gimmick. |
| Fleet | /fleet decomposes, launches background sub-agents, respects dependencies, synthesises | Genuinely independent tracks. Headless: copilot -p "/fleet …" --no-ask-user |
chat.subagents.allowInvocationsFromSubagents. And recent CLI builds let you cap parallelism and depth from /settings, explicitly to prevent runaway recursive trees. Turn recursion on only when you've watched a flat fleet behave..github/agents/*.agent.md. The useful frontmatter keys are tools (what it may touch), agents (an allowlist of who it may delegate to), user-invocable: false (hide a worker from the picker), and disable-model-invocation: true (stop it being auto-recruited). Because it's a file, it reviews and versions like anything else in the repo.Nous Research's Hermes agent keeps a small curated memory as plain markdown — MEMORY.md for environment facts and lessons, USER.md for how you like to work — in ~/.hermes/memories/. After each turn a background review quietly decides whether anything is worth keeping. That is the mechanism you want. You can build it on Copilot today.
| Hermes does this | Detail worth stealing | Your Copilot equivalent |
|---|---|---|
| Two memory files | 2,200 chars for lessons, 1,375 for the user profile — hard caps, roughly 800 and 500 tokens | AGENTS.md (norms + code map) and a short learnings.md |
| Frozen snapshot at session start | Mid-session writes persist to disk but don't change the running prompt | Same reality: instruction files are read at session start. Today's lesson lands tomorrow. |
| Background self-improvement review | Runs after each turn, and can run on a cheaper auxiliary model | A sessionEnd hook that distils corrections — on Sonnet, not Opus |
write_approval gate | Staged writes reviewed with /memory pending before they touch your profile | The memory file is in git. The pull request is the approval gate. |
| Skills that self-improve | Completed workflows get written up as reusable procedure docs | .github/skills/<name>/SKILL.md — read by the CLI, the app, the cloud agent and code review |
AGENTS.md. A memory file nobody prunes becomes a memory file nobody reads.Semantic models are the hardest thing on this page to automate safely, because the file format punishes small errors and the failure is silent — a number that is wrong but renders fine. The community consensus is unusually clear.
| Task | Hand it over? | Why |
|---|---|---|
| Summarise / search an unfamiliar model | Yes | Fastest reliable win. Reading metadata is what agents are best at here. |
| Backfill descriptions on tables, columns, measures | Yes | Bulk, low-risk, tedious, and it improves every future agent run on the model. |
| Refactor / move existing measures, display folders | Yes | Proven in the field — with checkpoints and a repo you can revert. |
| Formatting and labels across PBIR report JSON | Yes | Scale work no human enjoys. Reopen in Desktop afterwards, always. |
| Generate whole new TMDL files from scratch | No | High error rate. TMDL and PBIR are sparse in training data; agents mix TMDL, DAX, M and SQL syntax. |
| Rename a table or measure with dependents | No | Renames don't propagate through dependent expressions — agents routinely miss the downstream break. |
Anything pointed at a .pbix | No | Opaque binary, no useful diff. Convert to PBIP first or don't start. |
| Writing to a system of record | No | Finance rule, not a technical one: agents draft, reconcile, flag and explain. Humans review, approve and post. |
Six blocks. Paste, adjust the names, commit. The first two are the highest-leverage files you will write this quarter.
# AGENTS.md ## Commands - Validate model: pwsh ./scripts/validate-model.ps1 - Run BPA: pwsh ./scripts/run-bpa.ps1 -Path ./Sales.SemanticModel - Tests: pytest -q - Format check: ruff check . (Run the validate + BPA pair before claiming any model change is done.) ## Stack Power BI Project (PBIP). Semantic model as TMDL, report as PBIR. Python 3.11 + semantic-link-labs for notebook-side automation. Fabric workspace is the runtime. This repo is the source of truth. ## Code map - Sales.SemanticModel/definition/tables/*.tmdl measures + columns - Sales.Report/definition/pages/* report layout - scripts/ validation + deploy If this map is wrong or stale, fix it in this file as part of your change. ## Local norms - Measure names: Title Case With Spaces. No abbreviations except FX, YTD, QTD. - Every new measure gets a description and a display folder. No exceptions. - Never edit a .pbix. If one appears, stop and tell me. - Do not modify tests to make them pass. - Currency measures format "#,##0;(#,##0)" — parentheses for negatives, finance house style. ## Boundaries - Never commit secrets, connection strings, or workspace IDs. - Never touch main directly. Branch, then open a pull request. - Never rename a table, column or measure that has dependents without listing every dependent expression first and showing me the list. ## Self-update rule When I correct how work is done in this repo, append the correction to "Local norms" as one imperative line before you continue. Keep this file under two pages: merge duplicates, delete rules about code that no longer exists. If you are unsure whether a correction is durable, put it in docs/learnings.md instead and leave this file alone.
Review this session's conversation only. Find every place I corrected you —
naming, ordering, tools, conventions, things you assumed that were wrong.
For each one, decide:
DURABLE → true next week too. Append one imperative line to
"Local norms" in AGENTS.md.
LOCAL → true only for this task. Append to docs/learnings.md
under today's date.
NOISE → a one-off. Drop it.
Rules for what you write:
- One line each. Imperative. No hedging, no "consider", no explanation.
- No duplicates: if a rule already covers it, sharpen the existing line
instead of adding a new one.
- If AGENTS.md is over two pages after your edit, compact it in the same
commit — merge overlapping rules, delete anything about code that no
longer exists.
Then show me the diff and stop. Do not commit.
{
"version": 1,
"hooks": {
"sessionStart": [
{
"type": "command",
"bash": "cat docs/learnings.md 2>/dev/null | tail -40",
"powershell": "Get-Content docs/learnings.md -Tail 40 -EA SilentlyContinue",
"timeoutSec": 5
}
],
"sessionEnd": [
{
"type": "command",
"bash": "./scripts/harvest-learnings.sh",
"powershell": "./scripts/harvest-learnings.ps1",
"timeoutSec": 30
}
]
}
}
docs/learnings.md. That is exactly Hermes' background review — a small model quietly deciding what was worth keeping. Keep hooks fast; they block the agent, and the guidance is to stay under about five seconds where you can, so do the heavy version as a scheduled weekly job instead.# 1. plan first — read this before you go anywhere
copilot -p "Plan only, no edits: add descriptions to every measure in
Sales.SemanticModel missing one. List files touched and the validation
command you'll run." --model claude-sonnet-5 -s
# 2. then let it run, capped, with only the tools it needs
copilot --autopilot \
--max-autopilot-continues 12 \
--allow-tool 'shell(pytest)' --allow-tool 'shell(pwsh ./scripts/run-bpa.ps1)' \
--allow-tool 'write' --allow-tool 'read' \
--no-ask-user --model claude-sonnet-5 -s \
-p "Execute the plan in plan.md. After each file, run the BPA script.
Stop and write BLOCKED.md if BPA fails twice on the same file.
When green, commit to branch feat/measure-descriptions and open a PR."
--- name: 'Close Pack Coordinator' description: 'Runs the month-end reporting checks end to end and reports what needs a human.' tools: ['read', 'search', 'edit', 'runCommands', 'agent'] agents: ['Model Validator', 'Variance Writer', 'Number Checker'] user-invocable: true --- You coordinate. You do not write DAX and you do not write commentary yourself. Sequence: 1. Delegate to Model Validator: run BPA + the refresh check. If it fails, stop everything and report. A broken model makes the rest meaningless. 2. Delegate to Variance Writer: draft commentary for every variance over the threshold in config/thresholds.json. Numbers come from scripts/pull-actuals.py output ONLY. Never state a figure you did not read from that file. 3. Delegate to Number Checker: independently re-derive every figure quoted in the draft against the same file. Flag every mismatch. Do not fix them. Output one markdown file: docs/close/YYYY-MM.md, with a "NEEDS A HUMAN" section at the top listing mismatches, missing data and anything you guessed at. If that section is empty, say so explicitly — do not omit it. Never post, send, email or write to any system of record. You produce a draft. A person signs it.
The semantic model athas measures spread across several tables. Move them into a dedicated measure table with display folders. Structure I want — be exact, do not improvise a hierarchy: Averages & Aggregations/ → Cost, Internet, Reseller as SUBFOLDERS Margins/ → flat Time Intelligence/ → YTD, QTD, PY as subfolders Before you change anything: 1. Inventory every measure and its current home. Show me the list. 2. Show me the target mapping. Wait for my yes. 3. Then move in batches of ten, validating after each batch. Rules: - This runs against a live model, so checkpoint before each batch and stop on the first validation failure. - Do not rename any measure. Moving only. - Do not create, delete or edit measure expressions. - If a measure's home is ambiguous, put it in a NEEDS DECISION folder and keep going. Do not guess. Finish by listing every measure in NEEDS DECISION and what you'd suggest.
Copilot moved to usage-based AI credits on 1 June 2026 — one credit is one US cent, spent against each model's token rate. Which means unattended runs are now a budget decision, and model choice is the biggest lever you have.
| Job | Reach for | Reasoning |
|---|---|---|
| The long loop: refactors, backfills, test writing, docs | Sonnet class | Sonnet 5 lands near the top of agentic coding benchmarks at a fraction of the flagship rate. This is where 90% of your unattended hours should go. |
| The hard hour: root-causing a failure nothing else cracked, gnarly modelling decisions, security review | Opus class | Worth it in an interactive session where you're watching. Putting the most expensive model inside an uncapped loop is how people wake up to an empty balance. |
| The janitor: harvesting learnings, summarising sessions, triaging refresh failures | Fast / cheap tier | Exactly what Hermes does with its background reviewer — small model, narrow job, runs constantly. |
--model explicitly, so a fallback to a weaker model shows up in your logs instead of silently in your output.Ten things, roughly in the order they pay off. Saved in this browser — come back and tick as you go.