Key Concepts
One-change rule: Change one thing, run it with the same input, and compare the old result to the new one. If it improves, keep it. If not, revert and try a different change. Each run teaches you something specific.
Instructions, knowledge, structure: The three places a problem can live. Instructions fix how the agent thinks, knowledge fixes what it knows, and structure fixes what it is asked to do.
Quick Reference
The improvement loop
- Define good: write a sentence or two describing what a good result looks like before you change anything
- Change one thing
- Run it with the same input
- Compare the old result to the new one
- Keep it if it improved. If not, revert and try a different change.
Diagnose the problem type
- Agent misunderstood the job: make the instructions clearer
- Instructions are clear but the result is missing something: check what the agent was given (knowledge: inputs, context, tools)
- One agent is doing two jobs: change the structure and split the job
Rule of thumb
- If you have changed the instructions twice and nothing improved, stop. Look at what is coming in, or at how much the agent is being asked to do.
Common failure to avoid
- Rewriting instructions, switching the model, and adding a step all in one run. The result changes, but you cannot tell which change helped or hurt.
Notes
- It feels slower, but five focused changes beat twenty random ones.
- Defining good works the same way as defining wrong in the previous lesson: you can only check for what you can describe.
- Next up: testing in Profound, then putting the agent live.