Mindsets of Auditing
Friday, June 9, 2023
Click here for the recording on YouTube (Keep pressing Space to scroll through. Press ESC for an overview)GM! Whoami?
Full-time Security Researcher at Certora
Security Researcher at Spearbit
Judge and Scout on Code4rena
Secureum Alumni
yAcademy - Block 4
DISCLAIMER
Views are my own
The information presented is from my own research
Generalities applicable to all mindsets
Line by Line
- Assumed users: tincho, everyone?
- Focus on what the contract's supposed to do and compare it to what it's doing
- Focus on business logic and intention
- Tips: you can do this multiple times. First to get familiar with the code, then with different "mental modes"
The state machine (1/2)
- Assumed conscious users: zachobront, gmhacker
- Most auditors do that unconsciously
- All contracts can be reduced to something of this sort. It's the actual invisible puzzle.
- Think in terms of "Going from one state to another" when interacting with a contract
- Think in terms of "Reachability" of a bad state.
Drawing the state machine (2/2)
- Assumed users: gpersoon (uses PowerPoint), tincho (uses Miro)
- Understand all of the contracts fully, let the information sink in and sleep over it.
- Make some thorough diagrams and move parts around while letting the mind wander.
- Let the subconscious mind find the interesting bugs.
- Draw an arrow that leads to a state in which the desired behavior does not match the actual behavior.
Mental modes
- Comparison mode: take the function's name and compare the current implementation with existing/standard ones.
- Mirror mode: Takes linked/opposite variables/functions, and compare how they're coded.
- Speedrun mode : (Quoting Rajeev) Focus on Assets (the money), Actors (Power of "Roles" and normal users) and Actions (When and what can be done on what for whom)
- Blackboxing mode : Useful on complicated code bases. Understand the protocol/expected scenarios through tests, and play around with them.
- "What should happen" mode: Understand the system, its specs, its invariants, its properties. Build a mental image of the intent.
- "What shouldn't happen" mode: Find flaws in the flow. Taint the state machine. Poke at the code. Fuzz it. Write your own tests to bend it. Hallucinate scenarios of attacks. Use "fear".
Questioning, always (1/4)
Tip 1: Questions open your mind, answers close it
Tip 2: Keep asking "Why?". Become as curious as a child. That's how you go down the rabbit-hole
- "Train your mental triggers and find creative ways to poke at things" - zachobront
- "If you want to master something, teach it" - Prof. Feynmann's Technique applied to auditing:
1) Read "something"
2) Explain it. Simplify it as much as possible. Create relatable stories/scenarios/analogies
3) Re-read that "something" again. Identify gaps in knowledge. Write down questions
4) Repeat cycles of 2) and 3) until you feel ready to move on
Questioning, always (2/4)
Non-exhaustive list of creativity triggering questions and statements:
- Can you change someone else's state? (doing something for someone)
- Can you influence someone else's state? (manipulating something used somewhere by someone)
- Can a malicious contract be supplied to the system?
- Can I supply mismatched inputs? (e.g. TokenId not belonging to the User)
- Can I supply unexpected inputs? (e.g. Past timestamp)
- Can something silently pass instead of reverting?
Questioning, always (3/4)
Non-exhaustive list of creativity triggering questions and statements:
- Can someone be prevented from transferring their funds to the contract?
- Can rewards be blocked/reduced/delayed/inflated/claimed too early/claimed for someone else?
- Can funds get stolen/locked/stuck?
- Are state variables updated as they should be?
- Are state variables updated at all when they should be?
- Are state variables updated when they shouldn't be
Questioning, always (4/4)
- [Action] will revert if [Condition]
- [Action] will silently NOT work on [Action/User/Asset/State] if [Condition]
- [Action] will STILL work on [Action/User/Asset/State] even if [Condition]
- If [Action] (is|is not) done before [Action2] then [Action/User/Asset/State] is negatively impacted
- Does [Action] leave excess funds in the contract?
- Is something missing in [Action/Contract]? (e.g. a `fallback/receive`)
- Does [Action] even work correctly?
- Does [Action] even work at all?
- Can [Action] be done several times?
- What about [Edge-Case-Input]? (min/max/0 etc.)
- What about [Corner-Case-Flow]? (claiming/closing without staking/opening etc.)
Poking around (1/2)
Tip 1: It's ok to edit the code, really.
Tip 2: It's ok to add console.logs
Tip 3: Use a local git branch to save your work or a private repo to collaborate
Tip 4: Make use of the 3 Ps: People (Teammates), Process (Mindsets for creativity), Product (tools)
- Use fuzzing and invariants testing.
- Coverage is your friend. Use coverage gutters in combination with `forge coverage --report lcov`
- Add 1 positive assert and 1 negative assert every few lines (this should happen/this shouldn't happen)
- Test edge cases. Shuffle the flow of operations.
- Use Foundry's Snapshot cheatcode to compare outputs between operations
- Use Slither to determine sensitive areas
Poking around (2/2)
Add a
notes.mdfile: keep track of what you're doing + make an "on the fly checklist" with ideas and questionsWrite detailed POCs. "If you can't write a POC, the finding's probably not valid" - Alex the Entreprenerd
To level up:
Horsefacts - Invariant Testing WETH With Foundry: https://mirror.xyz/horsefacts.eth/Jex2YVaO65dda6zEyfM_-DXlXhOWCAoSpOx5PLocYgw
Certora - Formal Verification for Fun and Profit Lectures (1-11): https://www.youtube.com/watch?v=sdEfc-58CUE&list=PLKtu7wuOMP9XHbjAevkw2nL29YMubqEFj
Specific mindsets
BFS (Breadth First Search) (1/2)
- Assumed users: devtooligan, Dravee
- Making assumptions on internal functions instead of diving deep first
- Iterative approach. Progressively rising in complexity, helps with building a list of questions
- I can talk more about this from experience
BFS (Breadth First Search) (2/2)
- Understanding everything deeply in 1 go is unlikely
- Solution: understanding the code incrementally. From most simplified pseudo code to smallest details.
- Start with the entrypoints of the system. Write down the important operations as pseudo code (external calls, state updates, heavy calculations...). Add your ideas and questions to your `notes.md` file.
- Then do the same with the internal functions you've been building assumptions and curiosity on. Your goal is to be able to explain the most important things happening in each function of the system.
- Then focus on suspicious/complex lines. This is usually when you start poking at the code.
- Now that you're fairly familiar with the system: try to come up with creative ideas on how to break the intended flow.
Tip reminder: YOU CAN change the code and add `console.log`s to deepen your understanding. You CAN add require statements and see if existing tests get broken.
DFS (Depth First Search)
- Assumed users: sjkelleyjr
- Deep into dependencies first (libraries, internal functions, or even documentation)
- Like building understanding starting from the bottom bricks
Sink and Tainted Path to Sources
- Assumed users: samczsun, zachobront, Mudit Gupta
- Find a Sink (important operation that shouldn't be tainted)
- Walk backward. Is there any path where the data can be tainted by users.
- Use fear on what state shouldn't be reached. Work in reverse and prove that it can actually be reached.
Reimplementing things your way
- Assumed users: cmichel
- How would YOU implement "such" function?
- Compare your idea of how YOU would implement the function with how it's actually implemented?
- GOAT Method. You need an amazing amount of experience and intuition
Conclusion
OF COURSE you're free to use everything altogether and even more, if you have time.
It takes time. It takes luck. It takes experience.
See what works for YOU, or most importantly:
How do YOU enjoy auditing?