Debugging: Lessons Learned

Here are some lessons learned from that debugging book I mentioned a few posts back.


Understand the System
- Read all related documentation
- Draw a system diagram and understand how things are connected
- Know the capabilities of your debugging tools


Make It Fail
- Start from a clean initial state
- Consider automating lengthy steps
- Make it fail in situ; don't waste time simulating the environment
- For intermittent bugs: list possible factors and try varying them one at a time; output a logfile and look for patterns


Quit Thinking and Look
- Watch it fail
- Use Remote Desktop / VNC
- Add logging and monitors
- Don't start thinking until you've limited the number of possible causes


Divide and Conquer
- Binary search
- Use test data with an easily identifiable pattern
- Start at the failure point and work backwards
- If you discover other bugs that may be related, fix them before continuing your search


Change One Thing at a Time
- Don't panic
- Back out changes that have no effect
- Compare the logfile with that of a good system
- Check earlier versions


Keep an Audit Trail
- Keep a detailed written log


Check the Plug
- D'oh!
- Have the components been properly initialized?


Get a Fresh View
- Try explaining the problem to someone (or something)
- Ask an expert: co-workers, the vendor, documentation, bug database, the web
- Report symptoms (including possibly unrelated observations), but not your theories


If You Didn't Fix It, It Ain't Fixed
- Fix the root cause
- Make the problem happen again by undoing your fix