This document is a collection of observations about the usefulness of software instrumentation. Please feel free to add observations you find inspiring. If you're quoting someone, attribute your quote, and supply a link to the original source.
"The pervasive lack of instrumentation in software products is more a result of psychological bias than real technical concerns. Software vendors can work around these psychological issues by building instrumentation as a default into tools involved in the development and deployment process. Just as Knuth said 35 years ago." —Chen Shapira at http://prodlife.wordpress.com/2009/02/04/psychology-of-instrumentation/
"I've become convinced that all compilers written from now on should be designed to provide all programmers with feedback indicating what parts of their programs are costing the most; indeed, this feedback should be supplied automatically unless it has been specifically turned off." —Donald Knuth: Structured Programming with Goto Statements. Computing Surveys 6:4 (1974), 261–301.
"I have yet in 18 years to hear a valid reason why instrumentation should not be done. I have only heard extremely compelling reasons why it must be done." —Tom Kyte at http://tkyte.blogspot.com/2005/06/instrumentation.html
"To the developers that say “this is extra code that will just make my code run slower” I respond “well fine, we will take away V$ views, there will be no SQL_TRACE, no 10046 level 12 traces, in fact–that entire events subsystem in Oracle, it is gone”. Would Oracle run faster without this stuff? Undoubtedly—not. It would run many times slower, perhaps hundreds of times slower. Why? Because you would have no clue where to look to find performance related issues. You would have nothing to go on. Without this “overhead” (air quotes intentionally used to denote sarcasm there), Oracle would not have a chance of performing as well as it does. Because you would not have a change to make it perform well. Because you would not know where even to begin." —Tom Kyte at http://tkyte.blogspot.com/2005/06/instrumentation.html
"Showing where your code spends its time is a necessary function of any production application. The fact that the extra code required to do this consumes extra system capacity is no different from any other feature in your application. As with any required feature, you simply have to size your hardware with the feature in mind." —Tom Kyte, paraphrased by Cary Millsap from observing Tom speak at several public events.