WE live in an age ruled by numbers, even though quantifying human performance compromises our performance.

At the beginning of “The Tyranny of Metrics”, author Jerry Z. Muller quotes Aaron Haspel as saying: “Those who believe that what you cannot quantify does not exist also believe that what you can quantify, does.” The author believes that in our zeal to instill the evaluation process with scientific rigor, we have gone from measuring performance to focusing on measuring itself.

Using the example of the HBO series “The Wire,” Muller illustrates how measured performances have become the hallmark of “accountability”.

For instance, police commanders are so obsessed with hitting the numbers that they would discourage anything that might compromise their “clearance rate”, the metric of the percentage of crimes solved.

The mayor’s office demands that the rate of major crimes decline by 5 percent by the end of the year, and this is a target that can be reached only by overlooking actual crimes or downgrading their seriousness. “In each case, they are engaged in ‘juking the stats’ — improving their metrics either by distorting actual results, or by diverting their time and effort from crime prevention to less productive work,” the author says.

Another sector where numbers hold despotic sway is education, where “Teaching to the Test” distorts and distracts institutions as effort is diverted from the institution’s true purpose of education to meeting the targets.

Such metrics have been so overvalued as the basis of reward or sanction that there are strong incentives to game them.

As the author observes: “Gaming the metrics occurs in every realm: in policing, in primary, secondary, and higher education; in medicine; in non-profit organizations; and, of course, in business.”

It takes a variety of forms: Gaming through creaming, improving numbers by lowering standards, improving numbers through omission or distortion of data, or downright cheating.

As a result, the costs of measuring may be greater than the benefits.

“The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge — knowledge that seems solid but is actually deceptive,” Muller observes.

We are enslaved by metrics because we live in the age of measured accountability; of reward for measured performance; and faith in the virtues of making public those metrics through “transparency” — a phenomenon the author blamed on “metrics fixation”, rather than just metrics.

In other words, metrics might have their value when used judiciously in a certain context, but can cause a backlash when abused. Muller defined the metrics fixation as “the seemingly irresistible pressure to measure performance, to publicize it and to reward it, often in the face of evidence that this just doesn’t work very well”.

Metric fixation

The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement. In an age when scientific rigor tends to be overrated, judgment is felt to be personal, subjective, and self-interested, and thus unreliable, while metrics are supposed to provide solid, objective information.

This book argues that while metrics are a potentially valuable tool, the virtues of accountability metrics have been oversold, and their costs are often underestimated.

While metrics fixation could be felt prevalently, according to Muller, nowhere have the virtues of accountability, performance metrics and transparency been more touted than in the field of medicine, where numerous studies have shown that when surgeons are rated or remunerated according to their success rates.

Some respond by refusing to operate on patients with more complex or critical conditions, as they engage in “creaming”. Muller points out that “When hospitals are penalized based on the percentage of patients who fail to survive for 30 days beyond surgery, patients are sometimes kept alive for 31 days, so that their mortality is not reflected in the hospital’s metrics”.

The author has been drawn into the subject through his experience as the chair of history department at a private university. For him all seemed to be well at the beginning: teaching, researching, and writing were demanding, but satisfying.

Then things began to change, as the author’s institution gets evaluated every decade by an accrediting body, the Middle States Commission on Higher Education.

“Soon, I found my time increasingly devoted to answering queries for more and more statistical information about the activities of the department, which diverted my time from tasks such as research, teaching, and mentoring faculty,” he said.

The author was stimulated to delve more deeply into the forces leading to this wasteful diversion of time and effort.

At first, the expansion of the evaluation of teachers and schools based on the scores of their students on standardized tests was positive.

“But in time I came to hear searing critiques of it by erstwhile supporters … and classroom teachers of my acquaintance told me that while they loved teaching, they found that increasing regimentation of the curriculum, intended to maximize performance on the tests, was sucking away their enthusiasm,” Muller writes.

A key promise of metric fixation concerns the relationship between measurement and improvement, as attested by a dictum wrongly attributed to physicist Lord Kelvin: “If you cannot measure it, you cannot improve it.”

With advocates of “accountability” typically assuming that only counting can make institutions truly responsible, performance is equated with what can be reduced to standardized measurements.

This led to lowered morals, and in time people become adept at manipulating performance indicators through a variety of methods.

When metrics become the coin of the realm, to refuse to use them is to risk failure, he observes. There is pressure from elected officials and from foundation managers to conform.

Ultimately this led to what is called Goodhart’s Law, which states: “Any measure used for control is unreliable.” They can be worse than unreliable.

“Trying to force people to conform to pre-established numerical goals tends to stifle innovation and creativity — valuable qualities in most settings. And it almost inevitably leads to a valuation of short-term goals over long-term purposes,” the book concludes.

In situations where there are no real feasible solutions to a problem, the gathering and publication of performance data serves as a form of virtue signaling.

“There is no real progress to show, but the effort demonstrated in gathering and publicizing the data satisfies a sense of moral earnestness. In lieu of real progress, the progress of measurement becomes a simulacrum of success,” the author observes cynically.