https://www.jasondavies.com/wordcloud/
Goodhart's law states, "When a metric becomes a target, it ceases to be a good metric." The moment one single parameter gains undue importance, many other equally (if not more) important aspects are ignored or side-lined. Any reasonably complex system is bound to have a dashboard of quantitative specifications and qualitative features which together ascertain proper functioning. To a lazy or overwhelmed observer, metrics provide a poor shortcut to circumvent the tedious process of judging the efficacy of a system.
Once the players know that the observer has this parochial approach, they can choose to game their proposition to optimize for the single metric under the microscope, at the cost of everything else that matters. In many cases, this single metric is not just maximized, but also bloated beyond its true value to make it look larger-than-life, to make it stand out in the crowd. Once a sufficient number of players get into this habit, everybody else is pulled in just to stay afloat. We thus end up in a mad and meaningless rat race, caught in a vicious circle of megalomaniacal portrayals of ourselves, our work, and our creations. This is what I refer to as CV-lie-zation: The act of bloating up our CV to make us look much better than we are.
The overarching phenomenon of metric fever corrupts a multitude of processes, from marketing and buying of products to screening and selection of candidates in job interviews. A lot of our current systems are about running after these metrics, typically one in each domain. Grades in school, rankings of universities, citations of researchers, impact factor of academic journals, GDP or growth rate of an economy, profits of a business, TRP ratings of a television channel, followers on social media, customer ratings on online business platforms.
All of these metrics have their place in informing assessments, but are extremely limited when each of them becomes the prima donna in their own domain. Let's resist the urge to oversimplify and embrace putting in the time and effort required to make important decisions. Ask for and evaluate much more than the metric.
Every once in a while...
- Ask a student with "poor grades" what excites them at school,
- Explore the tenth page of your Google online search results,
- Study that old paper with one citation,
- Read that unknown book that's rated one star on Amazon,
- Go to a hidden restaurant that's rated one star,
- Watch a forgotten movie that's rated one star on IMDb,
Valuable discoveries are made at places that no one is looking at. There are pearls hidden in the 'one star'.
"If you only read the books that everyone else is reading, you can only think what everyone else is thinking." ~Haruki Murakami
