Tag Archives: statistics

England’s World Cup Failure: A nerd explains why all the pundits are wrong

worldcupfootballEvery time I hear another pundit explaining their theory behind England’s failure at the world cup I get all hot and bothered. My wife could literally not care less about football or my feelings on the subject, so I thought I would share my them with you 😉

You see, it seems that football does not benefit, as cricket and baseball do, from that important type of pundit, the statistician.

For if they did, they would realise that failing to make it into the last 16 this year is not a failure at all.

Why? Because we have to remember that there are, at time of writing, 209 national men’s teams registered with FIFA – and that FIFA estimates that 250 million people play the beautiful game. So just getting to the finals is a real achievement.

On the other hand you can argue that England should be in the top flight – it has a decent population, it has money to spend and many aspirational heroes.

Well.. it does do well, currently, England is ranked 10th in the world, and has often been higher. But does that mean it should always reach the last 16 of the world cup? No.

Take a step back. Even with so many teams, that world ranking should mean a team like England should make it ‘usually’, but certainly not always. Indeed, it has made the last 16 every time since 1958, indeed it’s a surprise to me that they have run so long without missing out.

It has always been known that football has a ‘luck factor’ indeed this is one of its best features – upsets happen – and that is why the league has a round-robin design, and also why some tournaments are done by knockout – the league aims to find the best teams, tournaments aim to find the best moments.

England did not play badly this world cup. The goals conceded were really pretty darn good, and England had more shots on goal than their opponents. But for the rub of the green, they could have been through and lauded by all. So how can the pundits have such strong opinions?

Easy. Because it’s their job to sound like they know.

WorldCupRankingsPlot

Addendum
So how have teams like Brazil, Spain, Germany, Italy, France and the Netherlands managed to spend so much time in the top 3? If it was all luck, they would not. Well, this means that there are actually recipes for better performance, elusive but real…

However, just as the government would hate for you realise they do not control the economy, football pundits and administrators alike would not like you to know (or indeed to know themselves) that this recipe is largely outside of their control – the biggest factors being: population size, other games to play, weather, virtuous circles (inspiration, promise of fame, etc) and last but certainly not least, luck.

Vaccination ‘Hesitance’ Put Bluntly

Question: Do you know anyone who’s had smallpox? How about polio? No?Polio Vaccine

Well no wonder there’s a sudden surge of doubt about vaccination – not because we’ve realise how well it works, but because we’ve forgotten what life was like before vaccines.

I am a parent. I can understand the idea of injecting my child with something that turns out to be harmful would be hard to bear. First do no harm, they say. Can we be sure there is no harm?

Sounds a like a fair question… but… isn’t. It’s not that simple.

Having a vaccine may seem like a dangerous intervention, and surely no action is better than a risky one?

Well if you believe that, you are not alone, but are making a cognitive error, the error to assume inaction is a virtue. As Haile Selassie said “Throughout history, it has been the inaction of those who could have acted; the indifference of those who should have known better; the silence of the voice of justice when it mattered most; that has made it possible for evil to triumph.”

In order to help those stuck in this cognitive trap, I would ask them to consider this thought experiment.

BluePillRedPill

Rather than choosing to go to a doctor for a vaccine, say the famous ‘MMR’ triple-jab, or choosing to stay at home, let’s imagine the choice was one of two medications, one the MMR, with its supposed* risk of autism, but which gives immunity to measles, mumps and rubella – and the other is a compound with no known benefits, but a known risk of causing measles, mumps and rubella. And of course the added possibility of causing an epidemic at the same time.

(*) Let’s add to the mix the fact that there is zero evidence of any link between MMR and autism, and also point out that failing to immunise your kids is reckless cruelty not only to them, but also to all those who cannot be vaccinated for ‘real’ reasons, such as being too poor, too young or too sick.

To those still on the fence, this last bit is for you.

Do you not realise that what seems like concerned parenting has actually cost real people their lives? For what?

Yes, drugs go wrong. Yes, corporations are often tempted to hide bad news. Yes, the human body is complex and we do not understand every last detail.

But if you think that those pressures overwhelm the ‘good’ in mankind that has led life expectancy to climb gloriously every single decade this last century, then you are missing all the good news.

smallpoxNo small part of the doubling in life expectancy was due to vaccines.

Please don’t take me with you back to 1900. Rather look at this picture of what smallpox does and thank your lucky stars she’s not your child.

Global Warming Canaries, Anyone? A short intro to why most scientists are worried about climate change…

The current state of arctic sea ice (see graph below) sends a chill down my spine.

Image

So what it says is that the ice is melting furiously, and looks like it’s not yet slowing down even though the days have started to draw in.

However, any scientist will tell you that no single data point can be used as evidence of global warming, there are simply too many fluctuations for anything to be concluded over anything but the longest timescales. We cannot simply look at the mean temperature for a hot year and say, there you go, global warming!

Now, the issue is, there are well-known cycles over pretty much all timescales – this pretty much undermines all serious attempts at prediction.

So, what to do? Well all is not lost; there are still some clever little leading indicators we can look at to give us that sobering wake up call.

#1:  CO2

Firstly, we know CO2 concentration is up, no doubt or argument, this can be seen in the famous Hawaii data above, complete with the seasonal ‘breathing’ by global plant-life. The argument is about whether the greenhouse models that say this will result in warming will turn out right. I honestly don’t know, but I wouldn’t even have to wonder if the CO2 levels weren’t going up, would I?

#2: A Record Breaking Rate of Record Breaking

Secondly, rather looking at averages or ‘new records’, we can look at the frequency of records. So rather than saying, “we just had the hottest summer ever in some parts of the US, there’s the proof” we can look at how often records are set all over the world – hottest, coldest, wettest, dryest and so on. This approach creates a filter; if it shows there are more records being broken on the hot side than the cold side, could this be an indicator? I hope not, because there are.

Again, it could be part of a long-term cycle that could bottom out any time now. But on the other hand, if it was going the other way, I wouldn’t have to hope, would I?

#3: Sea Ice

Now the sea ice. The sea ice is another proxy for temperature. The reason it’s interesting to climatologists is because it is a natural way to ‘sum-up’ the total warmth for the year and longer; if ice is reducing over several years, it means that there has been a net surplus of warmth.

CryoSat – The European Space Agency’s Sea Ice Monitoring satellite launched in 2010 – (Image credit ESA)

Today we are seeing a new record set for minimal northern sea ice. And not only is there less area of ice, but it is thinner than previously realized and some models now suggest we could be ice-free in late summer in my lifetime.

Now if that does not strike you cold, then I didn’t make myself clear. This is not some political posturing, not some ‘big-business’ spin, nor greeny fear mongering. It’s a cold clean fact you can interpret for yourself, and it could not be clearer.

So is it time to panic?

Well it can still be argued the melting is part of a cycle, it could of course reverse and hey, no biggy. After all, what does it matter how much ice there is?

Well, yet again, I hate to rely on the ‘hope’ that it’s a cycle. Because if it continues, the next effect will be felt much closer to home…

Sea Level

Sea level is the ultimate proxy for warming. Indeed, sea level change can be so serious, maybe it is the problem rather than the symptom. If the ice on Greenland and Antarctica melt, the rise in sea level would displace hundreds of millions of people and change the landscape so dramatically it’s a fair bet wars and famine will follow. Now that is serious.

So have we seen sea level rise? Well, yes. Here’s the plot:

Now, it looks pretty conclusive but hold the boat. Some say’s it’s proof of warming but not everyone agrees. It’s true it could again be a cycle. Also, the sea level rise is fairly gradual; what people are really arguing about is whether we should expect it to speed up. If temperature goes up a few degrees it could go up 5 or 10 times faster. The speed is the issue. Humanity can cope if the level goes up slowly enough, sure, countries like Tuvalu will be in big trouble either way, but countries like Bangladesh and cities like New York and London will only be in real trouble if the rate increases.

Actual Canaries

Canaries taken into mines in order to detect poisonous gases; the idea being they would suffer the gas faster than the people and if the canary dropped, it was time to vacate. Do we have systems that are hypersensitive to climate change?

Yes! There are many delicately balanced ecosystems that can can pushed over a tipping point with the lightest of touch. Is there an increase in the rate of species loss, or an increase in desertification? Yes!

We can also look at how far north certain plants can survive, how high up mountains trees can live or how early the first buds of spring arrive.

Again, these indicators fail to give solace. Everywhere we look we see changes, bleached coral, absent butterflies, retreating glaciers.

The conservative approach is to ascribe these changes to the usual cut and thrust of life on earth; some take solace from the fact that humankind has survived because we are the supreme adapters and that the loss of species is exactly how the stronger ones are selected.

Yes, we are great at adapting, however, to kill any complacency that may create, consider the following: for humans just ‘surviving’ is not the goal, that’s easy, we also need to minimize suffering and death, a much tougher aim. We’ve also just recently reduced our adaptability significantly by creating ‘countries’. Countries may seem innocuous, but they come with borders – and mean we can no longer migrate with the climate. Trade across border also needs to be of roughly the same value in both directions.  While some countries will actually see productivity benefits from global warming, most will not, and without the freedom to move, famine will result. Trade imbalances mean inequality will become extreme. The poorest will suffer the most.

So for now changes are happening, and advances in agricultural technology are easily coping; however, because ecosystems are often a fine balance between strong opposing forces, changes may be fast should one of the ropes snap.

Conclusion

Looking at the long history of the earth we have seen much hotter and much colder scenes. We have seen much higher and much lower sea levels. We are being wishful to assume we will stay as we have for the last 10,000 years. It may last, or it may change. Natural cycles could ruin us. And mankind is probably fraying the ropes by messing with CO2 levels.

Can we predict if we are about to fall off of our stable plateau? No, probably not. But is it possible? Heck yeah.

=============

If you liked this, you may like these earlier posts on the subject of global warming:

  1. What does the earth’s history tell us about climate? And how can we find out if our house will be one that sinks should sea levels rise? Find out here!
  2. Can we  change the planet’s dangerous behavior? Read my call for a study in mass behavior.

Big Pharma: Heroes or Villains?

What is ‘Big Pharma’?

Many people have argued that alternative and complimentary medicines are suppressed because they threaten the status quo for ‘big pharma’.  Before we accept this claim, let’s unpack the idea of big pharma a little to understand the incentives at play and when it may be right not to trust big pharma.

Let me start by making it clear – big pharma, as a label for the largest pharmaceutical companies, deserves a healthy dose of outrage; but before we toss the baby out with the bathwater, but lets see when – and why.

Big pharma is just another name for ‘big business’: a big business is an organism that has grown beyond the people that founded it, such that rather than having emotions, conscience or guilt, it has KPI’s like turnover, cashflow and return on investment. Big pharma is in the business of making money and as such should generally be expected to default to that option unless constrained by law. The collective conscience of shareholders only tends to kick in when dirty laundry is put out on show. Ok, so firstly, I think we can agree, a business is not a charity.

Next we take this lack of compassion and combine it with size and complexity – we see we now have an organism susceptible to plain outright crookery – from the white collar sort, like insider trading – to the very tangible – such as the dumping of toxic waste. These practices usually require the corruption of people – but not always – it is very easy for companies to do bad things without any individuals having malicious intent; it could simply be negligence or incompetence, or it could simply be that profit sometimes comes before fairness.

Take for example the problem of selling DVD’s in the world market. They are small and lightweight and easy to ship worldwide. This usually means that the price would be similar worldwide, if dealers in one country were to raise prices, residents would simply import the product. However, the enormous wealth differentials that exist between, say, the USA and Mexico, mean that the company could set a high price to extract maximum value from the US market, but then essentially price themselves out of the Mexican market. If they lowered the price, they would sell more product but with much reduced profit margins.

This problem is thrown into stark relief in the case of drugs, where the most profitable option is often to cater to the richer countries. This is sound business – set your price high, keep your factory trim, reduce shipping costs, keep high margins. However, if the drug can radically improve health outcomes, this policy could be seen as unethical.

This is the sort of problem big pharma face routinely; they are not selling entertainment, they are sometimes selling life itself, and often find they need to play profits against ethics in they way I describe above. It is thus hardly surprising that the general public have a distrust and general suspicion toward Big Pharma. In addition to drug import controls, there are many other situations where governments have had to step in to ensure the pharmaceutical companies ‘do the right thing’, such as the case with antiretroviral drugs (for HIV) coming into Africa.

Now think for a moment on this thought experiment: what would happen if a small publicly traded company discovered a cheap and easily reproducible cure for cancer? Would they really be able to hold on and extract full value for their shareholders? History actually suggests they wouldn’t – the drug would become public property, or would simply be nationalised if the company tried to resist. Inventions like the major vaccines and the first antibiotics were often not patented, and we see if we look at the pharmaceutical industry that their biggest profits come predictably not from miracle cures but from drugs that cater to the maladies of the richer classes. The top targets are heart disease, heart-burn, stroke, mental health and asthma. Once you  add disorders like diabetes you have accounted for the most profitable chunk of the industry.

This trend raises fresh concerns, because there are many severe ailments that are simply not attractive to profit making operations, the poster-boys being malaria, TB and HIV/Aids. Drug companies can be bullied into doing work in these areas, but it tends to fall to governments and charities to fund research in the afflictions of the poor, or on the so-called ‘orphan diseases’ – ailments that affect too few people to ever make a profitable market.

Economists will also argue that profit making businesses, being creatures under the strict control of incentives, will be unlikely to aim for ‘cures’ because cures are ‘one-offs’. While this criticism has some sad validity (in the board-room if not in the clinic), we have to remember that the big drug companies only exist because they make profits; in an imagined world where the first dollars were always spent on the most dire diseases and we only get to do botox and erectile dysfunction once those are all solved we would have no private industry at all, so far fewer trained scientists, far less public knowledge and certainly no map of the genome. We have to remember that to some extent at least, the aging american taking their cocktail of pills every day for the last 50 years has in some sense subsidized the field doctor in rural Africa. Yes, they also subsidized Wall Street excesses, but perhaps it’s a deal worth making.

Publication Bias

Another area where drug companies increasingly in need government intervention is in drug trials; specifically, they are presently allowed to pick and choose what to publish; this sounds OK at first, because, surely, you assume, the drug company has to make a bulletproof case before the drug is licensed? Well, if you do 100 trials, you may well find 50 good results, and publish those, and simply sweep the duds under the carpet. What’s more is those duds could have revealed possible side effects or interactions that could actually turn out to be real issues later on down the road. This is going to be a big one in the next few years.

The Big Picture

When criticizing the pharmaceutical industry it is easy to get caught up in the weeds, for there are weeds, but let’s also try to remember that this century has seen unparalleled improvements in life expectancy world-wide, and the improvements in child mortality in the third world do owe a lot to the sometimes cold-hearted business models intrinsic to western medicine.

Before I move on, and being a scientist, I wanted to make another point about big pharma. While it’s true that big money is involved, we have to remember that the Pareto principle applies here too – the majority of the profits come from the minority of the research. There are legions of perfectly good people, motivated by no more that the desire to help people in distress working in healthcare all around the world. Drugs are highly integrated with other therapies at the clinical level and the people actually running trials ‘in the trenches’ face-to-face with the patients (and often dealing with terrible trauma)  are rarely shareholders in big pharma, and many would not even think for a second they are part of what people would call big pharma. Yet it is they who have gradually built up our current understanding of the human body, not the men in suits.

Conclusion

To me, the idea of executives at the top 5 drug companies has become conflated with the idea of the ‘canon’ of western medicine. The idea that the whole world of ‘proven medications’, the result of countless years of hard graft (and the learnings from millions of deaths), can be dismissed because it’s under the control of ‘fatcats’ is a sick tragedy. Western medicine is simply a name for ‘what has been statistically been proven to help’, and the idea that even a tiny fraction of the scientists who developed it would be working to suppress good ideas from outside the ‘fold’ sounds frankly paranoid. Yes big pharma has some warped incentives that cause it to focus on the wrong things and leave the poor out in the cold, but all for-profit publicly-traded businesses do that! Ask yourself for a minute – even if a cure for heart disease were found that threatened the profits to Pfizer and friends, could they really recruit a worldwide network of conspirators who think a cure for cancer is something worth selling their very souls for to suppress?

It hard enough to run a real business, let alone running one so effectively in complete secrecy in the face of so much scrutiny. If they have that much skill and power, they should go legit, they would make a real killing!

Death by KPI. The unintelligent design of the modern company… and what to do about it.

KPI!

A beginner’s guide to KPIs…

Background, Context and all that…

Some of you luckier readers will be wondering what on earth a KPI is. Alas, many of my readers will know, and rarely will they have a happy tale to tell about them. Let me tell you mine.

You see, to me the story of the KPI is none other than the story of the modern trend to remove the human element (that most fallible of elements) from big business. I propose that there has crept up upon us, starting when we came down from the trees and now coming to its final fruition with the industrial revolution, a situation in which the workings of our society, our organisations, governments, armies or companies, are simply too complicated to be designed or managed by any one person, whatever force his or her personality might possess.

No, the time of the one big man, the head honcho, the brain, scheming away in his tower, is over – and we enter the era, nay the epoch, of the human hive.

For now we see that in order to achieve great things, it is the ability to sort and organise mankind, rather than the ability of each man on his own, that matters most.

I could wander from my thesis awhile to describe a few side effects, for example, to point out the age of ‘middle management’ is here to stay, or to philosophize about the world where no one is actually ‘driving’ and the species is wandering like a planchette on a Ouija board and how this explains why no one seems to be able to steer us away from the global warming cliff just ahead…

But no, I will not be pulled off course, I will return to our good friend, the KPI.

So it seems we have these complex organisms, such as the venerable institution that is the company, that have evolved to survive in the ecosystem that is our global economy. Money is the blood, and people, I fancy, are the cells. And just as no particular brain cell commands us, no particular person commands a company. Just as the body divides labour among cells, so does the company among its staff. We train young stem cells into muscle, tendon and nerve. We set great troops of workers to construct fabulous machines to carry our loads just as our own cells crystallize calcium to make our bones.

Having set the scene thus I must move on, for where are the KPIs in all this?

As clever as the cells of our bodies may be, to choose depending on the whims of circumstance to turn to skin or liver or fat, that is but nothing compared with the cleverness with which the cells are orchestrated to make a cohesive body, with purpose and aim,  with hopes and dreams, and of course sometimes even the means to achieve them. And the question is: how is that orchestration achieved?

Being a devout evolutionist, its is clear to me that it was by no design, but rather by the constant failure of all other permutations that led to the fabulously clever arrangement, and so it is with the organism that is the company.

It is this conviction that leads me to claim, contrary to the preaching of many business schools, that good companies are not designed, but evolve, and by a process of largely unconscious selection. Like bacteria on a petri dish, companies live or die on their choices, and by every succeeding generation, the intelligence of these choices is embroidered into the DNA of the company.

Yes, I am saying that the CEO of Microsoft, or Rio Tinto of Pfizer, is no more sure of his company’s recipe for success than any one cell in your brain is at understanding how it came to be that you can read these words. Ok, maybe that is unfair. They probably have a good shot at reproducing success in other companies, but they only understand how the company works, not why.

It is well to remember that very few of the innovations present in a modern successful company were developed within that company – the system of raising money from banks or through a system of stocks and shares, the idea of limiting liability to make these investments more palatable, of development the of modern contract of employment to furnish staff; nay the very idea that a group of people can actually get together and create the legal entity that is the ‘company’ has been developed over centuries.

Even within the typical office we see many innovations essential to run a business that could never have been ‘designed’ better by single mind – the systems to divide labour into departments – finance, marketing, sales, R&D, logistics, customer service; the reporting hierarchies and methods for making decisions; the new employee checklists, the succession plans, the new product stage-gate system, the call-report database, the annual budget, the balance sheet, the P&L – these are all evolved and refined tools that incorporate generations of brain power.

Whether it is the idea of share options or the idea of carbon copies, the list of machinery is endless and forms the unwritten DNA of the modern company. The company of today has little in common with the farmstead of 300 years ago, and indeed, just like the bullet train, it would not work too well if taken back in time 300 years. It only works in its present setting. It is part of a system – an ecosystem.

Now a business tool that has been evolving for some time, slowly morphing to its full and terrible perfection is the Key Performance Indicator or KPI.

The Key Performance Indicator

There is a saying in engineering circles: you cannot improve what you do not measure.

This philosophy accidentally leaked to the business community, probably at Harvard, which seems a great place to monetize wisdom, and so there is presently a fever of ‘measurement’ keeping middle managers in their jobs, and consultants on their yachts.

This sounds reasonable – but let’s pick at it a little.

When the engineer installs a sensor in a reactor to measure its pressure, it is usually just one element in a holistic system of feedback loops that use the measurement in real-time to control inputs to that reactor. Thus we can see that measurements in themselves are not enough, there needs to be an action that is taken that affects the measured property – a feedback loop.

Likewise any action taken in a complex system will tend to have multiple effects and while you may lower the pressure in a reactor, you may raise it somewhere else, thus the consequences of the action need to be understood.

And lastly, if the pressure in the reactor is now right, the value of the knowledge has diminished, and further action will be of no further benefit.

So just like a body, or a machine, a company needs to be measured in order to be controlled and improved, and it occurs to me that these key measurements, the KPIs, need to fulfil a similar set of requirements if they are to be of value.

A few years ago I decided to write a list of KPI must-haves, which I present here:

  1. the property measured must be (or correlate with) a company aim (eg profitability)
  2. measurements need to taken to where they can be interpreted and acted upon, using actions with predictable effects, creating a closed loop
  3. the secondary effects of that action need to be considered
  4. repeat only if the benefits repeat too

Now let’s take a look at some popular KPIs and see if they conform to these requirements.

Production KPIs

There are a multitude of KPIs used the ‘shop floor’ of any enterprise, be it a toothpick factory, a bus company, a florist or a newspaper press. Some will relate to machines – up-time metrics, % on time, energy usage, product yield, shelf life, stock turns – indeed far too many to cover here, so I will pick one of my favourites.

Availability” is a percentage measure of the fraction of time your machine (or factory or employee) is able to ‘produce’ or function. If you have a goose that lays golden eggs, then it is clear that ‘egg laying activity’ will correlate with profits, so point #1 above is satisfied, and measuring the egg laying frequency is potentially worth doing. If you then discover that your goose does not lay eggs during football matches you may consider methods to treat this, such as cancelling the cable subscription. A few trials can be performed to determine if the interference is effective, proving #2. Now all you need to do is worry about point #3; will the goose fly away in disgust at the new policy?

What about #4? Indeed you may get no further benefit if you continue to painstakingly record every laying event ad infinitum. KPIs cost time and money – they need to keep paying their way and may often be best as one-off measures; however, what if your goose starts to use the x-box?

So that is an example of when the KPI ‘availability’ may be worth monitoring, as observing trends may highlight causes for problems and allow future intervention. So surely availability of equipment is a must-have for every business!?

No. There are many times when plant availability is a pointless measure. Consider for example an oversized machine that can produce a year’s supply in 8 minutes flat. So long as it is available for 8 minutes each year, it does not matter much if it is available 360 days or 365 days. Likewise, if you cannot supply the machine with raw materials, or cannot sell all the product it makes, it will have forced idle time and availability is suddenly unimportant.

The simplest way to narrow down which machines will benefit from an availability or capacity KPI is to ask: is the machine is a bottleneck?.

For other types of issues, let’s look at some other KPIs.

Quality KPIs

Quality KPIs are interesting. Clearly, it is preferable to ship good products and have happy customers. Or is it?

In any production process, or indeed in any service industry, mistakes will be made. Food will spoil, packaging will tear, bits will be left out. It is now fashionable to practice a slew of systems designed to minimise these effects: to detect errors when they are made, to re-check products before they are shipped, to collect and collate customer complaints and to feed all this info back into an ever tightening feedback loop called ‘continuous improvement‘.

KPIs are core to this process and indeed KPIs were being used in these systems long before the acronym KPI became de rigueur. To the quality community, a KPI is simply a statistic which requires optimization. The word ‘Key’ in KPI not only suggests it represents a ‘distillation’ of other numerous and complex statistics, but implies that the optimising of this particular number would ‘unlock’ the door to a complex improvement.

Thus, a complex system is reduced to a few numbers, and if we can improve those numbers, then all will be well. This allows one to sleep at night without suffering  nightmares inspired by the complexity of one’s job.

The reject rate is a common quality KPI – it may encompass many reasons for rejection, but is a simple number or percentage. It is clearly good to minimize rejects (requirement #1) and observing when reject rates rise may help direct investigations into the cause thereof satisfying requirement #2.

However, from rule #2 we see that this KPI is only worth measuring if there will be follow-up: analysis and corrective action. This must not be taken for granted. I have visited many plants that monitor reject and when asked why, they report that head office wants to know. What a shame. Perhaps head office will react by closing that factory some time soon.

The failure to use KPIs for what they are intended is perhaps their most common failure.

Another quality KPI is the complaint rate. Again, we make the assumption that complaints are bad, and so if we wish to reduce these we should monitor them.

Hold the boat. How does the complaint rate fit in with company aims? We already know that mistakes happen, but eliminating quality issues is a game of diminishing returns, so rather than doggedly aiming for ‘zero defect’ we need to determine what complaint rate really is acceptable.

So here is another common KPI trap. Some KPIs are impossible to perfect, and it is a mistake to set the target at perfection. Think of your local train service. Is it really possible for a train system to run on time, all the time? The answer is an emphatic no!

The number of uncontrolled inputs into a public transit system – the weather, the passengers, strike actions, power outages and the like will all cause delays, and while train systems can allow more buffer time between scheduled stops to cater to such issues, this type of action actually dilutes other aspects of service quality (journey frequency and duration). Add to that finally the fact that a train cannot run early so losses cannot be recovered.

The transport company will of course work to prevent delays before they occur, and lay on contingency plans (spare trains) to reduce impacts, but the costs and practicality mean that any real and meaningful approach needs to accept a certain amount of delays will be inevitable. A train company could spend their entire annual profit into punctuality and they would still fall short of perfection.

So it is with most quality issues, the law of diminishing returns is the law of the land. Thus the real challenge is to determine at what point quality and service issues actually start to have an impact on sales and cashflow. This is another common pitfall of the KPI…

The correlation between the KPI and profitability is rarely a simple positive one, especially at the limits.

Some companies get no complaints. Is this good? No, often it is not! This company may be spending too much money on QC. The solution here is to work with and understand the customer – what issues would they tolerate and how often? If you did lose some customers by cutting quality, would the financial impact be greater than the savings?

However, and on the other hand, don’t make the opposite mistake: once a reputation for poor quality is earned, it is nearly impossible to shake.

Financial KPIs

As companies become bigger, there is a tendency to divide tasks according to specialized skill and training. Thus it can happen that the management of a big mining company may never set foot in a mine, may not know what their minerals look like, nor may they know how to actually dig them up or how to make them into anything useful. In other words, they would be useless team members after a nuclear apocalypse.

However, this is no different from our brain which is little use at growing hair, digesting fat or kicking footballs.

It is thus necessary that the organism (the company) develops a system to map in a flow of information from its body to its mind and then another mapping to take the decisions made in that mind and distribute them to the required points of action.

Indeed to blast in a quarry, or to kick the football are indeed best done by organs trained and capable and no less important than the remote commanders in the sequence of events.

And just as our bodies have nerves to transmit information about the position of our lips and the temperature of the tea to our mind, so the company has memos, telephones and meetings. And KPIs are the nouns and subjects in the language used.

Furthermore, some KPIs need to be further distilled and translated from the language of the engineer (Cpk), the quality manager (reject rate) or the plant manager (units shipped) to that of the accountant (revenue) , the controller (gross margin) and eventually that of the general manager and the shareholder (ROI). This is perhaps the main duty of middle management, bless their cotton socks.

Unfortunately, the mapping of everyday activities to financial KPIs is fraught with danger. The biggest concern comes from the multiple translation issue. That is to say, KPIs can suffer from a case of Chinese whispers, losing their true meaning along the way, resulting in the worse result: a perverse incentive.

Yes, ladies and gentlemen, this does happen.

Let’s say you want to improve your cash situation. You may choose to change the terms in your sales contracts for faster payment, in essence reducing the credit you allow your customers. This may have the desired effect, lowering the KPI  called “receivables” and this looks good on the balance sheet – but let’s look at requirement #3 in the KPI “must have” list. What are the ripple effects of this move? It is clear this will not suit some of your customers, who, considering recent economic trends, probably also want to improve their cash situation; thus you may lose customers to a competitor willing to offer better terms.

And so we see the clear reason for perverse incentives is the consideration of KPIs individually instead of collectively. There has to be a hierarchy upon which to play KPIs against one another. Is revenue more important than margin? Is on-time shipping a part-load preferable to shipping “in full” a little late?

So we see again that the systems used to distill company indicators the choice of which decisions are centralized and which are localized need to be developed and constantly refined using an iterative process. The art of translating the will of shareholders into a charter or mission statement and then translating that into targets for sales, service and sustainability is a task far too complex to perfect at first attempt.

KPI Epic Fails

While on the subject of KPIs I cannot resist the opportunity to bring to mind a few fun examples of KPIs gone badly wrong.

The Great Hanoi Rat Massacre

The French administration in Hanoi (Vietnam) were very troubled by the rat population in Hanoi around the start of the last century, and knowing as they did about rat’s implication in the transmission of the plague, set about to control the population. A simple KPI was set – “number killed” and payments were made to the killers on this basis. There was immediate success with rats being brought in by the thousand and then the tens of thousand per day. The administration was pleased though somewhat surprised by the sheer number. There surprise gradually transformed into disbelief as time wore on and the numbers failed to recede.

You guessed it. The innovative residents of Hanoi had started to breed rats.

The Magic Disappearing Waiting List

Here’s a more recent example from the National Health Service in the UK (the NHS).

A health service is not there to make a profit, it is there to help the population, to repair limbs, to ease suffering, to improve the length and quality of life – and to do this as best it can on a finite budget. So the decisions on where to invest are made with painstaking care – and needless to say, KPIs are involved. Not only big picture KPIs like life-expectancy, or cancer 5-year survival rates, but also on service aspects, such as operation or consultation waiting times.

It will therefore not be surprising to you to learn that the NHS middle management started to measure waiting times and develop incentives to bring these down, or even eliminate them. This sounds very reasonable, does it not?

Now ask yourself, how do you measure a waiting time? Say a surgery offers 30 minute slots – you may drop in and wait for a vacant slot, but as the wait may exceed a few hours it is just as well to book a slot some time in the future and come back then. So one way to measure waiting times is to measure the mean time between the call and the appointment. This of course neglects to capture the fact that some patients do no actually mind the wait and indeed may choose an appointment in two weeks time for their own convenience rather than due to a lack of available slots. Lets put that fatal weakness aside for the minute as I have not yet got to the amusing part.

After measurements had been made for some time, and much media attention had been paid to waiting times, the thumb-screws were turned and surgeries were being incentivized to cut down the times, with the assumption they would work longer hours, or perhaps create clever ‘drop-in’ hours each morning or similar.

Pretty soon however, the results started coming in, the waiting times as some surgeries were plummeting! Terrific news! How were they doing it?

Simple: they simply refused to take future appointments. They had told their patients: call each morning, and the first callers will get the slots for that day. This new system meant nobody officially waited more than a day. Brilliant! Of course it is doubtful the patients all felt that way.

How The Crime Went Up When It Went Down

If you work for the police, you will be painfully aware that measuring crime is difficult. And so it is with the measurement of many ‘bad’ things – for example medical misdiagnoses or industrial safety incidents.

Let’s look at workplace safety; while it may be fairly easy to count how many of your staff have been seriously injured at work, it is much harder to record faithfully the less serious safety incidents – or more specifically, the ones that might have been serious, but for reasons of sheer luck, were not. The so-called ‘near-misses’.

Now to the problem. Let us say you are a fork-lift driver in a warehouse and one day, it a moment of inattention, you knock over a tower of heavy crates. Luckily, no-one was around and more luckily, no damage was done. So what do you do? Do you immediately go to the bad-tempered foreman with whom you do not get along and tell him you nearly killed someone and worse, nearly caused him a lot of extra work? Or do you carefully stack the crates again and go home for dinner?

The police suffer a similar predicament. The reporting of a crime is often the last thing someone wants to do, especially if they are the criminal. Now let’s say you are an enterprising young administrator just starting out in the honourable role as a crime analyst at the Met. You want to tackle crime statistics in order to ensure the most efficient allocation of funds to the challenges most deserving thereof. Do bobbies on the beat pay for themselves with proportionately reduced crime? Does the ‘no broken windows‘ policy really work? Does the fear of capital punishment really burn hot in the mind of someone bent on murderous revenge? Such are the important questions you would wish to answer and you have a budget to tackle it. What do you do?

You set out to gather statistics of course, and then to develop those tricky little things, the KPIs.

Now let’s say a few years pass, and after some success, you are promoted a few times and your budget is increased. Yay! You have always wished for more money to get more accurate data! Another few months later and the news editors are aghast with the force. Crime is up! Blame the police – no, blame to left – no blame the right! Blame the media! It’s video games – no, it’s the school system!

No, actually it’s a change in the baseline.  The number of crimes recorded most likely went up because the effort in recording them went up. The crime rate itself may well have gone down.

The opposite can also happen. Say you run a coal mine and you will be given a huge bonus if you can get through the year with a certain level of near misses. Will you really pressure your team to report every little thing? I think not.

So the lesson here is: watch out for a KPI where you want the number to go one way to achieve your longer term aims, but where the number will also depend on measurement effort.

The Profit Myth

Most KPIs are dead dull. The very mention of KPIs will elicit groans and be followed swiftly by a short nap. The ‘volume of sales’ KPI is no different. The issue with the volume KPI is probably made worse by the clear fact that actually thinking about KPIs is a strong sedative. Surely selling more is good? Well, if you can fight through the fog of apathy, and actually think about this for a second, it is easy to see this is often not so.

To see why, it is important to understand price elasticity. It if often true that lowering price will increase sales, so an easy way to achieve a target ‘volume’ (number) of sales is to drop price. That way to can make your volume KPI look good, as well as your revenue KPI, and so long as there is still a positive margin on all the units sold your earnings KPI (aka profit!) will see upside. There can’t possibly be any downside, can there?

Of course, astute financial types can find this fault easily, and perhaps the question of ‘how’ would make a good one for a job interview. It turns out more profit is not always good (seriously!). Whether more profit is really worth taking depends on the ratio of the increased earnings to the increased investment that is needed to make them.

There is another KPI I mentioned earlier the “ROI” or return on investment. When I discussed it I implied that it was only of interest at the GM level, but really the reason only the GM tends to see this KPI is because it is difficult to calculate, and often only the GM has the clout to get it – but it should be considered by all. To me, it is the king of KPIs for a publicly listed company. And it turns out the ROI may actually go down with increasing profit.

If making more widgets requires no further investment, then the maths is easy, but that is rarely true.

The question is this: is it better to have a large ‘average’ business or a smaller one with higher profit margins? It turns out, from an investor’s perspective, that the latter is fundamentally preferable.

The ROI treats a business a bit like a bank account: asking what interest rate does it offer? The business should be run to give the highest interest rate‘(%), not the highest interest ($). It is always possible to get more interest from a bank account – just put more money in the bank.

Translating a Mission Statement into company KPIs

I mentioned above that the ROI is a pretty darn good KPI – so can we use it alone? Of course not. Recall that the KPIs are mere numbers we measure that try to tell us how we are doing against the company mission statement, and while the company mission statement may unashamedly describe vast profits as a goal, this is almost universally not the whole story.

The organism that is the modern company has one particular need besides profit today, and that is profit tomorrow. The ROI does not capture this need, so more KPIs are required, and trickier ones – ones that capture sustainability, morale, innovation and reputation (brand value). This is the turf of the mission statement.

Even though the more cynical of my readers will know the mission is usually ‘make lots of dosh’, and anything beyond that is window dressing, I would venture that the mission statement is the first step to figuring out which KPIs to put first.

Let’s dissect a few examples. In my own 2-minute analysis I decided they fall into four types:

  1. To appeal to employees:
    McGraw Hill:
    We are dedicated to creating a workplace that respects and values people from diverse backgrounds and enables all employees to do their best work. It is an inclusive environment where the unique combination of talents, experiences, and perspectives of each employee makes our business success possible. Respecting the individual means ensuring that the workplace is free of discrimination and harassment. Our commitment to equal employment and diversity is a global one as we serve customers and employ people around the world. We see it as a business imperative that is essential to thriving in a competitive global marketplace.
  2. To appeal to customers (aka the unabashed PR stunt)
    A recent one from BP:
    In all our activities we seek to display some unchanging, fundamental qualities – integrity, honest dealing, treating everyone with respect and dignity, striving for mutual advantage and contributing to human progress.
    I couldn’t leave this one out from Mattel:
    Mattel makes a difference in the global community by effectively serving children in need . Partnering with charitable organizations dedicated to directly serving children, Mattel creates joy through the Mattel Children’s Foundation, product donations, grant making and the work of employee volunteers. We also enrich the lives of Mattel employees by identifying diverse volunteer opportunities and supporting their personal contributions through the matching gifts program.
  3. To appeal to investors. This is usually a description of how they are different or what they will do differently in order to achieve big dosh.
    CVS:
    We will be the easiest pharmacy retailer for customers to use.
    Walt Disney:
    The mission of The Walt Disney Company is to be one of the world’s leading producers and providers of entertainment and information. Using our portfolio of brands to differentiate our content, services and consumer products, we seek to develop the most creative, innovative and profitable entertainment experiences and related products in the world.
  4. For the sake of it – some companies clearly just made one up because they thought they had to, and obviously bought a book on writing mission statements:
    American Standard’s mission is to “Be the best in the eyes of our customers, employees and shareholders.”

Now a great trick when analysing the statement of any politician, and thus any mission statement, is to see if a statement of the opposite is absurd. In other words, if a politician says “I want better schools”, the opposite would be that he or she wants worse schools, which is clearly absurd. Thus the original statement has no real content, it is merely a statement of what everyone would want, including the politician’s competitors. Thus to judge a politician, or a mission statement, it is important to look not at what they say, but at what they say differently from the rest.

Mission statements seem rather prone to falling into the trap of stating the blindingly obvious, and as a result become trivial, defeating the point. Such is the case with American Standard. Of course you want to be the best. And of course it is your customers, employees and shareholders who you want to convince. Well no kidding!

So discounting those, we can see that a good mission statement will focus on difference. If we look at CVS, their mission is to be easy to use. This may seem like a statement of the obvious, but I don’t think it is – because they have identified a strategy they think will get them market share. Now they can design KPIs to measure ease of use. This is the sort of thinking that led to innovations like the ‘drive-thru’ pharmacy.

If we look at Disney, you can go further. “…[To] be one of the world’s leading producers and providers of entertainment…” OK, so they admit being #1 is unrealistic, and if you want to be taken seriously, you need to be realistic. But if you are one of many, how do you shine? “Using our portfolio of brands to differentiate” They realise they can sell a bit of plastic shaped like a mouse for a lot more than anyone else can. There is a hidden nod to the importance of brand protection. So KPIs for market share and brand awareness fall right out. They finish off with “the most creative, innovative and profitable entertainment” Well you can’t blame them for that.

Very rarely, you see a mission statement that not only shows how the company intends to make money, but may also inspire and make pretty decent PR. I like this one from ADM:

To unlock the potential of nature to improve the quality of life.

I have no idea how to get a KPI from that though!

Summary

In this article I have tried to illustrate how measuring a KPI is much like taking the pulse of a body – it’s a one-off health check, yes, but more importantly it can be a longer term measurement of how your interventions are affecting company fitness in the longer term.

I also try to describe some common pitfalls in the use of KPIs and presented four simple tests of their value:

  1. the KPI must correlate to a company’s mission
  2. the KPI must form part of a corrective feedback loop
  3. perverse incentives can be avoided by never considering any single KPI in isolation
  4. repeat the treatment only if the benefits repeat too

I have personally used this checklist (with a few refinements) over the years to some good effect in my own industry (minerals & materials) and though I am confident many of my readers will have more refined methods, I live in hope that at least one idea here will of benefit to you.

Reality: a hard sell

I can’t help but wonder if doggedly debunking all spewings from the purveyors of woo is somewhat a fool’s errand.

There are a number of pseudo-scientific disciplines whose concepts are inherently highly attractive and contagious to the average Joe, saying things that make him feel good and making him want to pass on the good news. Think of how easy to sell these messages are:

  • organic food – frolicking chickens, steaming compost, happy farmers, healthy food – a return to basics, back to a purer time when humans actually had roots in the earth and cared for it; you too can go organic!
  • complimentary medicine – age-old wisdom, so long suppressed by big pharma is unlocked just for those open-minded enough to look. Are you open-minded! Yes? Here are your keys to healthy prosperity!
  • astrology – our fates, entwined with the universe, form a beautiful unity; enigmatic scholars have acted custodian to its cipher through the ages. Are you a spiritual soul? You too can share time’s secrets!
  • parapsychology – our minds are more powerful than science knows, and we all have potential beyond our wildest dreams! But hold on! Only those willing to break free from the trappings of conventional science will ever see the light…
  • and of course the big kahuna, religion – imagine for a minute the greatest most wonderful thing in all the world, and that is but nothing compared with the joys that await the believer, and for all eternity too!

It is little wonder the bible I had as a kid said “Good News” on the cover.

The issue is that the logical shredding of these pieces is often a sobering dose of reality that fills most people with instant sleepiness:

  • organic food is not always kinder to the planet and claimed health benefits are of the ‘hard to verify’ sort
  • alternative medicine actually does work, but only the level one would expect from getting time, care, attention and the placebo effect
  • the laws of physics do allow marvellous things (x-rays, computers, holograms) but it takes serious study to understand why they don’t allow for the positions of stars and planets to have predictable effects of the day-to-day ongoings in suburbia.
  • the mind is indeed fabulously clever and poorly understood, but those tedious laws of physics, and indeed dry, cold logic, are annoyingly sticky when it comes to clairvoyance, ESP, psychokinesis and precognition.

So YAWN! Boring!! Logic and analysis mean effort, work, thinking things through, totting up totals, cross-checking claims, testing, questioning and doubting. Pretty much the opposite of nice & easy. Accepting we are not all-powerful, we are not immortal and that we will all be forgotten someday is just no fun. These are not messages that will go viral, that will breed missionaries, that would generate a manic fervour. More like manic depression.

—–

So the deck is stacked. Pseudo-scientific ideas persist because they are tenacious memes, and they are almost impossible to kill. They are contagious and sticky, and lovely and easy, and fighting them off requires not just the will, but also the ammunition.

And that is why it worthwhile to continue to fight the good fight – to keep trying to debunk poor thinking – to provide the ammunition to that small number, those that may be on their own, surrounded by superstition, but with that gift in their heart that is that first inkling of doubt.

I will do it for those that think they are alone as I once did.

We live in a time of unprecedented opportunity – people have better access than ever to the tools to arm themselves to achieve a new sort of ideal: to make life choices with full access to all the facts. We are after all free to choose to believe anything, the problem only comes when we are not given the choice. No information, no balance, no choice.

Stuff I Wish I Had Read When I Was Younger

Over the years I have supervised and mentored several PhD students, and recently our firm started to award scholarships to undergrads, and I was asked to support one such scholar. These scholars are from the best and brightest and so I got to thinking…

Graduates today have it tough, competition is tough, people work longer and harder than ever and stress is hitting us earlier and earlier in life – or so it seems. I would argue that, to some real extent, things have always been getting worse, and therefore by induction, we can prove that they have haven’t really changed at all.

No, the graduates of today have unparalleled opportunity to learn, to travel and to experience. The brightest graduates have the world at their feet and will be its commanders when we are are all retired and done for.

So what could I do to support this scholar? In the end it was easy – I asked myself – what do I know now that I wish I had known sooner? Most of this is in attitudes and is deep in my psychology, and is the result of direct experience – but it turns out that a healthy chunk of my scientific learning experience can be re-lived – by reading some of the books I think steered my course.

So I made a point to summarize some of the best science related books I have read (and some of the most useful internet resources I have found), and dumped the list complete with hyper-links in an email to the scholar. I hope she goes on to be president!

Now having gone to the effort, it would be a crime to keep this email secret, so here it is, (almost) verbatim!

================

As promised, here is a list of useful resources I wish I had known about when I was an undergrad. I am glad I got round to this, it should be useful for several other students I work with, and has also led to me revisiting a few things! I think I may brush it up and pop in on my blog if you don’t mind…obviously I won’t mention you!
Anyway, back to the business. To me, science is not all about chemistry, molecules, atoms, valence electrons and so on. To me, is is the process of trying to understand the world, and this set of materials I have hand picked, should you get through even a part of it, will not only educate but inspire.

This may not be the very best list, and I am sure there are many great books I have not read, but I have stuck with ones that I have, so you will have to rely on other people for further recommendations.

Jarrod’s reading list: science/psychology/economics & so on

  • I’ll start with something really easy, relevant and engaging – an excellent (if quirky) summary of material science: The New Science of Strong Materials – Prof Gordon  has written another on Structures that is also worth reading.
  • Ok, this next one is not a book, but a paper; I like it because it shows that many stuffy professors are wrong when they prescribe boring scientific prose for papers. This paper uses the criminal “us” and “we” and discusses subjects as if with a friend. Shocking form, especially for a junior scientist. This paper by an unknown, changed the world.
  • Guns, Germs and Steel” – this is large-scale scientific thinking at its best- the book looks at how we can explain why the world is the way it is (especially the inequality) by looking at how technology spreads through societies.
  • Mistakes were made…but not by me” – this is required reading if you want to work with other people, so its basically for everyone then…
  • Then to take it to the next level – “How the mind works…” – Stephen Pinker‘s other books are also good if you like this one.
  • “Flatland”, (full text here) was written in 1884, and is essential reading because it defines the cliche “thinking outside of the box”.
  • To make your upcoming economics courses more interesting, first read this easy-to-read popular book: “The Undercover Economist“.
  • Also, Freakonomics– it’s shameless self promotion by egotistical authors, but hell they are smart, so put up with it.
  • The Tipping Point –  Malcolm Gladwell is a current thinker I really like; he’s not satisfied to focus on one thing for very long – his other books are on totally different stuff, but are equally thought provoking.
  • The selfish gene” – Obviously I would firstly recommend “On the Origin of Species”, (full text here) but if you are short of time (which you should be as an undergrad), you can learn most of the basics, and also get updated (well up to the 1970’s at any rate) by reading Dawkins’ classic.
  • I couldn’t ignore statistics, so I will include two – one classic, “How to Lie with Statistics”  and a more modern one “Reckoning with Risk“, they are quite different, but either will get the important points across.

Alas, books are perhaps becoming obsolete, so I better include some other media:-

  • The first one is so good I can’t believe its free – try watch at least one a week, but the odd binge is essential too. http://www.ted.com/
  • Next, an excellent physics recap (or primer) – but  you need lots of time (or a long commute!) to get through this lot – look on the left menu for Podacts/Webcasts on this webpage: http://muller.lbl.gov/teaching/physics10/pffp.html – I cannot begin to praise the worthwhileness of this enough. It used to be called “Physics for future presidents” because it teaches you enough to understand the risks of nuclear energy, and the likelihood that we will all run our cars on water – and let you know when you are being duped or dazzled by big words.
  • When I was somewhat younger there was a TV show called Cosmos, hosted by Carl Sagan, you may know of it. You could watch in now here, though obviously it is dated, so perhaps you shouldn’t; the reason I mention it, is because it was key in creating a generation of scientists, people who were inspired by Carl to be inspired by the universe. The previous generation had the space race and the moon landings to inspire them, but since then science has been on a downhill, with 3-mile island, global warming, etc, etc, and we have had no more Carl Sagans to cheer for us; Cosmos was a rare bit of resistance in the decline of the importance of science in society. You may also know that there have been battles in society (well in the circles on intelligentsia at any rate) about science – on the one had the ‘two cultures debate‘ and more recently, the ‘anti-science’ movement (suggested in books like “The Republican War on Science“. I do not wish to indoctrinate you, but rather make you aware that being a scientist used to be cooler and used to be more respected and something is indeed rotten in the state of Denmark.
  • Getting back on track, here is an excellent guide to critical thinking (something else sadly lacking in the world) – don’t read it, listen to the podcast versions (also on itunes):
    “A Magical Journey through the Land of Logical Fallacies” – Part 1 and Part 2
    I think this should be taught in school. Brian Dunning’s other Skeptoid podcasts put these lessons into practice showing how a scientific approach can debunk an awful lot of the nonsense that is out there (alternative medicine, water dowsers, fortune tellers, ghost hunters, etc etc).
  • If you do happen to have any time left, which I doubt, there are several other podcasts on critical thinking – that use a scientific approach to look at the world and current affairs: –

=============

Postscipt – Dear readers, please feel free to append your own recommendations to my letter in the comments section below. If there is one thing I know well, and that’s how little I know. I feel I only started to read ‘the good stuff’ far too late in life, and so those with more years than me (or better mentors), please do share. But bear in mind, this is principally a science oriented list, and is meant to be accessible to undergraduates – I left out books like Principia Mathematica (Newton) because it is really rather unreadable – and the Princeton Science Library (though awesome) is probably a bit too intense. Also, in the 30 minutes since I sent the email, I have already thought of several others I sort of, well, forgot:

That’s it for now…

The ref’s word is the law: pragmatism in science and the legal system

The Silent Contract

If you have ever played a sport competitively, there is good chance you have experienced a blatant error by the referee. If you are anything like me, and if the game is balanced on a knife edge, there is a good chance your blood pressure shot right up and you had some kind words to say about the ref’s eyesight and quite possibly the ref’s mother too.

And if you are like me you would look back later and realise you were somewhat out of order. It was, after all, at worst an error, and no ref is perfect.

However, in those moments of blind frustration,  that rule: “the ref’s word is final” seems so wrong. In big-brother/1984 style, once the ref says it was a foul, it was a foul and unless he changes his mind within a few seconds it will be forever recorded in history as a foul. The real true events become an irrelevant fantasy.

This policy clearly does not serve the ‘truth’, so why has it developed in so many sports?

Most footballers understand why – and most judges understand why.

Let us consider what would happen if the ref’s word were not final.

Imagine a system of democratic debate and judgement over all close-calls at a sports event. Recall that all contenders and most of the audience are highly partisan. It would not be football, it would be a debate. It is obvious you need someone impartial, and you need something quick.

So what actually happens is that all stakeholders enter a silent contract to accept the ref’s errors in order to keep the game lively.

Implications in Law

So we have an example where we in society are willing to accept that the truth is not the priority. I am OK with this on the football field, but this type of pragmatism is actually used every day in the legal system, which is rather scary to me.

There is a lot in common between a ref and a judge, or indeed a jury. It falls on them to decide the official version of truth based upon the available evidence.

However, because the consequences of legal judgement are routinely more extreme than a free-kick, a more considered system usually develops. In most western courts, there is now a ‘threshold’ (called ‘reasonable doubt’ in the US) beyond which ‘the probable” becomes ‘the fact’. If this sounds rather crude, its because it is – they are saying that after they look at the evidence, and place the judgement on a scale of probability (0-100%) that they will set a number, above which the person is guilty and below which they are not – giving our grey world a little more contrast.

If they set the threshold at 100%, to ensure that no innocent man is ever convicted, you would convict no one. If they lower it to get more practical conviction rates, they will soon start to convict the innocent. Thus a compromise needs to be reached – the exact level will be a moving feast, but will generally reflect the culture and the Zeitgeist.

The Alternative…

Who else has has to make a call about truth?

Scientists are also trying to discern ‘truth’ – about magnetism, about cellular function, about  black holes, well about most everything really.

Yes, they do use pragmatism – we, for example, continue to use Newton’s Laws in most calculations because we know they work well enough for most practical purposes. We also continue to build on theories that have inherent contradictions (meaning that are probably flawed), because history has shown that such pragmatism still  moves us forward and is better than getting ‘stuck’. Yes, it does lead to some waste, some researcher’s entire lives are built on earlier mistakes that were ‘pragmatically’ ignored (think of homeopathy).

However, in the end, the scientist has to keep record of all pragmatism and return to it and root it out, because in science, the truth is the target, and no compromise can be left on the books.

This raises an interesting tension when a scientist is brought into legal proceedings to provide ‘opinion’. Judges (and politicians) mistakenly think that science deals in facts (not probabilities), and would also ask a scientist to make a pragmatic call ‘yes, this is the suspect’s DNA’. A good scientist would leave judgement to the judge and sprinkle their statements liberally with the word ‘probability’.

But can a legal system make judgements with probabilities? Without thresholds and verdicts? Could we dole out punishment in proportion to the probability of guilt? Could we punish several people for one crime if we know one of them did it? I don’t know if we could.

Has such a system cropped up anywhere? I don’t know if it has.

Open Verdict

So science can leave matters undecided on the basis that more evidence may come, but science too, from time to time, may benefit from a little pragmatism – because just as in the law-courts, we have the power to set potential killers loose with our inability to be decisive. Think of  two words: ‘global’ and ‘warming’.

What exactly is ‘science’?

I used to think science was the practice of the scientific method; i.e. you propose a hypothesis, you develop a test of the hypothesis, execute it and prove the hypothesis.

That worked for me until the end of high school.

At university, I was a true nerd. I read all my textbooks cover to cover (mainly because as I was too shy for girls and too poor for booze). During this time, the definition above started to fail. So much of the science was maths, statistics, observation, pattern recognition, logic and quite a bit of rote learning. Not all of it fitted into my definition of science. I became a fan of a new definition: science is the study of the nature of reality .

But then I did post-grad, and I realised that not much in science is ‘proven’ (I guess this is the point of post grad study). Evolution, for example, is not proven. That the sun revolves around the earth is not ‘proven’. I discovered that the only things that could be proven were ‘ideas’ about ‘other ideas’. Bear with me on this one.

Let us say we define the number system – this is an ‘idea’ or conceptual construction. Within this construction we can ‘prove’ that one and one is two. Because we ‘made’ the system, with rules, then we can make factual and true statements about it. We can’t do this about the real world – we cannot say anything with absolute certainly because we rely on flaky inputs like our own highly fallible perception.

It’s like that old chestnut: how can you be sure you are not living in a giant simulation? Of course you can argue that it is pretty unlikely and I would agree, and right there we have a clue to a better definition of science.

It turns out that much of modern science deals in ‘likelihood’ and ‘probability’ rather than proof and certainty. For example, we can say that the theory of evolution is very likely to be more-or-less right, as there is a lot of corroborating evidence. Science cannot be run like a law court – where the prosecution only need to reach a threshold of reasonable doubt to ‘prove’ someone guilty.

Aside for nerds: Science says you can use logic to prove things absolutely, but logic only works with ideas, and there is a breakdown between ideas and reality, so one can never prove things in reality. So it is thoroughly wrong for a court to say that someone has been proven guilty. The courts use this language as a convenience, to “draw a line under” a case as they have not found a moral way to dole out punishments based on probabilities. Imagine a world in which a murder suspect gets a 5 year sentence because the was a 20% chance he was guilty! Sports referees often operate in this decisive way, perhaps because it saves a lot of arguing!

Anyway, good science cannot just give up and say once there is consensus something passes from theory to fact. This is sloppy. We have to keep our options open – forever.

Think for example of Newton’s Laws of Motion. They are called ‘Laws’ because the scientific community had so much faith in them they passed from theory (or a proposed model) to accepted fact. But they were then found wrong. Strange that we persist in calling them laws!

It took Einstein’s courage (and open mindedness) to try out theories that dispensed with a key plank of the laws – that time was utterly inflexible and completely constant and reliable.

So it is that the canon of scientific knowledge has become a complex web of evidence and theories that attempt to ‘best fit’ the evidence.

Alas, there are still many propositions that many so-called scientists would claim are fact or at least ‘above reproach’. Evolution is attacked (rather pathetically), but the defenders would do well to take care before they call it ‘fact’. It is not fact, it is a superbly good explanation for the evidence, which has yet to fail a test of its predictions. So it is very very likely to be right, but it cannot be said to be fact.

This is not just a point of pedantry (though I am a bit of a pedant) – it is critical to keep this in mind as it is the key to improving our model.

Two great examples of models people forget are still in flux…

1) The big bang theory

2) Quantum theory

I will not go into global warming here though it is tempting. That is one where it doesn’t even matter if it is fact, because game theory tells you that either way, we better stop making CO2 urgently.

Back to the big bang.

I heard on the Skeptic’s Guide podcast today about an NSF questionnaire that quizzed people about whether they believed the universe was started with a massive explosion, and they tried to paint the picture that if you didn’t believe that, then you were ignorant of science. This annoyed me, because the big bang theory is now too often spoken of as if it were fact. Yes, the theory contributes viable explanations for red-shifted pulsars, background radiation, etc, etc, but people are quick to forget that it is an extrapolation relying on a fairly tall pile of suppositions.

I am not saying it is wrong, all I am saying is that it would be crazy to stop exploring other possibilities at this point.

You get a feeling for the sort of doubts you should have from the following thought experiment:

Imagine you are a photon born in the big bang. You have no mass, so you cannot help but travel at ‘light speed’. But being an obedient photon, you obey the contractions in the Lorentz equations to the letter, and time thus cannot pass for you. However, you are minding your own business one day when suddenly you zoom down toward planet earth and head straight into a big radiotelescope. Scientists analyse you and declare that you are background radiation dating from the big bang and that you have been travelling for over 13 billion years (they know this because they can backtrack the expansion of the universe). Only trouble is, that for you, no time has passed, so for you, the universe is still new. Who is right? What about a particle that was travelling at 0.999 x the speed of light since the big bang? For it, the universe is some other intermediate age. So how old is the universe, really?

This reminds us of the fundamental proposition of relatively – time is like a gooey compressible stretchable mess, and so is space, so the distance across the universe may be 13.5 billion light years, or it might be a micron (how it felt to the photon). It all depends on your perspective. It is much like the statement that the sun does not revolve around the earth and that it is the other way around. No! The sun does revolve a round the earth. You can see it clearly does. From our perspective at least.

Now, quantum theory.

Where do I start? String theory? Entanglement? Please.

The study of forces, particles, EM radiation and the like is the most exciting part of science. But being so complex, so mysterious, so weird and counter intuitive, it is super vulnerable to abuse.

Most people have no idea how to judge the merits of quantum theories. Physicists are so deep in there, they have little time (or desire or capability) to explain themselves. They also love the mystique.

I do not want to ingratiate myself with physicists, so I will add that the vast majority have complete integrity. They do want to understand and then share. However, I have been working in the field for long enough to know that there are weaknesses, holes and downright contradictions in the modern theory that are often underplayed. In fact these weaknesses are what make the field so attractive to people like me, but is also a dirty little secret.

The fact is that the three forces (weak nuclear, strong nuclear and magnetic) have not been explained anything like as well as gravity has (by relativity). And don’t get me started on quantum gravity.

———————-

Anyway, thinking about all these issues, I concluded that science was (definition #3) the grand (platonic) model we are building of reality, ever evolving to best fit our observations.

My man, Plato

That works well for me. However, I recently came across a totally different definition for science:

# 4) “Science is a tool to help make the subjective objective.”

OK I paraphrased it to make it more snappy. It was really a discussion about how science was developed to overcome the fallibility of the human mind. Examples of weaknesses it needs to overcome are:

  1. The way our perception is filtered by preconceptions
  2. How we see pattern where there is none
  3. How we select evidence to match our opinion (confirmation bias)
  4. How we  read too much into anecdotal evidence
  5. etc etc.

I could go on. So ‘science’ is the collection of tricks we use to overcome our weaknesses.

I like this definition. We are all going about, and in our heads we are building our model of the world… and its time for an audit!