Three lessons in learning from learning machines
We neither enter nor exit the world with omniscience, but in the intervening time, learn enough about it to survive and perhaps even to have something to teach those who come after us. “Learning” refers to a change in our beliefs and behaviors that is rooted in our own or other’s experiences. Our success as a species, to a very large extent, is rooted in our collective capacity to learn.
We now have new members in our collective of learners: machines that can also learn. Machine learning (ML) is the latest incarnation of Artificial Intelligence, and it works by learning about complex patterns of associations between variables. Its “experience” is the data we feed it, and it changes its “beliefs and behaviors”- the models estimated to fit the data- through algorithms that minimize prediction errors.
The capacity of ML to learn is still rudimentary, and often it learns in ways that may be quite different from the ways we do. However, building learning machines has given us some powerful insights into the nature of learning problems themselves. Even if machines solve those problems differently from us (and in ways we would not or could not emulate), insight into the structure of the problems may still be useful for us. Here are three instances.
The exploration-exploitation trade-off
We learn about the world by acting upon it.
However, to the extent we learn about how to act by learning from what happens when we act, there is always a risk of biased initial beliefs becoming self-confirming.
Suppose you start out thinking it’s better to pursue project A rather than project B. The projects could be anything really- investments, hobbies, romantic partners. Given your beliefs, quite sensibly, you choose to invest in A. Let’s say you found the outcome to be what you expected. You will, again quite sensibly, learn that your beliefs about the wisdom of pursuing A instead of B should be strengthened. But it is possible that you started with the wrong idea about B — it would in fact have been better to do B instead of A- and since you did not act to choose B, you will never know. Your biased beliefs about B being inferior to A have become self-confirming, because acting on these beliefs prevented you from discovering they are biased.
To escape our legacy biases — flaws in our current understanding of the world- it is therefore necessary to take actions that are inconsistent with our current understanding. But every time we do so, we risk underutilizing the insight that is part of our current understanding. This is the challenge of balancing “exploration” (for better actions, by trying out things that we don’t currently believe to be valuable) with “exploiting” the value of our current understanding (by acting consistently with it). Every learning system that learns through its actions- engages in “learning by doing” — faces this challenge, and those who study complex adaptive systems (including organizations) have known this for a long time.
In the world of ML, learning by doing is referred to as reinforcement learning, and its central trade-off is indeed between exploration and exploitation, though the specific ways in which ML algorithms optimize this can be quite different from the way we approach it.
But the trade-off is there, and we might learn to recognize it and perhaps do better at managing it, by recognizing the central implication: it is quite sane- even essential- to do crazy things to learn better. How crazy? A thumb rule is to dial up the craziness the less familiar we are with a situation but the more confident we are that the worst outcome won’t be disastrous. Easier said than done of course, particularly in a culture that celebrates consistency and frowns on craziness. That’s why we might need to create space for being playful- to lower the social hurdles for doing something that seems unreasonable.
The bias-variance trade-off
We learn about the world by observing it.
If we find stable patterns in our experience (i.e., the cumulation of what we observe), this can give us an advantage in terms of being able to predict what is likely to happen again when parts of the pattern re-appear. This applies to everything from reading the weather, to reading the social cues in an interaction.
However, our experience could be made up of noisy observations. Random factors may play a role in generating what we perceive as a pattern; maybe the flock of birds flying east overhead at dawn was just a coincidence and had nothing to do with the fact that it rained later; perhaps looking at her phone during your conversation signaled nervousness, not disinterest. The world does not give us “clean”, noise-free observations, and this poses a major challenge for any learner. How can we ensure that what we have learnt from our experience is pattern, not noise?
If we take our experience to be a perfect, noise-free data set, we risk overgeneralizing from it (think of the false cues such as the birds and the surreptitious glance at the phone). But we cannot ignore our experience altogether or under-generalize from it. Perhaps the direction the birds flew in mattered, not the timing. The trade-off here is between placing too much (or too little) reliance on what we have observed so far to guide future action. We may develop convincing interpretations of our observations, but the interpretations may be too strongly shaped by the noise in what we observed. That makes them a poor- possibly dangerous-guide to future actions.
Supervised and unsupervised ML algorithms learn by observing existing data (rather than produce it by acting upon the world, as reinforcement learning algorithms do), and the central trade-off for these is the “bias-variance” trade-off. Setting aside the terminology, this is exactly the challenge of learning too little from experience (“under-fitting the data”) vs. too much (“overfitting”).
Again, the specific ways in which ML algorithms optimize this trade-off can be quite different from the way we approach it. But it can already be useful for us to recognize the trade off, and the central implication: we must treat our experience with some skepticism. How much skepticism? The noisier the context we are observing, the less confident we should be about the patterns we think we are seeing. Further, the more elaborate our explanations for what we have observed, the less confident we should be of them because complex explanations are more likely to be overfitting the data. Experience deserves a healthy respect, but not blind obedience.
The explanation-prediction trade off
We learn from the explanations we give each other.
However, the best explanations — easy to understand and transmit- may not be the most accurate ones.
A lot of the current success of ML comes from the use of deep learning models- which can fit extremely complicated models that capture patterns involving thousands of variables and interactions between them. These patterns might never have been seen by a human because they are just too complex.
Even after an algorithm has found it, understanding what this pattern means and why it exists can still be very hard for us- we typically cannot process more than a small number of variables and their interactions at a time (the number may be as low as 7). We often end up “black-boxing” such models- using them without fully grasping why and how they work. Of course, there is nothing mysterious going on here- it is just the sheer mundane complexity of it that makes it incomprehensible to us.
But the fact that such models have had remarkable success at being able to make predictions ranging from what an image or soundbite represents, to forecasting repayment risk, implies a sobering lesson for us: there may be order and pattern in the world that is far more complex than we will ever be capable of comprehending; our presumption to the contrary is at best, hope; at worst, hubris.
An active area in ML research involves attempts to make the models easier to understand for humans (called the Explainable AI movement). The goal is to optimize the tradeoff between explanation and prediction, but the tradeoff itself is known to be inescapable. This should make us more conscious of the fact that when we use the term “explanation”, to be precise we must really always precede it with the phrase “human comprehensible”: all explanations must always pass through the bottleneck of our cognitive capacities. The central implication is that this will require simplification- and therefore may require giving up on accuracy (of prediction). Intuitive explanations do not come “for free”- they might require giving up on accuracy.
Teachers and speakers have always known this secret. I may have a model with 26 variables that predicts CEO exits with an accuracy of 75%, but the explanation that I can use in my speeches is the one with fewer terms and a lower accuracy. How much accuracy should I give up? Unless I am dealing with an audience of atypically smart (or dumb) people, the real constraint is the nature of the problem itself- for some problems the decline in accuracy due to simplification will be steeper than for others. That’s what we need to figure out about what we are trying to explain to choose how blissfully ignorant our audience can remain.
But this is just as true when it comes to our own understanding of a problem domain- some just might have a very steep trade-off between understanding and accuracy. This means that the study of some types of complex systems (including organizations) might be able to progress faster by giving up on human comprehensible explanations.
The ideas in this essay are drawn from “Algorithm Supported Induction for Theory Building” (with Sreshtha, YR, He, VF and von Krogh, G in Organization Science, 2020) and “Self-confirming biased beliefs in organizational learning by doing” (with S. Park, Complexity, 2021).
