Overfitting

In a probably apocryphal story, an AI research team tried to train a computer to detect a tank among a forest.  They showed it 50 tanks in a forest so it could learn what a tank looked like and showed it 50 forests with no tank so it could learn about its absence.  Then they tested their trained machine and found it inadequate to detect a tank.  It did however detect the depicted weather since many of the pictures with a tank were taken on cloudy days, and the ones without a tank on sunny days.

AI’s don’t necessarily discriminate what’s important. In fact, one of the challenges in machine learning revolves around creating too sophisticated of a model.  With enough complexity, one can train the computer to be 100% accurate on its training data, but be very inaccurate on guessing anything else.  Creating too complex of a model leads to the problem of over-fitting and stems from not having enough data to give any statistical significance to the model it created. Instead, one should train the computer keeping Occam’s Razor in mind, that the simplest explanation is probably better and more accurate.

While interesting in machine learning, us humans also tend to overfit our explanations in life.  We collected one or two data points and create a theory about why something happens. For example, we go to a restaurant and then get a cold afterwards.  The first time we think nothing of it but then 4 months later, we go and we get another cold.  Maybe we’re getting a cold at the restaurant?  So we don’t go for a year but then some friends are going and we take extra Vitamin C beforehand and don’t get sick.  We create a rule, we can go to that restaurant but unless we take Vitamin C we will get sick.  Our explanations fit all the data we have, and yet in reality fail to predict what will really happen in the future.

We may think this example silly, but we should consider how complex a model we create when explaining the phenomenon around us, especially as the explanation gets more elaborate.  We should treat our hypothesis with a degree of skepticism and uncertainty rather than feeling like we KNOW how it will all unfold.

Photo Credit: New Boston Tailor and Monte Carol Tailor