The Hidden Cost of Correlations in Neural Networks

Remember when you first moved out of home and didn't have much money, you'd argue with your housemates about the electricity bill. You probably just split the electricity bill 50/50, but why? You could have avoided all arguments and just paid for what you used. There would be an additional cost as you both invest time to work out who to attribute each cost to; the very act of deciding to invest time in this bill-splitting exercise means you must first accept the cost–accuracy trade‑off that we talked about in the last post.

But we'll run into an additional issue when we split the bill. What if both housemates watched a film together? Did they both enjoy it equally? Did one of them choose the film? Should the person who didn't want to watch the film, or even enjoy watching it, have to pay for the electricity? We'll call this an ambiguity of attribution; we don't know who to attribute the electricity usage to, and the act of working it out is almost definitely not worth the information we get from it.

In neural networks, this ambiguity of attributions can be both expensive and damaging to accuracy. But they can sneak in much more easily and be very hard to spot. So let's look at how we can spot if our neurons are trying to work out these ambiguities.

This post follows on from the last post where we learned how some nodes are redundant, or offer a low predictive relevance.

Weighting Features and Rank

Let's stick with that tastiness target from our last post; we really want to predict how tasty a food is before we eat it, but let's pick a more obviously useful input feature, something we know affects taste: sugariness (I know what you're thinking, just go with it).

To be able to create a neural network that can predict how tasty food is, we'll first need to try some foods and see how tasty they are. We can then use this as our training data.

If we were to train a very simple neural network, we could just try to calculate the predictive relevance of the two features: how important is each feature to calculating the target? We're simply trying to calculate what x and y are in the below system of linear equations:

2x + 2y = 2
5x + 5y = 5
9x + 9y = 9

This is similar to what our neural network is calculating in training. x is how important sweetness is, and y is how important sugariness is. If we figure these values out, we can make new predictions about the tastiness of food with just these two values. We need a number to multiply all the sweetness scores by and a number to multiply the sugariness score by that will get us closest to our tastiness score. So we just need to figure out what x and y are.

Let's try and solve this system of equations: well, if you did any algebra in school, you'll notice that we can collapse them into a single equation. Divide everything in the first one by 2, everything in the second one by 5, and everything in the last one by 9. In all cases, we end up with x + y = 1.

If we can collapse the system of equations into one single equation, we could describe the system of equations as having a rank of 1 (there is only one independent piece of useful information). Now rank is a maths term; you don't need to understand all the ways to calculate it, but as we learn more about visualising and assessing neural networks, knowing what it represents will become essential.

If you want to figure out what two unknown values are (x and y) and you only have one unique equation (x + y = 1), you won't be able to find a single solution. There are infinitely many solutions: (x=1, y=0), (x=0, y=1), (x=0.5, y=0.5), and the difference between 0, 0.5, and 1 is significant when training a neural network.

Linearly Dependent Features

In our last post, we plotted our data with the features being the axis. Let's take three items of food and plot their activation values onto a plot.

We can see that they appear in a straight line from the bottom left to the top right. If we train the neural network with a target of predicting tastiness, we'll see the exact same pattern that we saw in the last post, with one feature or the other having low predictive relevance, except in this case it could be either feature.

We don't even need to train this network to identify that it has a problem; we can see a pattern in the feature values themselves: they always plot onto a line. In algebra, this relationship between the two features is called a linear dependency.

By visualising this, we can also better understand what we were talking about with our equations. When we said it had a rank of one, what we meant was that only one dimension is being used out of the two available on this plot. We could have visualised it with a single dimension by plotting them on a line. It's worth thinking about these concepts with this very simple example because it will become harder to comprehend as we add more dimensions and start looking at hidden nodes.

Linear dependency gets slightly harder to predict when we consider that it could correlate in different directions, like these two examples below. (Please note: negative correlation is a form of correlation. Some people incorrectly think that negative correlation = no correlation.)

The important question is whether we can predict one value from the other. We can predict that something will be less healthy as it becomes sweeter, but by how much. In the examples above, when we train our neural network, we're forcing the network to work out whether candy floss is tasty because it's sweet, because it's unhealthy, or one of the infinite combinations of the two.

Now imagine our neural network had been fully trained with our training data using the healthiness and sweetness feature values. All the network knows from our training is that there is a correlation line going from the top left to the bottom right, and how tasty something is can be predicted based on how far down that line we plot.

If at some point we do find a food that is healthy value of 9 and a sweetness value of 8, we’ll be told it’s bland, with a tastiness score of around 4. But when we find a food with a healthiness score of 1 and sweetness score of 2, we’ll be told it's tasty, with a tastiness score closer to 6. So in our last post we had an extra feature that just cost us time, but compared to a single neural network with a single sweetness feature, this is actually giving us worse predictions.

Three-Dimensional Linear Dependencies

Next let's consider a neural network with three features: salt, sugar and healthy. We again put lots of training data into this network to help us predict how tasty different food is. In this world we're imagining that there are now exactly two things that make a food unhealthy (these are hypothetical examples). If something is very salty but not very sweet, it is still unhealthy. In this example, we can still predict how healthy it is, but now with a combination of other features.

This plot has three dimensions, but we're only using two of them, or we could say it has a rank of two. We're still putting extra work in, and we're still not sure if things are tasty just because they're unhealthy or if it's some combination of sugar and salt that makes it taste good. We can already see how this simple concept can be harder to spot when we add more features. And in a later post we'll talk about the increased difficulty of spotting linear dependency in hidden nodes.

There is an ambiguity of attribution in these examples, just like working out that system of equations above or splitting the electricity bill from that time your housemate forced you to watch that boring movie. This ambiguity of attribution is a larger problem than simply having a redundant feature because we can't reliably determine the importance of each feature.

Activation Correlation

In a real neural network, we never have such clean data, and the weights are never perfect, since we're trying to find the closest prediction. We rarely see perfect linear dependence in practice, but near-duplicates are common.

I'll be using the term activation correlation here and in future posts to describe how correlated two nodes or features are. There are lots of other terms used in machine learning for this (e.g. Rank Deficiency, Multicollinearity), sometimes used to describe just a correlation of activations on the features or just the correlation between activations on the nodes. Unfortunately, this is just an issue with a new high-tech discipline with lots of people working in it; you get a lot of jargon. So just be aware I'll be using the term activation correlation, but someone else might call it something else.

There are several machine learning techniques to try and avoid nodes from correlating in training; however, without actually visualising the positioning of the activation values, it's impossible to know if they have been successful. To understand this better, we'll finally need to look at those hidden nodes in the next post.

Summary

When features are redundant or strongly related, they create ambiguity of attribution, meaning the model can still make good predictions but can’t clearly tell which feature deserves the credit. This happens because the model is effectively working with fewer independent pieces of information (lower “rank”), leading to many equally valid ways to assign importance. In more complex or less obvious cases (activation correlation across multiple features), this can not only make the model harder to interpret but also reduce prediction accuracy.

Glossary

• Ambiguity of attribution: When it's unclear which feature or node is contributing to the outcome.
• Linear dependency: When one feature can be predicted from another, so they are not truly independent.
• Activation correlation: When two activations move together or follow a similar pattern.
• Rank: The amount of independent information in a set of inputs.