Confusion Matrix and Cyber Security
In the field of machine learning and specifically the problem of statistical classification, a confusion matrix, also known as an error matrix, is a specific table layout that allows visualization of the performance of an algorithm. In predictive analytics, a confusion matrix is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives. This allows more detailed analysis than mere proportion of correct classifications (accuracy). Accuracy will yield misleading results if the data set is unbalanced; that is, when the numbers of observations in different classes vary greatly. How…?
So for example, if a model is trained to classify and differentiate between the colors red and blue, and then in a dataset there are 990 blue records and 10 red records, now if the model predicted all the records as blue the accuracy of the model comes out to be 99%, however we can see that it accurately predicted 0 red records. Hence judging a model with the help accuracy score in this type of cases is not ideal.
From the above given example we can see how a confusion matrix look like. But how to actually create a confusion matrix?
Say we are given a trained model say for predicting rain and we are tasked to find its confusion matrix based on a given testing dataset. Now to create a confusion matrix we require predicted values, so we are going to run the X(feature) of the testing dataset through our model to fetch all the predicted values. Now, we will create an empty matrix similar to one given above and then we will start comparing the predicted values with the actual values provided in the dataset. If the prediction of our model is that rain will fall and in the dataset we can see that rain does fall we will increase a count in the bottom-right box(TP True Positive), if in prediction we find that there will be no rain for some record and for the same record we find that there was no rain then we will increase count in top-left box(TN True Negative). If however we find that the model’s prediction for some record is that there will be rain and from the record we find that their was no rain we will raise the count in top-right box(FP False Positive), and lastly if our model predicts that their will be no rain but through the dataset we find that there was a rain for that record then the count is raised in bottom-left box(FN False Negative). Hence we can see a model has two types of errors:
1. False Positive.
2. False Negative.
Machine Learning In Cyber Security
In Cyber Security ML models are used to predict attacks; for example Intrusion Detection System(IDS, this system is used to detect attacks and then raise alerts) and Intrusion Protection System(IPS, this system is used to detect attacks and then proactively stop the attacks from happening); and vulnerabilities. Now, since the systems are just ML models at their core, they are prone to error, as no model can be 100% accurate all the time. Hence there is a requirement to study the confusion matrix of the model and what are the meaning and repercussions of both the type of errors.
False Positive
This is the case when our security system raises an alert when in reality there is no attack happening. These alerts are just a nuisance and hence are usually to be avoided for further investigations as it will require unnecessary work and hence will use up man power. However these alerts will also train the worker to turn a blind eye towards the alerts, and hence increasing the possibility of a True Positive’s alert to go unnoticed.
This building up of ignorance leads to attacks such as in Target data breach, “where the technology used to monitor intrusions provided multiple alerts on different occasions regarding suspicious activities,” says Pritesh Parekh, CISO at Zuora. “The alerts were buried in hundreds of false positives and became deprioritized on the list of security items, resulting in a major data breach,” Parekh says.
False Negative
A false negative occurs when the security system fails to identify a threat. It produces a “negative” outcome (meaning that no threat has been observed), even though a threat exists. Hence, here the system will not raise any alerts when the attack happens as the attack will be seen as normal activity. This type of error does lead to a lot of intrusions however the simple fix for this is having multiple layers of security especially that implements positive security models where, the system denies access to all requests except for those that match the characteristics of legitimate, desirable traffic.
That’s it kind readers…
Thankyou for reading. See you later!!! (￣︶￣）↗
