Precision vs Accuracy Vs Recall?

Precision vs Accuracy Vs Recall?



Accuracy - Accuracy is the most intuitive performance measure and it is simply a ratio of correctly predicted observation to the total observations. One may think that, if we have high accuracy then our model is best. Yes, accuracy is a great measure but only when you have symmetric datasets where values of false positive and false negatives are almost same. Therefore, you have to look at other parameters to evaluate the performance of your model. For our model, we have got 0.803 which means our model is approx. 80% accurate.

Accuracy = TP+TN/TP+FP+FN+TN

Precision - Precision is the ratio of correctly predicted positive observations to the total predicted positive observations. The question that this metric answer is of all passengers that labeled as survived, how many actually survived? High precision relates to the low false positive rate. We have got 0.788 precision which is pretty good.

Precision = TP/TP+FP

Recall (Sensitivity) - Recall is the ratio of correctly predicted positive observations to the all observations in actual class - yes. The question recall answers is: Of all the passengers that truly survived, how many did we label? We have got recall of 0.631 which is good for this model as it's above 0.5.

Recall = TP/TP+FN

F1 score - F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful than accuracy, especially if you have an uneven class distribution. Accuracy works best if false positives and false negatives have similar cost. If the cost of false positives and false negatives are very different, it's better to look at both Precision and Recall. In our case, F1 score is 0.701.

F1 Score = 2(Recall Precision) / (Recall + Precision)

Error vs variance vs bias?

Error vs variance vs bias?



In statistics and machine learning, the bias-variance tradeoff is the property of a set of predictive models whereby models with a lower bias in parameter estimation have a higher variance of the parameter estimates across samples, and vice versa. The bias-variance dilemma or problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learning algorithms from generalizing beyond their training set: The bias is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting). The variance is an error from sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (overfitting). The bias-variance decomposition is a way of analyzing a learning algorithm's expected generalization error with respect to a particular problem as a sum of threeterms, the bias, variance, and a quantity called the irreducible error, resulting from noise in the problem itself.

False negatives vs false positives? When is either one worse than the other?

False negatives vs false positives? When is either one worse than the other?


False positive: A false positive error, or in short a false positive, commonly called a ""false alarm"", is a result that indicates a given condition exists, when it does not. For example, in the case of ""The Boy Who Cried Wolf"", the condition tested for was ""is there a wolf near the herd?""; the shepherd at first wrongly indicated there was one, by calling ""Wolf, wolf!"" A false positive error is a type I error where the test is checking a single condition, and wrongly gives an affirmative (positive) decision. However it is important to distinguish between the type 1 error rate and the probability of a positive result being false. The latter is known as the false positive risk (see Ambiguity in the definition of false positive rate, below).

False negative: A false negative error, or in short a false negative, is a test result that indicates that a condition does not hold, while in fact it does. In other words, erroneously, no effect has been inferred. An example for a false negative is a test indicating that a woman is not pregnant whereas she is actually pregnant. Another example is a truly guilty prisoner who is acquitted of a crime. The condition ""the prisoner is guilty"" holds (the prisoner is indeed guilty). But the test (a trial in a court of law) failed to realize this condition, and wrongly decided that the prisoner was not guilty, falsely concluding a negative about the condition. It depends, must answer why it depends.


Subscribe to: Posts (Atom)