Classification Metrics

Evaluating the performance of a classification model involves comparing its predictions with actual labels using various metrics. Each metric highlights different aspects of the model's performance.

1. Accuracy

The ratio of correctly predicted instances to the total number of instances.

$$\text{Accuracy} = \frac{\text{TP} + \text{TN}}{\text{TP} + \text{TN} + \text{FP} + \text{FN}}$$

While simple and widely used, accuracy can be misleading for imbalanced datasets where one class dominates.

2. Precision

The ratio of correctly predicted positive observations to the total predicted positives.

$$\text{Precision} = \frac{\text{TP}}{\text{TP} + \text{FP}}$$

Precision is crucial when the cost of false positives is high.

3. Recall (Sensitivity or True Positive Rate)

The ratio of correctly predicted positive observations to all actual positives.

$$\text{Recall} = \frac{\text{TP}}{\text{TP} + \text{FN}}$$

Recall is critical when the cost of false negatives is high, such as in medical diagnoses.

4. F1 Score

The harmonic mean of precision and recall, balancing the two metrics.

$$\text{F1 Score} = 2 \cdot \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$

Useful in cases where an even balance between precision and recall is required, especially for imbalanced datasets.

5. Specificity (True Negative Rate)

The ratio of correctly predicted negative observations to all actual negatives.

$$\text{Specificity} = \frac{\text{TN}}{\text{TN} + \text{FP}}$$

Specificity is useful when the cost of false positives is high.

6. ROC Curve and AUC (Area Under Curve)

The ROC Curve plots the True Positive Rate (Recall) against the False Positive Rate (1 - Specificity) at different threshold levels.

The AUC (Area Under Curve) measures the overall ability of the model to distinguish between positive and negative classes.

AUC values close to 1 indicate a better model, as it can effectively distinguish between positive and negative classes.

7. Confusion Matrix

A matrix showing counts of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) classifications.

True values are when actual == predicted, and False values occur otherwise.