API Reference

ae(actual, predicted)

Compute the elementwise absolute error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
ae(actual, predicted)

mae(actual, predicted)

Compute the mean absolute error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mae(actual, predicted)

mdae(actual, predicted)

Compute the median absolute error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mdae(actual, predicted)

se(actual, predicted)

Compute the elementwise squared error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
se(actual, predicted)

sse(actual, predicted)

Compute the sum of squared errors between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
sse(actual, predicted)

mse(actual, predicted)

Compute the mean squared error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mse(actual, predicted)

rmse(actual, predicted)

Compute the root mean squared error between two numeric vectors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rmse(actual, predicted)

nrmse(actual, predicted; normalization=:range)

Compute the Normalized Root Mean Squared Error.

Normalizes RMSE to make it comparable across different scales.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector
• normalization::Symbol: Normalization method (default: :range)
  • :range - Normalize by range (max - min) of actual values
  • :mean - Normalize by mean of actual values (coefficient of variation of RMSE)
  • :std - Normalize by standard deviation of actual values
  • :iqr - Normalize by interquartile range of actual values

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
nrmse(actual, predicted)  # Normalized by range
nrmse(actual, predicted, normalization=:mean)  # CV(RMSE)

max_error(actual, predicted)

Compute the maximum absolute error between two numeric vectors.

Useful for understanding the worst-case prediction error.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
max_error(actual, predicted)

max_ae(actual, predicted)

Alias for max_error. Compute the maximum absolute error.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

bias(actual, predicted)

Compute the average amount by which actual is greater than predicted.

If a model is unbiased, bias(actual, predicted) should be close to zero.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
bias(actual, predicted)

percent_bias(actual, predicted)

Compute the average amount that actual is greater than predicted as a proportion of the absolute value of actual.

Returns a proportion (0.1 = 10%). Returns -Inf, Inf, or NaN if any elements of actual are 0.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
percent_bias(actual, predicted)

mpe(actual, predicted)

Compute the Mean Percentage Error (signed) as a proportion.

Unlike MAPE, MPE can indicate systematic bias: positive values indicate under-prediction on average, negative values indicate over-prediction.

Returns a proportion (0.1 = 10%). Returns Inf, -Inf, or NaN if actual contains zeros.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mpe(actual, predicted)

ape(actual, predicted)

Compute the elementwise absolute percent error between two numeric vectors.

Returns proportions (0.1 = 10%). Returns -Inf, Inf, or NaN if actual contains zeros.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
ape(actual, predicted)

mape(actual, predicted)

Compute the mean absolute percent error between two numeric vectors.

Returns a proportion (0.1 = 10%). Returns -Inf, Inf, or NaN if actual contains zeros. Due to instability at or near zero, smape or mase are often used as alternatives.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mape(actual, predicted)

smape(actual, predicted)

Compute the symmetric mean absolute percentage error between two numeric vectors.

Defined as 2 * mean(abs(actual - predicted) / (abs(actual) + abs(predicted))). Returns NaN only if both actual and predicted are zero at the same position. Has an upper bound of 2.

smape is symmetric: smape(x, y) == smape(y, x).

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
smape(actual, predicted)

wmape(actual, predicted)

Compute the Weighted Mean Absolute Percentage Error.

WMAPE weights errors by the magnitude of actual values, making it more robust than MAPE when actual values vary significantly in magnitude.

WMAPE = sum(|actual - predicted|) / sum(|actual|)

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
wmape(actual, predicted)

sle(actual, predicted)

Compute the elementwise squared log error between two numeric vectors.

Adds one to both actual and predicted before taking the natural logarithm to avoid taking the log of zero. Not appropriate for negative values.

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector
• predicted::AbstractVector{<:Real}: Predicted non-negative vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
sle(actual, predicted)

msle(actual, predicted)

Compute the mean squared log error between two numeric vectors.

Adds one to both actual and predicted before taking the natural logarithm to avoid taking the log of zero. Not appropriate for negative values.

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector
• predicted::AbstractVector{<:Real}: Predicted non-negative vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
msle(actual, predicted)

rmsle(actual, predicted)

Compute the root mean squared log error between two numeric vectors.

Adds one to both actual and predicted before taking the natural logarithm to avoid taking the log of zero. Not appropriate for negative values.

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector
• predicted::AbstractVector{<:Real}: Predicted non-negative vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rmsle(actual, predicted)

rse(actual, predicted)

Compute the relative squared error between two numeric vectors.

Divides sse(actual, predicted) by sse(actual, mean(actual)), providing the squared error relative to a naive model that predicts the mean for every data point.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rse(actual, predicted)

rrse(actual, predicted)

Compute the root relative squared error between two numeric vectors.

Takes the square root of rse(actual, predicted).

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rrse(actual, predicted)

rae(actual, predicted)

Compute the relative absolute error between two numeric vectors.

Divides sum(ae(actual, predicted)) by sum(ae(actual, mean(actual))), providing the absolute error relative to a naive model that predicts the mean for every data point.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rae(actual, predicted)

explained_variation(actual, predicted)

Compute the explained variation (coefficient of determination, R²) between two numeric vectors.

Subtracts rse(actual, predicted) from 1. Can return negative values if predictions are worse than predicting the mean.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
explained_variation(actual, predicted)

adjusted_r2(actual, predicted, n_features)

Compute the Adjusted R² (coefficient of determination adjusted for number of predictors).

Adjusted R² penalizes the addition of irrelevant features to a model.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector
• n_features::Integer: Number of features/predictors in the model

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
adjusted_r2(actual, predicted, 2)  # Model with 2 features

huber_loss(actual, predicted; delta=1.0)

Compute the Huber loss, which is quadratic for small errors and linear for large errors.

Huber loss is less sensitive to outliers than MSE. For errors smaller than delta, it behaves like MSE; for larger errors, it behaves like MAE.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector
• delta::Real: Threshold where loss transitions from quadratic to linear (default: 1.0)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
huber_loss(actual, predicted)
huber_loss(actual, predicted, delta=0.5)

log_cosh_loss(actual, predicted)

Compute the log-cosh loss, a smooth approximation of MAE.

Log-cosh is approximately equal to (x^2)/2 for small x and abs(x) - log(2) for large x. It has the advantage of being twice differentiable everywhere.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
log_cosh_loss(actual, predicted)

quantile_loss(actual, predicted; quantile=0.5)

Compute the quantile (pinball) loss for quantile regression.

The quantile loss asymmetrically penalizes over-prediction and under-prediction. At quantile=0.5, this is equivalent to MAE. For quantile<0.5, under-prediction is penalized more; for quantile>0.5, over-prediction is penalized more.

Arguments

• actual::AbstractVector{<:Real}: Ground truth numeric vector
• predicted::AbstractVector{<:Real}: Predicted numeric vector
• quantile::Real: Target quantile in (0, 1) (default: 0.5)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
quantile_loss(actual, predicted, quantile=0.5)  # Equivalent to MAE
quantile_loss(actual, predicted, quantile=0.9)  # Penalize under-prediction more

pinball_loss(actual, predicted; quantile=0.5)

Alias for quantile_loss. Compute the pinball loss for quantile regression.

tweedie_deviance(actual, predicted; power=1.5)

Compute the Tweedie deviance for generalized linear models.

The power parameter controls the distribution assumption:

• power=0: Normal distribution (equivalent to MSE)
• power=1: Poisson distribution
• power=2: Gamma distribution
• power=3: Inverse Gaussian distribution
• 1 < power < 2: Compound Poisson-Gamma (common for insurance claims)

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector
• predicted::AbstractVector{<:Real}: Predicted non-negative vector
• power::Real: Tweedie power parameter (default: 1.5)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
tweedie_deviance(actual, predicted, power=1.5)

mean_poisson_deviance(actual, predicted)

Compute the mean Poisson deviance.

Equivalent to tweedie_deviance with power=1. Appropriate for count data.

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector (counts)
• predicted::AbstractVector{<:Real}: Predicted positive vector

Examples

actual = [1, 2, 3, 4, 5, 6]
predicted = [1.1, 1.9, 3.1, 3.9, 5.1, 5.9]
mean_poisson_deviance(actual, predicted)

mean_gamma_deviance(actual, predicted)

Compute the mean Gamma deviance.

Equivalent to tweedie_deviance with power=2. Appropriate for positive continuous targets with variance proportional to the square of the mean.

Arguments

• actual::AbstractVector{<:Real}: Ground truth positive vector
• predicted::AbstractVector{<:Real}: Predicted positive vector

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mean_gamma_deviance(actual, predicted)

d2_tweedie_score(actual, predicted; power=1.5)

Compute the D² (deviance explained) score using Tweedie deviance.

Similar to R² but uses Tweedie deviance instead of squared error. D² = 1 - deviance(actual, predicted) / deviance(actual, mean(actual))

Arguments

• actual::AbstractVector{<:Real}: Ground truth non-negative vector
• predicted::AbstractVector{<:Real}: Predicted non-negative vector
• power::Real: Tweedie power parameter (default: 1.5)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
d2_tweedie_score(actual, predicted, power=1.5)

accuracy(actual, predicted)

Compute the classification accuracy (proportion of correctly classified observations).

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = ['a', 'a', 'c', 'b', 'c']
predicted = ['a', 'b', 'c', 'b', 'a']
accuracy(actual, predicted)

ce(actual, predicted)

Compute the classification error (proportion of misclassified observations).

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = ['a', 'a', 'c', 'b', 'c']
predicted = ['a', 'b', 'c', 'b', 'a']
ce(actual, predicted)

balanced_accuracy(actual, predicted)

Compute balanced accuracy, which accounts for imbalanced datasets.

Balanced accuracy is the macro-averaged recall: the average of recall scores for each class. It gives equal weight to each class regardless of its frequency.

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = [1, 1, 1, 1, 0, 0]  # Imbalanced: 4 positives, 2 negatives
predicted = [1, 1, 1, 0, 0, 0]
balanced_accuracy(actual, predicted)  # (0.75 + 1.0) / 2 = 0.875

cohens_kappa(actual, predicted)

Compute Cohen's Kappa coefficient for inter-rater agreement.

Kappa measures agreement between two raters, accounting for agreement by chance.

• κ = 1: Perfect agreement
• κ = 0: Agreement equivalent to chance
• κ < 0: Less agreement than chance

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = ['a', 'a', 'b', 'b', 'c', 'c']
predicted = ['a', 'b', 'b', 'b', 'c', 'a']
cohens_kappa(actual, predicted)

ScoreQuadraticWeightedKappa(rater_a, rater_b; min_rating=nothing, max_rating=nothing)

Compute the quadratic weighted kappa between two vectors of integer ratings.

Arguments

• rater_a::AbstractVector{<:Integer}: First rater's ratings
• rater_b::AbstractVector{<:Integer}: Second rater's ratings
• min_rating::Union{Integer,Nothing}: Minimum possible rating (default: minimum of both vectors)
• max_rating::Union{Integer,Nothing}: Maximum possible rating (default: maximum of both vectors)

Examples

rater_a = [1, 4, 5, 5, 2, 1]
rater_b = [2, 2, 4, 5, 3, 3]
ScoreQuadraticWeightedKappa(rater_a, rater_b, min_rating=1, max_rating=5)

MeanQuadraticWeightedKappa(kappas; weights=nothing)

Compute the mean quadratic weighted kappa, optionally weighted.

Uses Fisher's z-transformation for averaging.

Arguments

• kappas::AbstractVector{<:Real}: Vector of kappa values
• weights::Union{AbstractVector{<:Real},Nothing}: Optional weights (default: equal weights)

Examples

kappas = [0.3, 0.2, 0.2, 0.5, 0.1, 0.2]
weights = [1.0, 2.5, 1.0, 1.0, 2.0, 3.0]
MeanQuadraticWeightedKappa(kappas, weights=weights)

matthews_corrcoef(actual, predicted)

Compute the Matthews Correlation Coefficient (MCC) for binary classification.

MCC is considered one of the best metrics for binary classification, especially for imbalanced datasets. It returns a value in [-1, 1]:

• +1: Perfect prediction
• 0: Random prediction
• -1: Total disagreement

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
matthews_corrcoef(actual, predicted)

mcc(actual, predicted)

Alias for matthews_corrcoef. Compute the Matthews Correlation Coefficient.

confusion_matrix(actual, predicted)

Compute the confusion matrix for classification.

Returns a dictionary with keys:

• :matrix - The confusion matrix as a 2D array
• :labels - The class labels in order

For binary classification with classes 0 and 1:

• matrix[1,1] = TN (true negatives)
• matrix[1,2] = FP (false positives)
• matrix[2,1] = FN (false negatives)
• matrix[2,2] = TP (true positives)

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
cm = confusion_matrix(actual, predicted)
cm[:matrix]  # 2x2 confusion matrix
cm[:labels]  # [0, 1]

top_k_accuracy(actual, predicted_probs, k)

Compute top-k accuracy for multi-class classification.

Returns the fraction of samples where the true class is among the top k predictions.

Arguments

• actual::AbstractVector{<:Integer}: Ground truth class indices (1-indexed)
• predicted_probs::AbstractMatrix{<:Real}: Matrix of predicted probabilities (samples × classes)
• k::Integer: Number of top predictions to consider

Examples

actual = [1, 2, 3, 1]
predicted_probs = [0.8 0.1 0.1;   # Sample 1: class 1 most likely
                   0.2 0.5 0.3;   # Sample 2: class 2 most likely
                   0.1 0.3 0.6;   # Sample 3: class 3 most likely
                   0.3 0.4 0.3]   # Sample 4: class 2 most likely (actual is 1)
top_k_accuracy(actual, predicted_probs, 1)  # Standard accuracy
top_k_accuracy(actual, predicted_probs, 2)  # Top-2 accuracy

hamming_loss(actual, predicted)

Compute the Hamming loss (fraction of misclassified labels).

For single-label classification, this equals the classification error (1 - accuracy). For multi-label classification, it measures the fraction of incorrect labels.

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = ['a', 'a', 'b', 'b', 'c', 'c']
predicted = ['a', 'b', 'b', 'b', 'c', 'a']
hamming_loss(actual, predicted)

hamming_loss(actual, predicted)

Compute Hamming loss for multi-label classification.

Arguments

• actual::AbstractMatrix{Bool}: Ground truth binary matrix (samples × labels)
• predicted::AbstractMatrix{Bool}: Predicted binary matrix (samples × labels)

zero_one_loss(actual, predicted)

Compute the zero-one loss (fraction of samples with any incorrect prediction).

For single-label classification, this equals the classification error.

Arguments

• actual::AbstractVector: Ground truth vector
• predicted::AbstractVector: Predicted vector

Examples

actual = ['a', 'a', 'b', 'b']
predicted = ['a', 'b', 'b', 'b']
zero_one_loss(actual, predicted)

hinge_loss(actual, predicted)

Compute the hinge loss for binary classification (used in SVMs).

Actual values should be -1 or 1. Predicted values are the decision function values (not probabilities).

Arguments

• actual::AbstractVector{<:Real}: Ground truth (-1 or 1)
• predicted::AbstractVector{<:Real}: Decision function values (raw scores)

Examples

actual = [1, 1, -1, -1]
predicted = [0.8, 0.3, -0.5, 0.1]  # Decision function values
hinge_loss(actual, predicted)

squared_hinge_loss(actual, predicted)

Compute the squared hinge loss (used in some SVMs).

Arguments

• actual::AbstractVector{<:Real}: Ground truth (-1 or 1)
• predicted::AbstractVector{<:Real}: Decision function values (raw scores)

Examples

actual = [1, 1, -1, -1]
predicted = [0.8, 0.3, -0.5, 0.1]
squared_hinge_loss(actual, predicted)

auc(actual, predicted)

Compute the area under the ROC curve (AUC).

Uses the Mann-Whitney U statistic for fast computation without building the ROC curve.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted scores (higher = more likely positive)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
auc(actual, predicted)

gini_coefficient(actual, predicted)

Compute the Gini coefficient from AUC.

Gini = 2 * AUC - 1

The Gini coefficient ranges from -1 to 1:

• 1: Perfect prediction
• 0: Random prediction
• -1: Perfectly wrong prediction

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted scores (higher = more likely positive)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
gini_coefficient(actual, predicted)

ks_statistic(actual, predicted)

Compute the Kolmogorov-Smirnov statistic for binary classification.

KS statistic is the maximum difference between the cumulative distribution functions of positive and negative classes. Higher values indicate better discrimination.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted scores (higher = more likely positive)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
ks_statistic(actual, predicted)

ll(actual, predicted)

Compute the elementwise log loss (cross-entropy loss).

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted probabilities for positive class

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
ll(actual, predicted)

logloss(actual, predicted)

Compute the mean log loss (cross-entropy loss).

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted probabilities for positive class

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
logloss(actual, predicted)

brier_score(actual, predicted)

Compute the Brier score for probability predictions.

The Brier score is the mean squared error of predicted probabilities compared to binary outcomes. Lower is better (0 is perfect, 1 is worst).

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted probabilities for positive class [0, 1]

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [0.9, 0.8, 0.4, 0.5, 0.3, 0.2]
brier_score(actual, predicted)

precision(actual, predicted)

Compute precision (positive predictive value).

Proportion of positive predictions that are actually positive.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 1, 1, 1, 1, 1]
precision(actual, predicted)

recall(actual, predicted)

Compute recall (sensitivity, true positive rate).

Proportion of actual positives that are correctly predicted.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 1, 1]
recall(actual, predicted)

sensitivity(actual, predicted)

Alias for recall. Compute sensitivity (true positive rate).

Proportion of actual positives that are correctly predicted.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

specificity(actual, predicted)

Compute specificity (true negative rate, selectivity).

Proportion of actual negatives that are correctly predicted.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
specificity(actual, predicted)  # 2/3 = 0.667

npv(actual, predicted)

Compute the Negative Predictive Value.

Proportion of negative predictions that are actually negative.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
npv(actual, predicted)  # 2/3 = 0.667 (of negative predictions, 2 of 3 are correct)

fbeta_score(actual, predicted; beta=1.0)

Compute the F-beta score, a weighted harmonic mean of precision and recall.

When beta=1, this is the F1 score (equal weight to precision and recall). When beta<1, precision is weighted more heavily. When beta>1, recall is weighted more heavily.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)
• beta::Real: Weight parameter (default: 1.0)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 1, 1]
fbeta_score(actual, predicted)  # F1 score
fbeta_score(actual, predicted, beta=0.5)  # F0.5 score (precision-weighted)
fbeta_score(actual, predicted, beta=2.0)  # F2 score (recall-weighted)

fpr(actual, predicted)

Compute the False Positive Rate (fall-out, 1 - specificity).

Proportion of actual negatives that are incorrectly predicted as positive.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
fpr(actual, predicted)  # 1/3 = 0.333

fnr(actual, predicted)

Compute the False Negative Rate (miss rate, 1 - recall).

Proportion of actual positives that are incorrectly predicted as negative.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
fnr(actual, predicted)  # 1/3 = 0.333

youden_j(actual, predicted)

Compute Youden's J statistic (informedness).

J = sensitivity + specificity - 1 = TPR - FPR

Ranges from -1 to 1, where 1 indicates perfect prediction.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
youden_j(actual, predicted)

markedness(actual, predicted)

Compute markedness (deltaP, Δp).

Markedness = PPV + NPV - 1 = precision + NPV - 1

The counterpart to informedness (Youden's J) in the prediction direction.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
markedness(actual, predicted)

fowlkes_mallows_index(actual, predicted)

Compute the Fowlkes-Mallows index.

FM = sqrt(PPV × TPR) = sqrt(precision × recall)

Geometric mean of precision and recall, ranges from 0 to 1.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
fowlkes_mallows_index(actual, predicted)

positive_likelihood_ratio(actual, predicted)

Compute the Positive Likelihood Ratio (LR+).

LR+ = TPR / FPR = sensitivity / (1 - specificity)

Indicates how much more likely a positive prediction is for actual positives. Higher values are better.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
positive_likelihood_ratio(actual, predicted)

negative_likelihood_ratio(actual, predicted)

Compute the Negative Likelihood Ratio (LR-).

LR- = FNR / TNR = (1 - sensitivity) / specificity

Indicates how much more likely a negative prediction is for actual positives. Lower values are better.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
negative_likelihood_ratio(actual, predicted)

diagnostic_odds_ratio(actual, predicted)

Compute the Diagnostic Odds Ratio (DOR).

DOR = LR+ / LR- = (TP × TN) / (FP × FN)

The ratio of the odds of a positive prediction in actual positives to the odds in actual negatives. Higher values indicate better discrimination.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Binary predictions (1 for positive, 0 for negative)

Examples

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 0, 1, 1, 0, 0]
diagnostic_odds_ratio(actual, predicted)

lift(actual, predicted; percentile=0.1)

Compute the lift at a given percentile.

Lift measures how much better the model is at identifying positives compared to random selection. Lift = (% positives in top X%) / (% positives overall).

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted scores (higher = more likely positive)
• percentile::Real: Top fraction to consider (default: 0.1 = top 10%)

Examples

actual = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
predicted = [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.0]
lift(actual, predicted, percentile=0.3)  # Lift in top 30%

gain(actual, predicted; percentile=0.1)

Compute the cumulative gain at a given percentile.

Gain measures what percentage of total positives are captured in the top X%.

Arguments

• actual::AbstractVector{<:Real}: Binary ground truth (1 for positive, 0 for negative)
• predicted::AbstractVector{<:Real}: Predicted scores (higher = more likely positive)
• percentile::Real: Top fraction to consider (default: 0.1 = top 10%)

Examples

actual = [1, 1, 1, 0, 0, 0, 0, 0, 0, 0]
predicted = [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.0]
gain(actual, predicted, percentile=0.3)  # What % of positives in top 30%

f1(actual, predicted)

Compute the F1 score in the context of information retrieval.

Computes 2 * precision * recall / (precision + recall) where precision is the proportion of retrieved documents that are relevant, and recall is the proportion of relevant documents that are retrieved.

Returns 0 if there are no true positives.

Arguments

• actual::AbstractVector: Ground truth relevant documents (order doesn't matter)
• predicted::AbstractVector: Retrieved documents (order doesn't matter)

Examples

actual = ['a', 'c', 'd']
predicted = ['d', 'e']
f1(actual, predicted)

precision_at_k(actual, predicted; k=10)

Compute precision@k for information retrieval.

The fraction of top k predictions that are relevant.

Arguments

• actual::AbstractVector: Ground truth relevant items
• predicted::AbstractVector: Ranked predictions (highest rank first)
• k::Integer: Number of top predictions to consider (default: 10)

Examples

actual = ["a", "b", "c", "d"]
predicted = ["a", "x", "b", "y", "z"]
precision_at_k(actual, predicted, k=3)  # 2/3 (2 of top 3 are relevant)

recall_at_k(actual, predicted; k=10)

Compute recall@k for information retrieval.

The fraction of relevant items that appear in the top k predictions.

Arguments

• actual::AbstractVector: Ground truth relevant items
• predicted::AbstractVector: Ranked predictions (highest rank first)
• k::Integer: Number of top predictions to consider (default: 10)

Examples

actual = ["a", "b", "c", "d"]
predicted = ["a", "x", "b", "y", "z"]
recall_at_k(actual, predicted, k=3)  # 2/4 = 0.5 (found "a" and "b" in top 3)

f1_at_k(actual, predicted; k=10)

Compute F1@k for information retrieval.

Harmonic mean of precision@k and recall@k.

Arguments

• actual::AbstractVector: Ground truth relevant items
• predicted::AbstractVector: Ranked predictions (highest rank first)
• k::Integer: Number of top predictions to consider (default: 10)

Examples

actual = ["a", "b", "c", "d"]
predicted = ["a", "x", "b", "y", "z"]
f1_at_k(actual, predicted, k=3)

dcg(relevance; k=nothing)

Compute the Discounted Cumulative Gain at position k.

DCG measures the usefulness of a ranking based on relevance scores, with a logarithmic discount to penalize relevant items appearing lower in the ranking.

DCG = Σ (2^rel_i - 1) / log2(i + 1)

Arguments

• relevance::AbstractVector{<:Real}: Relevance scores in ranked order (highest rank first)
• k::Union{Integer,Nothing}: Number of positions to consider (default: all)

Examples

relevance = [3, 2, 3, 0, 1, 2]  # Relevance scores for ranked items
dcg(relevance)  # DCG for all positions
dcg(relevance, k=3)  # DCG@3

idcg(relevance; k=nothing)

Compute the Ideal Discounted Cumulative Gain at position k.

IDCG is the DCG of the best possible ranking (relevance scores sorted descending).

Arguments

• relevance::AbstractVector{<:Real}: Relevance scores (order doesn't matter)
• k::Union{Integer,Nothing}: Number of positions to consider (default: all)

Examples

relevance = [3, 2, 3, 0, 1, 2]
idcg(relevance)  # Ideal DCG for all positions
idcg(relevance, k=3)  # Ideal DCG@3

ndcg(relevance; k=nothing)

Compute the Normalized Discounted Cumulative Gain at position k.

NDCG = DCG / IDCG

Normalizes DCG to [0, 1] by dividing by the ideal DCG.

Arguments

• relevance::AbstractVector{<:Real}: Relevance scores in ranked order (highest rank first)
• k::Union{Integer,Nothing}: Number of positions to consider (default: all)

Examples

relevance = [3, 2, 3, 0, 1, 2]  # Actual ranking
ndcg(relevance)  # NDCG for all positions
ndcg(relevance, k=3)  # NDCG@3

mean_ndcg(relevances; k=nothing)

Compute the mean NDCG over multiple queries.

Arguments

• relevances::AbstractVector{<:AbstractVector{<:Real}}: List of relevance score vectors
• k::Union{Integer,Nothing}: Number of positions to consider (default: all)

Examples

relevances = [[3, 2, 1, 0], [2, 1, 2, 1], [1, 1, 0, 0]]
mean_ndcg(relevances)  # Mean NDCG across queries
mean_ndcg(relevances, k=2)  # Mean NDCG@2

apk(k, actual, predicted)

Compute the average precision at k.

Loops over the first k values of predicted. For each value that is in actual and hasn't been predicted before, increments the score by (number of hits so far) / position. Returns the final score divided by min(length(actual), k).

Returns NaN if actual is empty.

Arguments

• k::Integer: Number of predictions to consider
• actual::AbstractVector: Ground truth relevant documents (order doesn't matter)
• predicted::AbstractVector: Retrieved documents in ranked order (most relevant first)

Examples

actual = ['a', 'b', 'd']
predicted = ['b', 'c', 'a', 'e', 'f']
apk(3, actual, predicted)

mapk(k, actual, predicted)

Compute the mean average precision at k.

Evaluates apk for each pair of elements from actual and predicted lists, then returns the mean.

Returns NaN if actual or predicted is empty.

Arguments

• k::Integer: Number of predictions to consider for each query
• actual::AbstractVector{<:AbstractVector}: List of ground truth vectors
• predicted::AbstractVector{<:AbstractVector}: List of prediction vectors

Examples

actual = [['a', 'b'], ['a'], ['x', 'y', 'b']]
predicted = [['a', 'c', 'd'], ['x', 'b', 'a', 'b'], ['y']]
mapk(2, actual, predicted)

reciprocal_rank(actual, predicted)

Compute the Reciprocal Rank for a single query.

Arguments

• actual::AbstractVector: Ground truth relevant items
• predicted::AbstractVector: Ranked predictions (highest rank first)

Examples

actual = ["a", "b"]
predicted = ["c", "a", "b", "d"]
reciprocal_rank(actual, predicted)  # 1/2 = 0.5 (first relevant at position 2)

mrr(actual, predicted)

Compute the Mean Reciprocal Rank.

MRR is the average of reciprocal ranks of the first relevant item for each query.

Arguments

• actual::AbstractVector{<:AbstractVector}: List of ground truth relevant items for each query
• predicted::AbstractVector{<:AbstractVector}: List of ranked predictions for each query

Examples

actual = [["a", "b"], ["c"], ["d", "e"]]
predicted = [["b", "a", "c"], ["a", "c", "d"], ["e", "d", "f"]]
mrr(actual, predicted)  # (1/2 + 1/2 + 1/1) / 3 = 0.667

hit_rate(actual, predicted; k=10)

Compute the hit rate (recall@k) for recommendation systems.

Hit rate is the fraction of queries where at least one relevant item appears in the top k predictions.

Returns NaN if actual is empty.

Arguments

• actual::AbstractVector{<:AbstractVector}: List of ground truth relevant items for each query
• predicted::AbstractVector{<:AbstractVector}: List of ranked predictions for each query
• k::Integer: Number of top predictions to consider (default: 10)

Examples

actual = [["a", "b"], ["c"], ["d", "e"]]
predicted = [["a", "x", "y"], ["x", "y", "z"], ["e", "f", "g"]]
hit_rate(actual, predicted, k=3)  # 2/3 queries have a hit in top 3

coverage(predicted, catalog)

Compute the catalog coverage of recommendations.

Coverage measures what fraction of items in the catalog have been recommended at least once across all predictions.

Arguments

• predicted::AbstractVector{<:AbstractVector}: List of predictions for all queries
• catalog::AbstractVector: Full catalog of items

Examples

catalog = ["a", "b", "c", "d", "e", "f"]
predicted = [["a", "b"], ["a", "c"], ["b", "d"]]
coverage(predicted, catalog)  # 4/6 = 0.667 (recommended: a, b, c, d)

novelty(predicted, item_popularity)

Compute the novelty of recommendations.

Novelty measures how unexpected/surprising the recommendations are, based on the popularity of recommended items. Higher novelty means recommending less popular (long-tail) items.

Arguments

• predicted::AbstractVector{<:AbstractVector}: List of predictions for all queries
• item_popularity::Dict: Dictionary mapping items to their popularity (0-1)

Examples

popularity = Dict("a" => 0.9, "b" => 0.5, "c" => 0.1, "d" => 0.05)
predicted = [["a", "b"], ["c", "d"]]
novelty(predicted, popularity)  # Average -log2(popularity)

mase(actual, predicted; m=1)

Compute the Mean Absolute Scaled Error for time series data.

MASE compares the prediction error to the error of a naive forecast. The naive forecast predicts the value from m periods ago (seasonal naive for m > 1).

MASE < 1 indicates the model outperforms the naive forecast. MASE = 1 indicates performance equal to naive forecast. MASE > 1 indicates the model underperforms the naive forecast.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series (ordered by time)
• predicted::AbstractVector{<:Real}: Predicted time series
• m::Integer: Seasonal period / frequency (default: 1)
  • m=1: Non-seasonal naive forecast (random walk)
  • m=4: Quarterly seasonality (compare with same quarter last year)
  • m=7: Weekly seasonality for daily data
  • m=12: Monthly seasonality (compare with same month last year)
  • m=52: Weekly seasonality for weekly data

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
mase(actual, predicted)  # Non-seasonal (m=1)
mase(actual, predicted, m=2)  # With seasonal period 2

msse(actual, predicted; m=1)

Compute the Mean Squared Scaled Error for time series data.

Similar to MASE but uses squared errors, making it more sensitive to large errors.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series (ordered by time)
• predicted::AbstractVector{<:Real}: Predicted time series
• m::Integer: Seasonal period / frequency (default: 1)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
msse(actual, predicted)

rmsse(actual, predicted; m=1)

Compute the Root Mean Squared Scaled Error for time series data.

Square root of MSSE, on the same scale as the original data.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series (ordered by time)
• predicted::AbstractVector{<:Real}: Predicted time series
• m::Integer: Seasonal period / frequency (default: 1)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
rmsse(actual, predicted)

tracking_signal(actual, predicted)

Compute the tracking signal for forecast monitoring.

Tracking signal = Cumulative Forecast Error / MAD

Used to detect forecast bias. Values outside [-4, 4] typically indicate bias.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series

Examples

actual = [100, 110, 105, 115, 120]
predicted = [98, 108, 110, 112, 125]
tracking_signal(actual, predicted)

forecast_bias(actual, predicted)

Compute the forecast bias (cumulative forecast error normalized by number of periods).

Positive bias indicates systematic under-forecasting. Negative bias indicates systematic over-forecasting.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series

Examples

actual = [100, 110, 105, 115, 120]
predicted = [98, 108, 110, 112, 125]
forecast_bias(actual, predicted)

theil_u1(actual, predicted)

Compute Theil's U1 statistic (inequality coefficient).

U1 ranges from 0 to 1:

• 0: Perfect forecast
• 1: Worst possible forecast

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
theil_u1(actual, predicted)

theil_u2(actual, predicted; m=1)

Compute Theil's U2 statistic (compares to naive forecast).

U2 < 1: Model outperforms naive forecast U2 = 1: Model equals naive forecast U2 > 1: Model underperforms naive forecast

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series
• m::Integer: Seasonal period for naive forecast (default: 1)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2]
theil_u2(actual, predicted)

wape(actual, predicted)

Compute the Weighted Absolute Percentage Error for time series.

WAPE = Σ|actual - predicted| / Σ|actual|

Unlike MAPE, WAPE is well-defined when actual values are zero and gives more weight to larger values.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series

Examples

actual = [100, 200, 150, 300, 250]
predicted = [110, 190, 160, 290, 260]
wape(actual, predicted)

directional_accuracy(actual, predicted)

Compute the directional accuracy (hit rate for direction of change).

Measures how often the forecast correctly predicts whether the value goes up or down.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series

Examples

actual = [100, 110, 105, 115, 120]  # Changes: +10, -5, +10, +5
predicted = [100, 108, 106, 112, 118]  # Changes: +8, -2, +6, +6
directional_accuracy(actual, predicted)  # All directions match = 1.0

coverage_probability(actual, lower, upper)

Compute the coverage probability of prediction intervals.

Measures what fraction of actual values fall within the prediction intervals.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• lower::AbstractVector{<:Real}: Lower bounds of prediction intervals
• upper::AbstractVector{<:Real}: Upper bounds of prediction intervals

Examples

actual = [100, 110, 105, 115, 120]
lower = [95, 105, 100, 108, 112]  # 95% lower bounds
upper = [105, 115, 112, 122, 128]  # 95% upper bounds
coverage_probability(actual, lower, upper)  # Should be ≈ 0.95 if well-calibrated

pinball_loss_series(actual, predicted; quantile=0.5)

Compute the pinball (quantile) loss for time series probabilistic forecasts.

Same as quantile_loss but named to be consistent with time series literature.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Quantile forecast
• quantile::Real: Target quantile in (0, 1) (default: 0.5 = median)

Examples

actual = [100, 110, 105, 115, 120]
predicted_median = [98, 108, 110, 112, 118]  # Median forecasts
pinball_loss_series(actual, predicted_median, quantile=0.5)

winkler_score(actual, lower, upper; alpha=0.05)

Compute the Winkler score for prediction interval evaluation.

The Winkler score rewards narrow intervals and penalizes intervals that don't contain the actual value.

Lower scores are better.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• lower::AbstractVector{<:Real}: Lower bounds of prediction intervals
• upper::AbstractVector{<:Real}: Upper bounds of prediction intervals
• alpha::Real: Significance level (default: 0.05 for 95% intervals)

Examples

actual = [100, 110, 105, 115, 120]
lower = [95, 105, 100, 108, 112]
upper = [105, 115, 112, 122, 128]
winkler_score(actual, lower, upper, alpha=0.05)

autocorrelation_error(actual, predicted; max_lag=10)

Compute the error in autocorrelation structure.

Measures how well the forecast preserves the autocorrelation structure of the actual series. Lower values indicate better preservation of temporal patterns.

Arguments

• actual::AbstractVector{<:Real}: Ground truth time series
• predicted::AbstractVector{<:Real}: Predicted time series
• max_lag::Integer: Maximum lag to consider (default: 10)

Examples

actual = [1.1, 1.9, 3.0, 4.4, 5.0, 5.6, 6.2, 7.1, 8.0, 9.2]
predicted = [0.9, 1.8, 2.5, 4.5, 5.0, 6.2, 6.0, 7.0, 8.1, 9.0]
autocorrelation_error(actual, predicted)