scoring_utils

safe_roc_auc_score

safe_roc_auc_score(y_true, y_score, **kwargs)

Compute the Area Under the Receiver Operating Characteristic Curve (ROC AUC) score.

This function is a safe wrapper around sklearn.metrics.roc_auc_score that handles cases where the input data may have missing classes or binary classification problems.

Parameters:

Name	Type	Description	Default
y_true		array-like of shape (n_samples,) True binary labels or binary label indicators.	required
y_score		array-like of shape (n_samples,) or (n_samples, n_classes) Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions.	required
**kwargs		dict Additional keyword arguments to pass to sklearn.metrics.roc_auc_score.	{}

Returns:

Name	Type	Description
float		The ROC AUC score.

Raises:

Type	Description
ValueError	If there are missing classes in y_true that cannot be handled.

score_classification

score_classification(
    optimize_metric: Literal[
        "roc", "auroc", "accuracy", "f1", "log_loss"
    ],
    y_true,
    y_pred,
    sample_weight=None,
    *,
    y_pred_is_labels: bool = False
)

General function to score classification predictions.

Parameters:

Name	Type	Description	Default
optimize_metric		{"roc", "auroc", "accuracy", "f1", "log_loss"} The metric to use for scoring the predictions.	required
y_true		array-like of shape (n_samples,) True labels or binary label indicators.	required
y_pred		array-like of shape (n_samples,) or (n_samples, n_classes) Predicted labels, probabilities, or confidence values.	required
sample_weight		array-like of shape (n_samples,), default=None Sample weights.	None

Returns:

Name	Type	Description
float		The score for the specified metric.

Raises:

Type	Description
ValueError	If an unknown metric is specified.

score_regression

score_regression(
    optimize_metric: Literal["rmse", "mse", "mae"],
    y_true,
    y_pred,
    sample_weight=None,
)

General function to score regression predictions.

Parameters:

Name	Type	Description	Default
optimize_metric		{"rmse", "mse", "mae"} The metric to use for scoring the predictions.	required
y_true		array-like of shape (n_samples,) True target values.	required
y_pred		array-like of shape (n_samples,) Predicted target values.	required
sample_weight		array-like of shape (n_samples,), default=None Sample weights.	None

Returns:

Name	Type	Description
float		The score for the specified metric.

Raises:

Type	Description
ValueError	If an unknown metric is specified.

score_survival

score_survival(
    optimize_metric: Literal["cindex"],
    y_true,
    y_pred,
    event_observed,
    sample_weight=None,
)

General function to score regression predictions.

Parameters:

Name	Type	Description	Default
optimize_metric		{"rmse", "mse", "mae"} The metric to use for scoring the predictions.	required
y_true		array-like of shape (n_samples,) True target values.	required
y_pred		array-like of shape (n_samples,) Predicted target values.	required
sample_weight		array-like of shape (n_samples,), default=None Sample weights.	None

Returns:

Name	Type	Description
float		The score for the specified metric.

Raises:

Type	Description
ValueError	If an unknown metric is specified.