Results

Result containers that store the outputs of a nested cross-validation run, including per-fold metrics, predictions, and fitted models.

Classifier results

class nestkit.results.ClassifierOuterFoldResult(fold_idx, train_indices, test_indices, best_params, best_inner_score, inner_cv_results, fit_time, score_time, fitted_estimator, y_true, y_proba_raw, y_pred_default, outer_scores_default=<factory>, confusion_matrix_default=<factory>, y_proba_calibrated=None, calibration_method=None, calibrator=None, oof_calibration_diagnostics=None, y_pred_optimized=None, outer_scores_optimized=None, confusion_matrix_optimized=None, threshold_result=None, conformal_result=None, conformal_prediction_sets=None, conformal_set_sizes=None, conformal_coverage=None)

Bases: object

Result of a single outer fold evaluation (classification).

Parameters:

• fold_idx (int)

• train_indices (ndarray)

• test_indices (ndarray)

• best_params (dict)

• best_inner_score (float)

• inner_cv_results (dict)

• fit_time (float)

• score_time (float)

• fitted_estimator (BaseEstimator | None)

• y_true (ndarray)

• y_proba_raw (ndarray)

• y_pred_default (ndarray)

• outer_scores_default (dict)

• confusion_matrix_default (ndarray)

• y_proba_calibrated (ndarray | None)

• calibration_method (str | None)

• calibrator (Any | None)

• oof_calibration_diagnostics (dict | None)

• y_pred_optimized (ndarray | None)

• outer_scores_optimized (dict | None)

• confusion_matrix_optimized (ndarray | None)

• threshold_result (ThresholdResult | None)

• conformal_result (ClassifierConformalResult | None)

• conformal_prediction_sets (list[list[int]] | None)

• conformal_set_sizes (ndarray | None)

• conformal_coverage (float | None)

fold_idx: int

train_indices: ndarray

test_indices: ndarray

best_params: dict

best_inner_score: float

inner_cv_results: dict

fit_time: float

score_time: float

fitted_estimator: BaseEstimator | None

y_true: ndarray

y_proba_raw: ndarray

y_pred_default: ndarray

outer_scores_default: dict

confusion_matrix_default: ndarray

y_proba_calibrated: ndarray | None = None

calibration_method: str | None = None

calibrator: Any | None = None

oof_calibration_diagnostics: dict | None = None

y_pred_optimized: ndarray | None = None

outer_scores_optimized: dict | None = None

confusion_matrix_optimized: ndarray | None = None

threshold_result: ThresholdResult | None = None

conformal_result: ClassifierConformalResult | None = None

conformal_prediction_sets: list[list[int]] | None = None

conformal_set_sizes: ndarray | None = None

conformal_coverage: float | None = None

class nestkit.ClassifierResults(n_outer_folds, feature_names=None, original_index=None)

Bases: _BaseNestedCVResults

Aggregated nested CV results for classification.

Parameters:

• n_outer_folds (int)

• feature_names (list[str] | None)

• original_index (Any | None)

property has_calibration: bool

property has_threshold_optimization: bool

property has_conformal: bool

finalize()

Compute aggregate statistics after all folds are added.

Must be called exactly once, after all n_outer_folds fold results have been added via add_fold(). Subsequent calls are no-ops (guarded by an internal _finalized flag).

Return type:

None

threshold_comparison()

Side-by-side comparison of default vs optimized threshold performance.

Return type:

DataFrame

calibration_report()

Detailed calibration diagnostics per fold.

Return type:

DataFrame

conformal_report()

Per-fold conformal coverage and set size statistics.

Returns:

One row per outer fold with coverage, mean set size, fraction of singleton and empty prediction sets.

Return type:

pd.DataFrame

Raises:

ValueError – If conformal prediction was not enabled.

classification_report_pooled(threshold='default')

sklearn-style classification report on pooled OOF predictions.

Parameters:

threshold (str)

Return type:

str

Regressor results

class nestkit.results.RegressorOuterFoldResult(fold_idx, train_indices, test_indices, best_params, best_inner_score, inner_cv_results, fit_time, score_time, fitted_estimator, y_true, y_pred, outer_scores=<factory>, residuals=<factory>, prediction_interval_lower=None, prediction_interval_upper=None, coverage=None, mondrian_bin_assignments=None)

Bases: object

Result of a single outer fold evaluation (regression).

Parameters:

• fold_idx (int)

• train_indices (ndarray)

• test_indices (ndarray)

• best_params (dict)

• best_inner_score (float)

• inner_cv_results (dict)

• fit_time (float)

• score_time (float)

• fitted_estimator (BaseEstimator | None)

• y_true (ndarray)

• y_pred (ndarray)

• outer_scores (dict)

• residuals (ndarray)

• prediction_interval_lower (ndarray | None)

• prediction_interval_upper (ndarray | None)

• coverage (float | None)

• mondrian_bin_assignments (ndarray | None)

fold_idx: int

train_indices: ndarray

test_indices: ndarray

best_params: dict

best_inner_score: float

inner_cv_results: dict

fit_time: float

score_time: float

fitted_estimator: BaseEstimator | None

y_true: ndarray

y_pred: ndarray

outer_scores: dict

residuals: ndarray

prediction_interval_lower: ndarray | None = None

prediction_interval_upper: ndarray | None = None

coverage: float | None = None

mondrian_bin_assignments: ndarray | None = None

class nestkit.RegressorResults(n_outer_folds, feature_names=None, original_index=None)

Bases: _BaseNestedCVResults

Aggregated nested CV results for regression.

Parameters:

• n_outer_folds (int)

• feature_names (list[str] | None)

• original_index (Any | None)

finalize()

Compute aggregate statistics after all folds are added.

Must be called exactly once, after all n_outer_folds fold results have been added via add_fold(). Subsequent calls are no-ops (guarded by an internal _finalized flag).

Return type:

None