Plotting#

Visualization functions for inspecting nested cross-validation results. All functions return a Matplotlib Axes and can be imported directly from nestkit.plotting.

Fold-level plots#

nestkit.plotting.folds.plot_outer_scores(results, metric, point_alpha=0.7, ylim=None, ax=None, **kwargs)[source]#

Box-and-strip plot of outer fold scores.

Parameters:

results (ClassifierResults or RegressorResults) – Fitted nested CV results object.
metric (str) – Name of the scoring metric to plot.
point_alpha (float, optional) – Opacity of individual fold score markers.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib boxplot call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.folds.plot_inner_cv_heatmap(results, cmap='YlOrRd', annot=True, fmt='.3f', ax=None, **kwargs)[source]#

Heatmap of mean inner CV scores across hyperparameter grid per outer fold.

Each row is an outer fold, each column is a hyperparameter configuration from the inner search grid. Color intensity reflects the inner CV score. This reveals whether the optimal region of the hyperparameter space is consistent across outer folds.

Parameters:

results (ClassifierResults or RegressorResults) – Fitted nested CV results object with inner_reports_ attribute.
cmap (str, optional) – Matplotlib colormap name.
annot (bool, optional) – Whether to annotate cells with score values.
fmt (str, optional) – Format string for annotations.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to matplotlib.axes.Axes.imshow.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.folds.plot_score_stability(results, metrics=None, point_alpha=0.7, xlim=None, ax=None, **kwargs)[source]#

Strip plot of multiple metrics across outer folds.

Shows the spread of each metric’s outer fold scores on a single axis, giving a holistic view of result stability. Each metric is a row with individual fold scores plotted as points.

Parameters:

results (ClassifierResults or RegressorResults) – Fitted nested CV results object.
metrics (list of str or None, optional) – Metric names to include. If None, all metrics from outer_scores_default_ are used.
point_alpha (float, optional) – Opacity of individual fold score markers.
xlim (tuple of float or None, optional) – Explicit x-axis limits (score axis).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib scatter call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Inner tuning plots#

nestkit.plotting.tuning.plot_inner_tuning_curve(inner_report, param, metric=None, band_alpha=0.2, ylim=None, ax=None, **kwargs)[source]#

Score vs hyperparameter curve for a single inner CV report.

Parameters:

inner_report (InnerReport) – A single inner CV report (e.g., results.inner_reports_[i]).
param (str) – Name of the hyperparameter to place on the x-axis.
metric (str or None, optional) – Scoring metric to use. If None, the default metric from the inner report is used.
band_alpha (float, optional) – Opacity of the +/- 1 std band.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib plot call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.tuning.plot_param_selection(results, param, bar_color=None, ax=None, **kwargs)[source]#

Frequency of each hyperparameter value selected across outer folds.

Shows a bar chart where each bar represents a unique value from the search grid, and its height is the number of outer folds that selected that value.

Parameters:

results (ClassifierResults or RegressorResults) – Fitted nested CV results object.
param (str) – Name of the hyperparameter to visualize.
bar_color (str or None, optional) – Color of the bars. None uses the default color cycle.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib bar call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Calibration plots#

nestkit.plotting.calibration.plot_calibration_curves(results, fold_idx=None, fold_alpha=0.4, full_range=False, ylim=None, xlim=None, ax=None, **kwargs)[source]#

Reliability diagrams showing raw vs calibrated probabilities per fold.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
fold_idx (int, list of int, or None, optional) – Outer fold index or indices to plot. If None (default), all folds are shown.
fold_alpha (float, optional) – Opacity of individual fold curves.
full_range (bool, optional) – If True, set both axes to [0, 1].
ylim (tuple of float or None, optional) – Explicit axis limits (override full_range).
xlim (tuple of float or None, optional) – Explicit axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.calibration.plot_calibration_improvement(results, annot=False, annot_fmt='.3f', full_range=False, ylim=None, ax=None, **kwargs)[source]#

Paired bar plot of ECE before vs after calibration per fold.

Shows raw and calibrated ECE side by side for each fold, with the gap (improvement) annotated above each pair. Mean and standard deviation of the gap are reported in the legend.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
annot (bool, optional) – If True, annotate each bar with its ECE value and display the gap above each pair.
annot_fmt (str, optional) – Format string for annotations.
full_range (bool, optional) – If True, set y-axis to [0, 1].
ylim (tuple of float or None, optional) – Explicit y-axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib bar calls.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Threshold plots#

nestkit.plotting.threshold.plot_threshold_sensitivity(results, fold_idx=0, line_alpha=0.8, full_range=False, xlim=None, ylim=None, ax=None, **kwargs)[source]#

Metrics as a function of decision threshold for a single fold.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
fold_idx (int, optional) – Index of the outer fold to visualize.
line_alpha (float, optional) – Opacity of metric curves.
full_range (bool, optional) – If True, set both axes to [0, 1].
xlim (tuple of float or None, optional) – Explicit axis limits (override full_range).
ylim (tuple of float or None, optional) – Explicit axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.threshold.plot_threshold_distribution(results, bar_alpha=0.7, bins=None, full_range=False, xlim=None, ax=None, **kwargs)[source]#

Distribution of optimal thresholds across folds.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
bar_alpha (float, optional) – Opacity of histogram bars.
bins (int or None, optional) – Number of histogram bins. None uses a heuristic.
full_range (bool, optional) – If True, set x-axis to [0, 1].
xlim (tuple of float or None, optional) – Explicit x-axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.threshold.plot_threshold_comparison(results, full_range=False, ylim=None, ax=None, **kwargs)[source]#

Default vs optimized metrics side-by-side.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
full_range (bool, optional) – If True, set y-axis to [0, 1].
ylim (tuple of float or None, optional) – Explicit y-axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Comparison plots#

nestkit.plotting.comparison.plot_comparison(comparator, metric, threshold='default', point_alpha=0.6, line_alpha=0.15, ylim=None, ax=None, **kwargs)[source]#

Paired box-and-strip plot of per-fold scores across models.

Parameters:

comparator (ModelComparator) – Fitted model comparator containing two or more result sets.
metric (str) – Name of the scoring metric to compare.
threshold ({'default', 'optimized'}, optional) – Which threshold’s scores to use.
point_alpha (float, optional) – Opacity of individual fold score markers.
line_alpha (float, optional) – Opacity of lines connecting paired observations.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.comparison.plot_score_differences(comparator, metric, model_a, model_b, threshold='default', bar_alpha=0.7, bar_color=None, ylim=None, ax=None, **kwargs)[source]#

Per-fold score differences between two models.

Parameters:

comparator (ModelComparator) – Fitted model comparator containing the two models.
metric (str) – Name of the scoring metric to compare.
model_a (str) – Names of the two models to compare.
model_b (str) – Names of the two models to compare.
threshold ({'default', 'optimized'}, optional) – Which threshold’s scores to use.
bar_alpha (float, optional) – Opacity of the bars.
bar_color (str or None, optional) – Color of the bars. None uses the default color cycle.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.comparison.plot_critical_difference(comparator, metric, threshold='default', bar_alpha=0.7, bar_color=None, ax=None, **kwargs)[source]#

Demsar critical difference diagram.

Parameters:

comparator (ModelComparator) – Fitted model comparator containing three or more result sets.
metric (str) – Name of the scoring metric to rank.
threshold ({'default', 'optimized'}, optional) – Which threshold’s scores to use.
bar_alpha (float, optional) – Opacity of the bars.
bar_color (str or None, optional) – Color of the bars. None uses the default color cycle.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.comparison.plot_bayesian_posterior(comparator, metric, model_a, model_b, rope=0.01, threshold='default', color_a='blue', color_b='red', color_rope='gray', fill_alpha=0.3, ax=None, **kwargs)[source]#

Posterior distribution of score differences with ROPE.

Parameters:

comparator (ModelComparator) – Fitted model comparator containing the two models.
metric (str) – Name of the scoring metric to compare.
model_a (str) – Names of the two models to compare.
model_b (str) – Names of the two models to compare.
rope (float, optional) – Half-width of the Region of Practical Equivalence.
threshold ({'default', 'optimized'}, optional) – Which threshold’s scores to use.
color_a (str, optional) – Fill color for the “A is better” region.
color_b (str, optional) – Fill color for the “B is better” region.
color_rope (str, optional) – Fill color for the equivalence region.
fill_alpha (float, optional) – Opacity of all filled regions.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Importance plots#

nestkit.plotting.importance.plot_importance(aggregator, top_k=20, show_folds=True, bar_alpha=0.7, fold_color='red', fold_alpha=0.5, fold_size=18, ax=None, **kwargs)[source]#

Bar plot of mean feature importance with optional per-fold jitter.

Parameters:

aggregator (FeatureImportanceAggregator) – Fitted importance aggregator with per-fold importance data.
top_k (int, optional) – Number of top features to display.
show_folds (bool, optional) – Whether to overlay per-fold importance values.
bar_alpha (float, optional) – Opacity of the bars.
fold_color (str, optional) – Color of per-fold scatter markers.
fold_alpha (float, optional) – Opacity of per-fold scatter markers.
fold_size (float, optional) – Size of per-fold scatter markers.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.importance.plot_selection_frequency(aggregator, top_k=10, bar_alpha=0.7, label_fontsize=7, full_range=False, xlim=None, ax=None, **kwargs)[source]#

Feature selection frequency across folds.

Parameters:

aggregator (FeatureImportanceAggregator) – Fitted importance aggregator with per-fold importance data.
top_k (int, optional) – The top-k threshold used to count selection frequency.
bar_alpha (float, optional) – Opacity of the bars.
label_fontsize (int, optional) – Font size for y-axis feature labels.
full_range (bool, optional) – If True, set x-axis to [0, 1].
xlim (tuple of float or None, optional) – Explicit x-axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.importance.plot_rank_stability_features(aggregator, top_k=20, cmap='YlOrRd', label_fontsize=7, ax=None, **kwargs)[source]#

Feature rank stability heatmap across folds.

Parameters:

aggregator (FeatureImportanceAggregator) – Fitted importance aggregator with per-fold rank data.
top_k (int, optional) – Number of top features to display.
cmap (str, optional) – Colormap for the heatmap.
label_fontsize (int, optional) – Font size for y-axis feature labels.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.importance.plot_shap_summary(aggregator, top_k=20, ax=None, **kwargs)[source]#

Pooled SHAP beeswarm plot.

Concatenates raw SHAP values from all outer test folds and renders a beeswarm summary plot using the shap package. Requires the aggregator to have been created with method='shap'.

Parameters:

aggregator (FeatureImportanceAggregator) – Fitted importance aggregator with raw SHAP values stored.
top_k (int, optional) – Maximum number of features to display.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

Raises:

ValueError – If raw SHAP values are not available in the aggregator.

Summary plots#

nestkit.plotting.summary.plot_roc_curves(results, fold_alpha=0.4, mean_color='b', mean_lw=2, band_alpha=0.2, full_range=False, ylim=None, xlim=None, ax=None, **kwargs)[source]#

Per-fold ROC curves with mean and confidence-interval band.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
fold_alpha (float, optional) – Opacity of individual fold curves.
mean_color (str, optional) – Color of the mean ROC curve.
mean_lw (float, optional) – Line width of the mean ROC curve.
band_alpha (float, optional) – Opacity of the +/- 1 std band.
full_range (bool, optional) – If True, set both axes to [0, 1].
ylim (tuple of float or None, optional) – Explicit axis limits (override full_range).
xlim (tuple of float or None, optional) – Explicit axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_precision_recall_curves(results, fold_alpha=0.5, full_range=False, ylim=None, xlim=None, ax=None, **kwargs)[source]#

Per-fold precision-recall curves.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
fold_alpha (float, optional) – Opacity of individual fold curves.
full_range (bool, optional) – If True, set both axes to [0, 1].
ylim (tuple of float or None, optional) – Explicit axis limits (override full_range).
xlim (tuple of float or None, optional) – Explicit axis limits (override full_range).
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_confusion_matrices(results, threshold='default', normalize=None, cmap='Blues', fontsize=10, ax=None, **kwargs)[source]#

Per-fold and aggregate confusion matrices.

Parameters:

results (ClassifierResults) – Fitted nested CV classification results object.
threshold ({'default', 'optimized'}, optional) – Which threshold’s confusion matrices to display.
normalize ({None, 'true', 'pred', 'all'}, optional) – Normalization mode. 'true' normalizes by row (true label), 'pred' by column (predicted label), 'all' by total count. None displays raw counts.
cmap (str, optional) – Colormap for the heatmaps.
fontsize (int, optional) – Font size for cell values.
ax (matplotlib.axes.Axes or None, optional) – Ignored (subplots are always created). Kept for API consistency.
**kwargs – Additional keyword arguments passed to imshow.

Returns:

The first axes of the created subplots.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_predicted_vs_actual(results, point_alpha=0.4, point_size=12, xlim=None, ylim=None, ax=None, **kwargs)[source]#

Scatter of predicted vs actual values with identity line.

Parameters:

results (RegressorResults) – Fitted nested CV regression results object.
point_alpha (float, optional) – Opacity of scatter points.
point_size (float, optional) – Size of scatter points.
xlim (tuple of float or None, optional) – Explicit axis limits.
ylim (tuple of float or None, optional) – Explicit axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_residuals(results, fold_idx=None, bins=30, fold_alpha=0.5, xlim=None, ylim=None, ax=None, **kwargs)[source]#

Residual distributions per fold.

Parameters:

results (RegressorResults) – Fitted nested CV regression results object.
fold_idx (int, list of int, or None, optional) – Outer fold index or indices to plot. If None (default), all folds are shown.
bins (int, optional) – Number of histogram bins.
fold_alpha (float, optional) – Opacity of fold histograms.
xlim (tuple of float or None, optional) – Explicit axis limits.
ylim (tuple of float or None, optional) – Explicit axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_residual_qq(results, xlim=None, ylim=None, ax=None, **kwargs)[source]#

QQ plot of pooled residuals.

Parameters:

results (RegressorResults) – Fitted nested CV regression results object.
xlim (tuple of float or None, optional) – Explicit axis limits.
ylim (tuple of float or None, optional) – Explicit axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_prediction_intervals(results, band_alpha=0.25, point_size=8, ylim=None, ax=None, **kwargs)[source]#

Predictions with interval bands.

Parameters:

results (RegressorResults) – Fitted nested CV regression results object.
band_alpha (float, optional) – Opacity of the prediction interval band.
point_size (float, optional) – Size of scatter points.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes

nestkit.plotting.summary.plot_rank_stability(results, top_k=5, fold_alpha=0.6, ylim=None, ax=None, **kwargs)[source]#

Configuration rank stability across folds.

Parameters:

results (ClassifierResults or RegressorResults) – Fitted nested CV results object.
top_k (int, optional) – Number of top configurations to display per fold.
fold_alpha (float, optional) – Opacity of fold lines.
ylim (tuple of float or None, optional) – Explicit y-axis limits.
ax (matplotlib.axes.Axes or None, optional) – Axes to plot on. If None, a new figure is created.
**kwargs – Additional keyword arguments passed to the underlying matplotlib call.

Returns:

The axes with the plot.

Return type:

matplotlib.axes.Axes