eegproc.deep_learning.cross_validation.nested module

Nested leave-N-subjects-out cross-validation entry point.

eegproc.deep_learning.cross_validation.nested.nested_lnso_cv(model_builder_function, feature_array, label_array, subject_id_array, trial_id_array=None, n_outer_subjects_to_leave_out=1, n_inner_subjects_to_leave_out=1, n_epochs=50, batch_size=32, hyperparameters=None, preprocessing_strategy=None, selection_metric='joint_loss', selection_level='window', evaluation_level='trial', maximize_metric=None, metrics=('accuracy', 'f1', 'precision', 'recall'), log_predictions=True, log_variational_intervals=False, n_prediction_latent_samples=0, latent_sampling_seed=None, n_uncertainty_samples=30, ci_level=0.95, verbose=0, extra_fit_kwargs=None, n_jobs=1, gpu_ids=None, cpus_per_worker=None)

Run nested Leave-N-Subjects-Out CV.

Outer folds can be executed concurrently using multiprocessing with the spawn start method. Inner folds and hyperparameter configurations remain sequential inside each worker, preventing nested process oversubscription.

Return type:

dict

Trial-level evaluation

trial_id_array must contain one trial ID per input window. Predictions are made per window, then probabilities are averaged within each (subject_id, trial_id) group. selection_level controls inner-CV hyperparameter selection and evaluation_level controls the unprefixed primary metrics. Both window- and trial-level metrics/logs are always returned.

Parameters added for concurrency

n_jobs:

Number of persistent outer-fold worker processes. 1 preserves the original sequential behavior unless gpu_ids is supplied.

gpu_ids:

Local GPU indices assigned one-per-worker. For example, gpu_ids=(0, 1, 2, 3) with n_jobs=4. When this is None and multiple workers are requested, visible Slurm/TensorFlow GPUs are assigned automatically, one per worker.

cpus_per_worker:

TensorFlow intra-op CPU threads available to each worker. Keep n_jobs * cpus_per_worker within the CPUs allocated by Slurm.

Notes

Worker state is serialized with cloudpickle, so locally defined model builders and preprocessing callables are supported. The training entry point must still be protected by if __name__ == "__main__":.

type model_builder_function:

Callable[..., Model]

param model_builder_function:

type feature_array:

ndarray

param feature_array:

type label_array:

ndarray

param label_array:

type subject_id_array:

ndarray

param subject_id_array:

type trial_id_array:

ndarray | None, default: None

param trial_id_array:

type n_outer_subjects_to_leave_out:

int, default: 1

param n_outer_subjects_to_leave_out:

type n_inner_subjects_to_leave_out:

int, default: 1

param n_inner_subjects_to_leave_out:

type n_epochs:

int, default: 50

param n_epochs:

type batch_size:

int, default: 32

param batch_size:

type hyperparameters:

dict | None, default: None

param hyperparameters:

type preprocessing_strategy:

Optional[Callable], default: None

param preprocessing_strategy:

type selection_metric:

str, default: 'joint_loss'

param selection_metric:

type selection_level:

Literal['window', 'trial'], default: 'window'

param selection_level:

type evaluation_level:

Literal['window', 'trial'], default: 'trial'

param evaluation_level:

type maximize_metric:

bool | None, default: None

param maximize_metric:

type metrics:

list[str] | tuple[str, ...], default: ('accuracy', 'f1', 'precision', 'recall')

param metrics:

type log_predictions:

bool, default: True

param log_predictions:

type log_variational_intervals:

bool, default: False

param log_variational_intervals:

type n_prediction_latent_samples:

int, default: 0

param n_prediction_latent_samples:

type latent_sampling_seed:

int | None, default: None

param latent_sampling_seed:

type n_uncertainty_samples:

int, default: 30

param n_uncertainty_samples:

type ci_level:

float, default: 0.95

param ci_level:

type verbose:

int, default: 0

param verbose:

type extra_fit_kwargs:

dict | None, default: None

param extra_fit_kwargs:

type n_jobs:

int, default: 1

param n_jobs:

type gpu_ids:

list[int] | tuple[int, ...] | None, default: None

param gpu_ids:

type cpus_per_worker:

int | None, default: None

param cpus_per_worker: