Sources#

Sources are responsible for loading experiment runs from various deep learning experiment tracking tools or logs. Each source loads a list of Run objects containing parameters, metrics, and optional temporal data using the load() method.

Loaded runs be combined using the + operator to merge multiple sources into a single list of runs.

Tip

To create custom sources, inherit from AbstractSource and implement the load() method.

Abstract Source#

class ablate.sources.AbstractSource[source]#
abstract load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

Experiment Sources#

class ablate.sources.Autrainer(results_dir, experiment_id)[source]#

Autrainer source for loading runs from an autrainer experiment.

Analogous to autrainer, all metrics are reported at the iteration where the tracking metric reaches its best development value.

Parameters:

  • results_dir (str) – The directory where autrainer results are stored.

  • experiment_id (str) – The ID of the autrainer experiment to load.

Raises:

FileNotFoundError – If the specified experiment ID does not exist in the results directory.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

class ablate.sources.ClearML(project_name)[source]#

ClearML source for loading runs from a ClearML server.

Parameters:

project_name (str) – The name of the ClearML project to load runs from.

Raises:

ImportError – If the clearml package is not installed.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

class ablate.sources.MLflow(experiment_names, tracking_uri)[source]#

MLflow source for loading runs from a MLflow server.

Parameters:

  • experiment_names (Union[str, List[str]]) – An experiment name or a list of experiment names to load runs from.

  • tracking_uri (str | None) – The URI or local path to the MLflow tracking server. If None, use the default tracking URI set in the MLflow configuration. Defaults to None.

Raises:

ImportError – If the mlflow package is not installed.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

class ablate.sources.TensorBoard(logdirs)[source]#

TensorBoard source for loading runs from event logs.

Parameters:

logdirs (Union[str, List[str]]) – A path or list of paths to TensorBoard event log directories.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

class ablate.sources.WandB(project, entity=None)[source]#

Weights & Biases (WandB) source for loading runs from a WandB project.

Parameters:

  • project (str) – The name of the WandB project to load runs from.

  • entity (Optional[str]) – Optional WandB entity (team or user). If None, uses the default entity from the WandB configuration. Defaults to None.

Raises:

ImportError – If the wandb package is not installed.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.

Mock Source#

class ablate.sources.Mock(grid, num_seeds=1, steps=25)[source]#

Mock source for generating runs based on a grid of hyperparameters.

For each run, accuracy, f1, and loss metrics are randomly generated.

Parameters:

  • grid (Dict[str, List[Any]]) – Dictionary mapping parameter names to lists of values.

  • num_seeds (int) – Number of runs to generate per config with different seeds. Defaults to 1.

  • steps (int) – Number of steps to use for temporal metrics. Defaults to 25.

load()[source]#

Load the data from the source.

Return type:

List[Run]

Returns:

A list of runs with their parameters, metrics, and optionally temporal data.