ctlearn package

Submodules

ctlearn.tools.train_model module

Tool to train a CTLearnModel on R1/DL1a data using the DLDataReader and DLDataLoader.

class ctlearn.tools.train_model.TrainCTLearnModel(**kwargs: Any)

Bases: Tool

Tool to train a ~ctlearn.core.model.CTLearnModel on R1/DL1a data.

The tool trains a CTLearn model on the input data (R1 calibrated waveforms or DL1a images) and saves the trained model in the output directory. The input data is loaded from the input directories for signal and background events using the ~dl1_data_handler.reader.DLDataReader and ~dl1_data_handler.loader.DLDataLoader. The tool supports the following reconstruction tasks: - Classification of the primary particle type (gamma/proton) - Regression of the primary particle energy - Regression of the primary particle arrival direction based on the offsets in camera coordinates - Regression of the primary particle arrival direction based on the offsets in sky coordinates

aliases = {'background': 'TrainCTLearnModel.input_dir_background', 'pattern-background': 'TrainCTLearnModel.file_pattern_background', 'pattern-signal': 'TrainCTLearnModel.file_pattern_signal', 'reco': 'TrainCTLearnModel.reco_tasks', 'signal': 'TrainCTLearnModel.input_dir_signal', ('o', 'output'): 'TrainCTLearnModel.output_dir'}

batch_size

Size of the batch to train the neural network.

classes = [<class 'ctlearn.core.model.CTLearnModel'>, <class 'ctlearn.core.model.SingleCNN'>, <class 'ctlearn.core.model.ResNet'>, <class 'ctlearn.core.model.LoadedModel'>, <class 'dl1_data_handler.reader.DLDataReader'>, <class 'dl1_data_handler.reader.DLImageReader'>, <class 'dl1_data_handler.reader.DLWaveformReader'>, <class 'dl1_data_handler.reader.DLFeatureVectorReader'>]

description = '\n    Tool to train a ``~ctlearn.core.model.CTLearnModel`` on R1/DL1a data.\n\n    The tool trains a CTLearn model on the input data (R1 calibrated waveforms or DL1a images) and\n    saves the trained model in the output directory. The input data is loaded from the input directories\n    for signal and background events using the ``~dl1_data_handler.reader.DLDataReader`` and\n    ``~dl1_data_handler.loader.DLDataLoader``. The tool supports the following reconstruction tasks:\n    - Classification of the primary particle type (gamma/proton)\n    - Regression of the primary particle energy\n    - Regression of the primary particle arrival direction based on the offsets in camera coordinates\n    - Regression of the primary particle arrival direction based on the offsets in sky coordinates\n    '

dl1dh_reader_type

. Possible values: []

early_stopping

Early stopping parameters for the Keras callback. E.g. {‘monitor’: ‘val_loss’, ‘patience’: 4, ‘verbose’: 1, ‘restore_best_weights’: True}.

examples = '\n    To train a CTLearn model for the classification of the primary particle type:\n    > ctlearn-train-model \\\n        --signal /path/to/your/gammas_dl1_dir/ \\\n        --pattern-signal "gamma_*_run1.dl1.h5" \\\n        --pattern-signal "gamma_*_run10.dl1.h5" \\\n        --background /path/to/your/protons_dl1_dir/ \\\n        --pattern-background "proton_*_run1.dl1.h5" \\\n        --pattern-background "proton_*_run10.dl1.h5" \\\n        --output /path/to/your/type/ \\\n        --reco type \\\n\n    To train a CTLearn model for the regression of the primary particle energy:\n    > ctlearn-train-model \\\n        --signal /path/to/your/gammas_dl1_dir/ \\\n        --pattern-signal "gamma_*_run1.dl1.h5" \\\n        --pattern-signal "gamma_*_run10.dl1.h5" \\\n        --output /path/to/your/energy/ \\\n        --reco energy \\\n\n    To train a CTLearn model for the regression of the primary particle\n    arrival direction based on the offsets in camera coordinates:\n    > ctlearn-train-model \\\n        --signal /path/to/your/gammas_dl1_dir/ \\\n        --pattern-signal "gamma_*_run1.dl1.h5" \\\n        --pattern-signal "gamma_*_run10.dl1.h5" \\\n        --output /path/to/your/direction/ \\\n        --reco cameradirection \\\n\n    To train a CTLearn model for the regression of the primary particle\n    arrival direction based on the offsets in sky coordinates:\n    > ctlearn-train-model \\\n        --signal /path/to/your/gammas_dl1_dir/ \\\n        --pattern-signal "gamma_*_run1.dl1.h5" \\\n        --pattern-signal "gamma_*_run10.dl1.h5" \\\n        --output /path/to/your/direction/ \\\n        --reco skydirection \\\n    '

file_pattern_background

List of specific file pattern for matching files in input_dir_background

file_pattern_signal

List of specific file pattern for matching files in input_dir_signal

finish()

Finish up.

This is called automatically after ~ctapipe.core.tool.Tool.start when ~ctapipe.core.tool.Tool.run is called.

input_dir_background

Input directory for background events

input_dir_signal

Input directory for signal events

lr_reducing

Learning rate reducing parameters for the Keras callback. E.g. {‘factor’: 0.5, ‘patience’: 5, ‘min_delta’: 0.01, ‘min_lr’: 0.000001}.

model_type

. Possible values: []

n_epochs

Number of epochs to train the neural network.

name = 'ctlearn-train-model'

optimizer

Optimizer to use for training. E.g. {‘name’: ‘Adam’, ‘base_learning_rate’: 0.0001, ‘adam_epsilon’: 1.0e-8}.

output_dir

Output directory for the trained reconstructor.

random_seed

Random seed for shuffling the data before the training/validation split and after the end of an epoch.

reco_tasks

List of reconstruction tasks to perform. ‘type’: classification of the primary particle type; ‘energy’: regression of the primary particle energy; ‘cameradirection’: reconstruction of the primary particle arrival direction in camera coordinates; ‘skydirection’: reconstruction of the primary particle arrival direction in sky coordinates.

save_best_validation_only

Set whether to save the best validation checkpoint only.

save_onnx

Set whether to save model in an ONNX file.

setup()

Set up the tool.

This method runs after the configuration and command line options have been parsed.

Here the tool should construct all Components, open files, etc.

sort_by_intensity

Set whether to sort the telescope images by intensity in the data loader. Requires DLDataReader mode to be stereo.

stack_telescope_images

Set whether to stack the telescope images in the data loader. Requires DLDataReader mode to be stereo.

start()

Main function of the tool.

This is automatically called after ~ctapipe.core.tool.Tool.initialize when ~ctapipe.core.tool.Tool.run is called.

validation_split

Fraction of the data to use for validation

ctlearn.tools.train_model.main()

ctlearn.tools.predict_model module

ctlearn.predict_LST1 module

ctlearn.core.model module

This module defines the CTLearnModel classes, which holds the basic functionality for creating a Keras model to be used in CTLearn.

class ctlearn.core.model.CTLearnModel(**kwargs: Any)

Bases: Component

Base component for creating a Keras model to be used in CTLearn.

This class defines the basic functionality for creating a Keras model to be used in CTLearn. It provides the necessary methods to build the backbone of the model and the fully connected head for the specified tasks.

attention_mechanism

Type of squeeze and excitation attention mechanism to use.

attention_reduction_ratio

Reduction ratio for the squeeze and excitation attention mechanism.

head_activation_function

Dictionary containing the activation function for the fully connected head for each task (‘type’, ‘energy’, ‘cameradirection’, ‘skydirection’). Note: The default activation functions are ‘relu’ for ‘type’ and ‘energy’ tasks, and ‘tanh’ for ‘cameradirection’ and ‘skydirection’ tasks. The ‘type’ task uses ‘softmax’ as the final activation function.

head_layers

Dictionary containing the number of neurons in the fully connected head for each task (‘type’, ‘energy’, ‘cameradirection’, ‘skydirection’). Note: The number of neurons in the last layer must match the number of classes or the number of reconstructed values.

init_padding

Initial padding to apply to the input data.

class ctlearn.core.model.LoadedModel(**kwargs: Any)

Bases: CTLearnModel

LoadedModel is a pre-trained Keras model.

This class extends the functionality of CTLearnModel by implementing methods to load a pre-trained Keras model. The model can be used as a backbone for the CTLearn model.

load_model_from

Path to a Keras model file (Keras3) or directory Keras2)

overwrite_head

Set to overwrite the fully connected head from the loaded model.

trainable_backbone

Set to set the backbone model to be trainable.

class ctlearn.core.model.ResNet(**kwargs: Any)

Bases: CTLearnModel

ResNet is a residual neural network model.

This class extends the functionality of CTLearnModel by implementing methods to build a residual neural network model.

architecture

List of dicts containing the number of filters and residual blocks. E.g. [{'filters': 12, 'blocks': 2}, ...].

init_layer

Parameters for the first convolutional layer. E.g. {'filters': 64, 'kernel_size': 7, 'strides': 2}.

init_max_pool

Parameters for the first max pooling layer. E.g. {'size': 3, 'strides': 2}.

name

Name of the model backbone.

residual_block_type

Type of residual block to use.

class ctlearn.core.model.SingleCNN(**kwargs: Any)

Bases: CTLearnModel

SingleCNN is a simple convolutional neural network model.

This class extends the functionality of CTLearnModel by implementing methods to build a simple convolutional neural network model.

architecture

List of dicts containing the number of filters, kernel sizes and number of repetition. E.g. [{'filters': 12, 'kernel_size': 3, 'number': 1}, ...].

batchnorm

Apply batch normalization to the convolutional layers.

bottleneck_filters

Number of filters in the bottleneck layer.

name

Name of the model backbone.

pooling_parameters

Parameters for the max or average pooling layers. E.g. {'size': 2, 'strides': 2}.

pooling_type

Type of pooling to apply to the convolutional layers with pooling_parameters.

ctlearn.core.model.build_fully_connect_head(inputs, layers, activation_function, tasks)

Build the fully connected head for the CTLearn model.

Function to build the fully connected head of the CTLearn model using the specified parameters.

Parameters

inputskeras.layers.Layer

Keras layer of the model.

layersdict

Dictionary containing the number of neurons (as value) in the fully connected head for each task (as key).

activation_functiondict

Dictionary containing the activation function (as value) for the fully connected head for each task (as key).

taskslist

List of tasks to build the head for.

Returns

logitsdict

Dictionary containing the logits for each task.

ctlearn.core.loader module

class ctlearn.core.loader.DLDataLoader(DLDataReader, indices, tasks, batch_size=64, random_seed=None, sort_by_intensity=False, stack_telescope_images=False, **kwargs)

Bases: PyDataset

Generates batches for Keras application.

DLDataLoader is a data loader class that inherits from ~keras.utils.Sequence. It is designed to handle and load data for deep learning models in a batch-wise manner.

Attributes:

data_readerDLDataReader

An instance of DLDataReader to read the input data.

indiceslist

List of indices to specify the data to be loaded.

taskslist

List of tasks to be performed on the data to properly set up the labels.

batch_sizeint

Size of the batch to load the data.

random_seedint, optional

Whether to shuffle the data after each epoch with a provided random seed.

Methods:

__len__():

Returns the number of batches per epoch.

on_epoch_end():

Updates indices after each epoch if random seed is provided.

__getitem__(index):

Generates one batch of data using _get_mono_item(index) or _get_stereo_item(index).

_get_mono_item(index):

Generates one batch of monoscopic data.

_get_stereo_item(index):

Generates one batch of stereoscopic data.

on_epoch_end()

Updates indices after each epoch. If a random seed is provided, the indices are shuffled.

This method is called at the end of each epoch to ensure that the data is shuffled if the shuffle attribute is set to True. This helps in improving the training process by providing the model with a different order of data in each epoch.

ctlearn.core.attention module

This module defines the squeeze-excite blocks for channel-wise and/or spatial-wise attention mechanisms.

ctlearn.core.attention.channel_squeeze_excite_block(inputs, ratio=4, name=None)

A channel-wise squeeze-excite block.

This function creates a channel-wise squeeze-excite block that recalibrates the importance of each channel by using global average pooling followed by two dense layers.

Parameters

inputskeras.layers.Layer

Input tensor to the squeeze-excite block.

ratioint

Reduction ratio for the squeeze-excite block. Default is 4.

namestr, optional

Name for the squeeze-excite block. Default is None.

Returns

keras.layers.Layer

Output tensor for the channel squeeze-excite block.

ctlearn.core.attention.dual_squeeze_excite_block(inputs, ratio=16, name=None)

A channel & spatial (dual) squeeze-excite block.

This function creates a dual squeeze-excite block that combines both channel-wise and spatial-wise squeeze-excite mechanisms. The channel squeeze-excite block focuses on recalibrating the importance of each channel, while the spatial squeeze-excite block focuses on recalibrating the importance of each spatial location.

Parameters

inputskeras.layers.Layer

Input tensor to the squeeze-excite block.

ratioint

Reduction ratio for the channel squeeze-excite block. Default is 16.

namestr, optional

Name for the squeeze-excite block. Default is None.

Returns

keras.layers.Layer

Output tensor for the squeeze-excite block.

References

• [Squeeze and Excitation Networks](https://arxiv.org/abs/1709.01507)

• [Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks](https://arxiv.org/abs/1803.02579)

ctlearn.core.attention.spatial_squeeze_excite_block(inputs, name=None)

A spatial squeeze-excite block.

This function creates a spatial squeeze-excite block that recalibrates the importance of each spatial location by using a convolutional layer with a sigmoid activation.

Parameters

inputskeras.layers.Layer

Input tensor to the squeeze-excite block.

namestr, optional

Name for the squeeze-excite block. Default is None.

Returns

keras.layers.Layer

Output tensor for the spatial squeeze-excite block.

Module contents