Pipeline example
==================


Canonical pipeline
------------------

The main paper workflow is:

``HARPS-N -> flux spectra -> temperature spectra -> planetary injections + CCF RV -> shell HDF5 -> HO/CV5 experiments``

Reference experiment folders:

- ``experiments/cnnShell_HO/``
- ``experiments/cnnShell_CV/``

Experiment scripts
------------------

Hold-out scripts (HO):

- ``experiments/cnnShell_HO/cnnShellTemp.py``
- ``experiments/cnnShell_HO/cnnShellFlux.py``
- ``experiments/cnnShell_HO/cnnShellDetection.py``
- ``experiments/cnnShell_hyperparameter_opt/cnnShellOptuna.py``

Cross-validation scripts (CV5):

- ``experiments/cnnShell_CV/cv5fold_cnn.py``
- ``experiments/cnnShell_CV/cv_cnn_predict.py``
- ``experiments/cnnShell_CV/cv_cnn_detection.py``
- ``experiments/cnnShell_CV/join_chunks.py``

Notebooks and analysis scripts
------------------------------

``notebooks/`` contains runnable Python analysis scripts:

- ``notebooks/ccf_calculator.py``
- ``notebooks/shells_plots_on_the_fly.py``

Shell Data
------------

Shell datasets are generated by the scripts in ``data_generators/``.

Detailed instructions for each generator script are available in ``data_generators/data_generator_README.md``.

- Full paper-scale shell set: about 17 GB.
- One realization with temperature and flux: about 1.7 GB.
- Single modality only: about 0.8 GB.

The lighter workflows are usually enough for diagnostics or training a single model.