search

snad-space / flare-classifier

Stellar flare search in ZTF DR
0 stars 1 forks source link

dvc pipelines #15

Closed anlava closed 1 year ago

anlava commented 1 year ago

DVC pipeline for training and validation of three models: Random forest, CatBoost and MLP.

DVC pipeline structure (dvc dag command output):


 +----------------------------------------------------+            +-------------------------------------+                       
                     | data/light-curves/generated_1m_cleaned.parquet.dvc |            | data/light-curves/lc_1M.parquet.dvc |                       
                     +----------------------------------------------------+            +-------------------------------------+                       
                                                             ****                         ****                                                       
                                                                 *****               *****                                                           
                                                                      ***         ***                                                                
                                                                   +------------------+                                                              
                                                             ******| prepare_datasets |*******                                                       
                                               ********************+------------------+ ******************                                           
                                 **************     *******         **            **           ******     ************                               
                   **************            *******             ***                **               *******          ************                   
            *******                  ********                  **                     **                    ****                  ************       
+----------+                    *****                 +-----------+                    **                    +----------------+               *******
| train_rf |                  **                      | train_mlp |                  **                      | train_catboost |                  **  
+----------+                **                        +-----------+                **                        +----------------+                **    
            **            **                                       **            **                                         ***             ***      
              **        **                                           **        **                                              **         **         
                **    **                                               **    **                                                  **     **           
            +-------------+                                       +--------------+                                        +-------------------+      
            | evaluate_rf |                                       | evaluate_mlp |                                        | evaluate_catboost |      
            +-------------+                                       +--------------+                                        +-------------------+

To start pipeline run dvc repro command. All parameters contained inparams.yaml file

hombit commented 1 year ago

Looks great! Could you please add a short instruction on how to run it to the Readme?