P1: clean code

[stark-t]

• [x] utils_datapaths: read all orignal data

• [x] utils_datasampling: create balanced test and validation datasets

• [x] utils_create_datasets: write yolo annotations and images into train, test, val folders

• [ ] run_NN_train: run (in theory) any object detector on same datasets.

[stark-t]

• [ ] testing trained models.

[stark-t]

• [x] write requirements.txt to install relevant python packages

[stark-t]

Configuration File

How to deal best with configuration file? manz use .yaml, .xml, ... Right now we use a simple .py file.

Main variables in config file:

• data_path : path to original dataset
• data_path_sampled : path to sampled dataset (this will be created by utils_create_dataset.py)
• data_yaml : path to yaml-file with paths to sampeld dataset (maybe include all config info in here?)
• save_path : path to save runs, predictions
• hyperparams_path : path to yaml-file with all yolo hyperparameters
• weights : path or name to pretrained weights
• epochs :
• batch_size :
• image_size :

[stark-t]

• [ ] os idepented directories

[stark-t]

Configuration File

How to deal best with configuration file? manz use .yaml, .xml, ... Right now we use a simple .py file.

Main variables in config file:

* data_path : path to original dataset

* data_path_sampled : path to sampled dataset (this will be created by utils_create_dataset.py)

* data_yaml : path to yaml-file with paths to sampeld dataset (maybe include all config info in here?)

* save_path : path to save runs, predictions

* hyperparams_path : path to yaml-file with all yolo hyperparameters

* weights : path or name to pretrained weights

* epochs :

* batch_size :

* image_size :

• [x] write yaml file (extended file of yolov5)

[valentinitnelav]

Configuration File

How to deal best with configuration file? manz use .yaml, .xml, ... Right now we use a simple .py file.

Main variables in config file:

• data_path : path to original dataset
• data_path_sampled : path to sampled dataset (this will be created by utils_create_dataset.py)
• data_yaml : path to yaml-file with paths to sampeld dataset (maybe include all config info in here?)
• save_path : path to save runs, predictions
• hyperparams_path : path to yaml-file with all yolo hyperparameters
• weights : path or name to pretrained weights
• epochs :
• batch_size :
• image_size :

For now, I would leave the yolo hyperparameters to be assigned at run time via the .sh cluster job scripts. We might have to run several of those and test different hyperparameters files, same for epochs, batch size, image size, etc. So, any argument that train.py takes, should be given via a .sh script to the cluster. Also if we introduce another detector, I am sure they will have their hyperparameters options as well, and is more clear for me to have then separate .sh scripts for those detectors as well. We will see how this evolves.

[stark-t]

os idepented directories

@valentinitnelav This should be fixed now right?

[stark-t]

@valentinitnelav I updated the configuration file to the dicusiion we had. We should test if this works.