ocean-data-factory-sweden / kso

Notebooks to upload/download marine footage, connect to a citizen science project, train machine learning models and publish marine biological observations.
GNU General Public License v3.0
4 stars 12 forks source link

Issue in Tutorial 8 when aggregating frames in preparation for model training #329

Closed Bergylta closed 7 months ago

Bergylta commented 7 months ago

Before submitting a bug report, please be aware that your issue must be reproducible with all of the following, otherwise it is non-actionable, and we can not help you:

If this is a custom dataset/training question you must include your train*.jpg, test*.jpg and results.png figures, or we can not help you. You can generate these with utils.plot_results().

πŸ› Bug

A clear and concise description of what the bug is.

To Reproduce (REQUIRED)

Input: Workflow: KSO_tagging_species_new(Hardbottom) Folder: /mimer/NOBACKUP/groups/snic2021-6-9/tmp_dir/KSO_christian_new_frames/ Threshold:0,2

# Run the preparation script
mlp.prepare_dataset(
    agg_df=pp.aggregated_zoo_classifications,
    out_path=output_folder.selected,
    img_size=(720, 540),
    perc_test=percentage_test.value,
)

Output:

Species
Bolocera tuediae
INFO:root:Retrieving movies that are available locally
INFO:root:All 135 movies are mapped from the server
INFO:root:There are 135 movies
INFO:root:There are movies available, but the subject metadata does not contain frame numbers and will therefore not be used.
100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 806/806 [02:53<00:00,  4.65it/s]
Saving frames...: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 806/806 [01:20<00:00,  9.99it/s]
INFO:root:Frames extracted successfully
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File /usr/src/app/kso-dev/kso_utils/project.py:1213, in MLProjectProcessor.prepare_dataset.<locals>.on_button_clicked(b)
   1211 self.species_of_interest = species_list.value
   1212 # code for prepare dataset for machine learning
-> 1213 self.modules["yolo_utils"].frame_aggregation(
   1214     project=self.project,
   1215     server_connection=self.server_connection,
   1216     db_connection=self.db_connection,
   1217     out_path=out_path,
   1218     perc_test=perc_test,
   1219     class_list=self.species_of_interest,
   1220     img_size=img_size,
   1221     remove_nulls=remove_nulls,
   1222     track_frames=track_frames,
   1223     n_tracked_frames=n_tracked_frames,
   1224     agg_df=agg_df,
   1225 )

File /usr/src/app/kso-dev/kso_utils/yolo_utils.py:831, in frame_aggregation(project, server_connection, db_connection, out_path, perc_test, class_list, img_size, out_format, remove_nulls, track_frames, n_tracked_frames, agg_df)
    822     raise Exception(
    823         "No frames found for the selected species. Please retry with a different configuration."
    824     )
    826 # Pre-process frames
    827 # Comment out for the moment as we do not typically need this for all cases
    828 # process_frames(out_path + "/images", size=tuple(img_size))
    829 
    830 # Create training/test sets
--> 831 split_frames(out_path, perc_test)

File /usr/src/app/kso-dev/kso_utils/yolo_utils.py:285, in split_frames(data_path, perc_test)
    282 if counter >= index_test + 1:
    283     # Avoid leaking frames into test set
    284     if movie_name != latest_movie or movie_name == title:
--> 285         file_valid.write(pathAndFilename + "\n")
    286     else:
    287         file_train.write(pathAndFilename + "\n")

TypeError: unsupported operand type(s) for +: 'PosixPath' and 'str'

Expected behavior

A clear and concise description of what you expected to happen.

Environment

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.