ocean-data-factory-sweden / kso

Notebooks to upload/download marine footage, connect to a citizen science project, train machine learning models and publish marine biological observations.
GNU General Public License v3.0
4 stars 12 forks source link

fix: removed tut_utils, shortened subjects fields and changed clip ex… #374

Closed victor-wildlife closed 4 months ago

victor-wildlife commented 4 months ago

Removed tut_utils To improve the architecture of kso I moved all the functions out of tutorials_utils as broken down below. The reason is we shouldn't have "tutorial-specific" functions but rather functions organised by key elements of the system (e.g. yolo, zooniverse, movies, frames, widgets...) that can be called from the projectprocessor. I also removed functions we no longer use.

Functions removed:

Functions moved to:

  1. Widgets:
  1. Zoo:

    • check_clip_size
    • expand_list
    • create_clips
    • extract_clips
    • check_frame_size
    • launch_table (now launch_classifications_table)
  2. Movie:

    • check_ffmpeg_availability
  3. Frame:

    • draw_annotations_in_frame
  4. Yolo:

    • encode_image
    • get_annotations_viewer
    • get_species_mapping
    • process_detections
    • plot_processed_detections

Removed fields in the subjects sql table I removed workflow_id, classifications_count, retired_at and retirement_reason from the subjects table in schema because we now retrieve this information from the actual classifications rather than the subject information stored in Zooniverse. The subject information stored in Zooniverse is not accurate as we have modified workflows' retirement criteria multiple times and removed some subjects or used different format.

Updated clip extractions logic I cleaned the process to create clips to upload to Zooniverse by removing the redundant sample_subjects_from_workflows function

Broke down the choose_footage function I broke down the interactive choose_footage to three steps to facilitate debugging and maintenance. Now users need to 1) choose whether they are using existing or new footage, 2) select the footage of interest and 3) there is a function (check_selected_movies) to store in the project_processor the paths, names and ids of the movies selected.

Other I tidied imports and text format using flake8

review-notebook-app[bot] commented 4 months ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

victor-wildlife commented 4 months ago

@jannesgg This pull request is ready for you to review. I am not sure why the lining test check fails for zooniverse_utils (on my local computer black says is all good). Also, I might need a fresh pair of eyes to look at the "movies_selected" and "movies_path" missing arguments. I have looked for those words using git grep on my repo but I don't have any results...