Question: Labels of auto-annotate

[nyxrobotics]

Hello, I am trying to customize this software for my own project. I am trying to create a dataset with two labels, "person" and "roboone".

maskal.yaml

```yaml # training-parameters classes: ['person', 'roboone'] # train-sampler minority_classes: ['person', 'roboone'] # folders weightsroot: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/weights" resultsroot: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/results" dataroot: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets" initial_train_dir: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/initial_train" traindir: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/train" valdir: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/val" testdir: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/test" # data options use_initial_train_dir: False duplicate_initial_model_and_data: False initial_train_file: "" # active-learning sampling supervisely_meta_json: "/home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/meta.json" # training-parameters cuda_visible_devices: '0' transfer_learning_on_previous_models: True learning_rate: 0.01 warmup_iterations: 500 train_iterations_base: 2500 train_iterations_step_size: 2500 step_image_number: 500 eval_period: 500 checkpoint_period: -1 weight_decay: 0.0001 learning_policy: 'steps_with_decay' step_ratios: [0.5, 0.8] gamma: 0.1 train_batch_size: 1 num_workers: 2 # train-sampler train_sampler: "TrainingSampler" repeat_factor_smallest_class: 1.0 # evaluation-parameters confidence_threshold: 0.5 nms_threshold: 0.2 # active-learning sampling experiment_name: 'exp1' strategy: 'uncertainty' mode: 'mean' initial_datasize: 5 pool_size: 5 equal_pool_size: True loops: 5 dropout_probability: 0.25 mcd_iterations: 10 iou_thres: 0.5 auto_annotate: True export_format: labelme ### Light-weight network_config: COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml pretrained_weights: COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml ```

After the first auto-annotation, the labelme will look like the following image.

What I would like to do is the following

• When the Enter key is pressed,
  • Datasets with labels (bottle, skis, cup, motorcycle, chair) that are not included in classes are assumed to have errors and left in the annotate directory.
  • If a polygon extends outside of the image, it is deformed to fit inside the image.

Which files should be changed to implement these features?

[pieterblok]

@nyxrobotics thanks for using MaskAL and giving your input for potential improvements

Regarding your classes, it might be enough to disable/comment out this line in active_learning/sampling/prepare_dataset.py: https://github.com/pieterblok/maskal/blob/1f40d97d5fb96781b3f9f2225bb1e436d66c1585/active_learning/sampling/prepare_dataset.py#L981

By doing so, it skips line 1012 and continues with line 1016. Then it will hopefully load only your specified classes ("person" and "roboone"): https://github.com/pieterblok/maskal/blob/1f40d97d5fb96781b3f9f2225bb1e436d66c1585/active_learning/sampling/prepare_dataset.py#L1016

Regarding the polygon deformation: are you sure the masks are drawn outside the image?

The input for the automatic annotation is the model output from Detectron2 (mask): https://github.com/pieterblok/maskal/blob/1f40d97d5fb96781b3f9f2225bb1e436d66c1585/active_learning/sampling/prepare_dataset.py#L1010

As far as I know this always generates a HxWxD sized matrix (being H the original height of the input image, W the original width of the input image, and D being the number of individual detections). If you want to alter the polygon output, please go to this function: https://github.com/pieterblok/maskal/blob/1f40d97d5fb96781b3f9f2225bb1e436d66c1585/active_learning/sampling/prepare_dataset.py#L1092

[pieterblok]

Closing this issue due to inactivity...

[nyxrobotics]

It has been a while. I commented out line 981 and got the following error on startup. I have already annotated 54 sheets now. I would like to use these 54 sheets to automatically annotate the next 5 sheets. Any advice would be appreciated.

Processing train-dataset: /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/train
11435 images found!
54 annotations found!

Checking train annotations...
len(diff_img_annot): 5
0%|                                                                                                                                                                   | 0/5 [00:00<?, ?it/s]
2024-06-26 22:52:42,650 - active_learning.sampling.prepare_dataset - ERROR - Error in function check_json_presence: IndexError('list index out of range') 

2024-06-26 22:52:42,650 - active_learning.sampling.prepare_dataset - ERROR - Cannot create initial-datasets 

Closing application

log

```bash /home/nyx/.local/lib/python3.8/site-packages/pandas/core/computation/expressions.py:20: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed). from pandas.core.computation.check import NUMEXPR_INSTALLED 2024-06-26 22:52:39,076 - __main__ - INFO - Starting main-application Configuration: classes : ['person', 'roboone'] minority_classes : ['person', 'roboone'] weightsroot : /home/nyx/lib/roboone_auto_dataset/maskal_ws/weights resultsroot : /home/nyx/lib/roboone_auto_dataset/maskal_ws/results dataroot : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets initial_train_dir : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/initial_train traindir : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/train valdir : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/val testdir : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/test use_initial_train_dir : False duplicate_initial_model_and_data : False initial_train_file : supervisely_meta_json : /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/meta.json cuda_visible_devices : 0 transfer_learning_on_previous_models : True learning_rate : 0.01 warmup_iterations : 500 train_iterations_base : 2500 train_iterations_step_size : 2500 step_image_number : 500 eval_period : 500 checkpoint_period : -1 weight_decay : 0.0001 learning_policy : steps_with_decay step_ratios : [0.5, 0.8] gamma : 0.1 train_batch_size : 1 num_workers : 2 train_sampler : TrainingSampler repeat_factor_smallest_class : 1.0 confidence_threshold : 0.5 nms_threshold : 0.2 experiment_name : exp1 strategy : uncertainty mode : mean initial_datasize : 5 pool_size : 5 equal_pool_size : True loops : 5 dropout_probability : 0.25 mcd_iterations : 10 iou_thres : 0.5 auto_annotate : True export_format : labelme network_config : COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml pretrained_weights : COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml Processing train-dataset: /home/nyx/lib/roboone_auto_dataset/maskal_ws/datasets/train 11435 images found! 54 annotations found! Checking train annotations... len(diff_img_annot): 5 0%| | 0/5 [00:00

[nyxrobotics]

The polygons I created seem to be incorrect. I will check.