Batch execution should be able to read URLs

miguelriemoliveira commented 3 months ago

For example, paths with env variables, rospackages, etc.

The parser for the config.yml does this.

miguelriemoliveira commented 3 months ago

I @Kazadhum ,

I was doing this and realized that the j2 template is still not functional.

It is now doing the initial command to remove collections.

miguelriemoliveira commented 3 months ago

I will try to fix this as well.

miguelriemoliveira commented 3 months ago

The remove_collections_from_dataset script actually works with env variables, so this works:

preprocessing:
  cmd: |
    rosrun atom_calibration remove_collections_from_atom_dataset \
    -json $ATOM_DATASETS/riwmpbot_real/merged/dataset.json \
    -csf "lambda x: int(x) not in {{ collections_to_remove }}" \
    -o $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json

rendered the command is:

reprocessing:
  cmd: |
    rosrun atom_calibration remove_collections_from_atom_dataset \
    -json $ATOM_DATASETS/riwmpbot_real/merged/dataset.json \
    -csf "lambda x: int(x) not in [11, 25, 27, 29, 32, 34, 45, 57]" \
    -o $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json

miguelriemoliveira commented 3 months ago

calibrate also works with env vars.

miguelriemoliveira commented 3 months ago

The problem now is collecting the files:

Traceback (most recent call last): File "/home/mike/workspaces/catkin_ws/src/calibration/atom/atom_evaluation/scripts/batch_execution", line 158, in main() File "/home/mike/workspaces/catkin_ws/src/calibration/atom/atom_evaluation/scripts/batch_execution", line 147, in main raise ValueError('File ' + file + ' should be collected but does not exist.') ValueError: File $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json should be collected but does not exist.

miguelriemoliveira commented 3 months ago

Fixed. I am resolving the file with this atom function

https://github.com/lardemua/atom/blob/ba0abc65f59d33063919fd36d39df5c242c77ede/atom_core/src/atom_core/config_io.py#L152

miguelriemoliveira commented 3 months ago

Now only the mechanism for the runs is missing ...

miguelriemoliveira commented 3 months ago

Run mechanism implemented. Here's an example of the rendered file:

#
#           █████╗ ████████╗ ██████╗ ███╗   ███╗
#          ██╔══██╗╚══██╔══╝██╔═══██╗████╗ ████║
#          ███████║   ██║   ██║   ██║██╔████╔██║
#          ██╔══██║   ██║   ██║   ██║██║╚██╔╝██║
#   __     ██║  ██║   ██║   ╚██████╔╝██║ ╚═╝ ██║    _
#  / _|    ╚═╝  ╚═╝   ╚═╝    ╚═════╝ ╚═╝     ╚═╝   | |
#  | |_ _ __ __ _ _ __ ___   _____      _____  _ __| | __
#  |  _| '__/ _` | '_ ` _ \ / _ \ \ /\ / / _ \| '__| |/ /
#  | | | | | (_| | | | | | |  __/\ v  v / (_) | |  |   <
#  |_| |_|  \__,_|_| |_| |_|\___| \_/\_/ \___/|_|  |_|\_\
#  https://github.com/lardemua/atom

# this yml file contains a set of commands to be run in batch.
# use jinja2 syntax for referencing variables

# Preprocessing will run only once before all experiments.
preprocessing:
  cmd: |
    rosrun atom_calibration remove_collections_from_atom_dataset \
    -json $ATOM_DATASETS/riwmpbot_real/merged/dataset.json \
    -csf "lambda x: int(x) not in [11, 25, 27, 29, 32, 34, 45, 57]" \
    -o $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json

# Define batches to run
experiments:

    hand_pattern_joints_run001:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 1 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: True" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'
    hand_pattern_joints_run002:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 2 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: True" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'
    hand_pattern_joints_run003:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 3 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: True" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'

    hand_pattern_no_joints_run001:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 1 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: False" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'
    hand_pattern_no_joints_run002:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 2 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: False" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'
    hand_pattern_no_joints_run003:
      cmd: |
        rosrun atom_calibration split_atom_dataset \
        -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered.json -tp 80 -ss 3 \
        && \
        rosrun atom_calibration calibrate  -json $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_train.json \
        -v -uic \
        -max_nfev 1 \
        -jsf "lambda x: False" \
        -psf "lambda x: x in ['hand_pattern']" \
        && \
        rosrun atom_evaluation single_rgb_evaluation \
        -train_json $ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json \
        -test_json  $ATOM_DATASETS/riwmpbot_real/merged/dataset_filtered_test.json \
        -sfr -sfrn  $ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv \
        -psf "lambda x: x in ['hand_pattern']" \
        -uic \
        --copy_pattern_transforms
      files_to_collect:
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration.json'
        - '$ATOM_DATASETS/riwmpbot_real/merged/atom_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/original_calibration_params.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/command_line_args.yml'
        - '$ATOM_DATASETS/riwmpbot_real/merged/single_rgb_evaluation.csv'
        - 'package://riwmpbot_real_calibration/urdf/optimized.urdf.xacro'
# End the loop

miguelriemoliveira commented 3 months ago

All is working fine. Will push a closing commit.

miguelriemoliveira commented 3 months ago

Done.

lardemua / atom

Batch execution should be able to read URLs #876