PickNikRobotics / generate_parameter_library

Declarative ROS 2 Parameters
BSD 3-Clause "New" or "Revised" License
230 stars 43 forks source link

Use exist_ok=True in makedirs instead of checking for existence #212

Closed emersonknapp closed 1 month ago

emersonknapp commented 1 month ago

This was a tricky one to track down, as it happened extremely rarely.

The case is, you've got two different Python parameter modules you're generating within a single CMake project

generate_parameter_module(first_generated_params
  ${PROJECT_NAME}/first_params.yaml
)
generate_parameter_module(second_generated_params
  ${PROJECT_NAME}/second_params.yaml
)

When make finally is autorun by colcon build, it's done with parallelization and the two commands can actually be run simultaneously. When this happened (and it was rare but not as uncommon as I would have thought, definitely not reliably reproducible but would fail CI runs sporadically) it looks like this:

Running generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py install/my_package/lib/python3.8/site-packages/my_package/first_generated_params.py src/my_pkg/my_pkg/first_params.yaml
Running generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py install/my_pkg/lib/python3.8/site-packages/my_pkg/second_generated_params.py src/my_pkg/my_pkg/second_params.yaml
Traceback (most recent call last):
  File "generate_parameter_library_py/bin/generate_parameter_library_python", line 33, in <module>
    sys.exit(load_entry_point('generate-parameter-library-py', 'console_scripts', 'generate_parameter_library_python')())
  File "generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py", line 66, in main
    run(output_file, yaml_file, validate_file)
  File "generate_parameter_library_py/generate_parameter_library_py/generate_python_module.py", line 44, in run
    os.makedirs(output_dir)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: 'install/my_pkg/lib/python3.8/site-packages/my_pkg'
make[2]: *** [CMakeFiles/second_generated_parameters.dir/build.make:71: install/my_pkg/lib/python3.8/site-packages/my_pkg/second_generated_parameters.py] Error 1

This fix mitigates that concern by using a non-throwing call to the makedirs API

jcarpinelli-bdai commented 1 month ago

Thank you! Do you plan on backporting this to humble?