Closed seanyen closed 5 years ago
On Windows, the multiprocessing will require the script who consumes the catkin_pkg to be patched in order to work properly.
Can you please provide more information about this?
@dirk-thomas based on Python manual, the multiprocessing usage on Windows requires additional treatment to get it worked properly. https://docs.python.org/2/library/multiprocessing.html
Here is one example how the downstream could be impacted by this multiprocessing issue: https://github.com/ros-infrastructure/rosdep/issues/645
based on Python manual, the multiprocessing usage on Windows requires additional treatment to get it worked properly. https://docs.python.org/2/library/multiprocessing.html
I am sorry but this page is really long and I don't find the information you are trying to point to. Please either use a specific link to the section describing the documentation you are referring to or quote the relevant text from the docs here.
Here is the specific link to the Windows section: https://docs.python.org/2/library/multiprocessing.html#windows
Ok, I have read the section "16.6.3.2. Windows" - in particular "Safe importing of main module".
Have you tried to apply the suggest call of freeze_support()
to address the problem described in the rosdep
ticket?
For rosdep
, the least change making it worked to me are:
rosdep.py
(It seems to be only required for Python2. I ran some tests on Python3.7 and I don't need to postfix .py
as file extension.)if __name__ == '__main__':
before rosdep_main()
Alternatively, we can use the console_scripts
feature will auto generate such compliant format, so this PR https://github.com/ros-infrastructure/rosdep/pull/656 (cc @nuclearsandwich) just happens to help solve the multiprocessing problems.
However, my questions are that since catkin_pkg
are a so much low level module in ROS stack, there are many tools using catkin_pkg which potentially need this treatment, do we address them all one by one? or we firstly disable it (or optionally opt-out by flags) on Windows to unblock users who hits this problem on the downstream projects (before the fix is addressed on the downstream tools)?
do we address them all one by one?
I wonder if this can't be worked around in catkin_pkg
- specifically where the multiprocessing
module is being used. That would address all problems at once.
we firstly disable it on Windows to unblock users who hits this problem
I would rather not apply a workaround if the possibility of a actual fix in catkin_pkg
hasn't been ruled out. From past experiences a hacky fix like this tends to stay around for a long since there is no reason to look at it again.
I would rather not apply a workaround if the possibility of a actual fix in
catkin_pkg
hasn't been ruled out. From past experiences a hacky fix like this tends to stay around for a long since there is no reason to look at it again.
That's fair enough. I will look for impacted tools and send fix PR there instead.
I would rather not apply a workaround if the possibility of a actual fix in
catkin_pkg
hasn't been ruled out. From past experiences a hacky fix like this tends to stay around for a long since there is no reason to look at it again.That's fair enough. I will look for impacted tools and send fix PR there instead.
There might be a misunderstanding. I am suggesting to try to find a fix for catkin_pkg
(so a patch against this repository) to address the problem you are seeing with e.g. rosdep
and likely other tools too.
@dirk-thomas Thanks for the clarification! I created #251 to propose a change to be only scoped in catkin_pkg. Please take a look and let me know what do you think.
@seanyen With #251 closed how do you want to proceed with this ticket?
@dirk-thomas Since workaround is not really preferred at this moment, and I haven't really found a good way to isolate a fix to only catkin_pkg. I will keep pushing the entry_points
approach to downstream projects, which I believe it is also more sustainable. That being said, I will close this one now. Thanks for your time!
On Windows, the multiprocessing will require the script who consumes the catkin_pkg to be patched in order to work properly.
Instead of touching up all the scripts consuming catkin_pkg, disable multiprocessing on Windows. Otherwise, catkin_pkg won't work for a workspace with more than 100 packages on Windows.