Documentation on common colcon bundle scenarios.

Need some documentation that is maintained over time, that people can refer about common bundling scenarios. Also tied to issue #119 but here are some common ones that i have had to work with.

I) Installing python2.7 packages available in pypi

Create a source list in etc/ros/rodep/sources.list.d/ so that rosdep can resolve the package form the package.xml file.
rosdep update, make sure you add the package as a dependency in package.xml
colcon bundle should be able to pick it up from there.

Eg. Bundling AWSIoTPythonSDK 1) Setup source list in rosdep/sources.list.d folder, and add the details of the OS, package manager and package in the corresponding yaml file as below. Note that the contents of the file being per syntax at https://docs.ros.org/independent/api/rosdep/html/rosdep_yaml_format.html The following is the content of the yaml file

awsiotpythonsdk-pip:
  ubuntu:
    pip:
      packages: [AWSIoTPythonSDK]

3) Add the package name as a dependency in package.xml in the rospackage. In this case, it would be

  <depend>awsiotpythonsdk-pip</depend>

4) Now need to run the following commands, and we should be good to roll

rosdep update
rosdep install -y --from-paths src --ignore-src
colcon build
colcon bundle

5) For debugging, you can check if the package is being resolved by running some rodep db commands. Check rosdep --help for info

II) Installing whl files that are not in pypi and are closed sourced.

In some cases, you might have a python package in the form of a whl file, that is not present on pypi. In these cases, the current way to bundle the package is to locally run a pypi server, and ask the colcon bundle to look for packages in local servers.

1) Setup the local pypi server (https://pypi.org/project/pypiserver/ ) For your wheel file that has the name my_wheel_file.whl what you have to do is


pip intall pypiserver  # this install the pypiserver
mkdir ~/packages
cp my_wheel_file.whl ~/packages/
pypi-server -p 8080 ~/packages

2) Once the local pypi server is setup. You can verify what packages are being hosted by the server at

wget http://localhost:8080/simple

3) Locally, you should now be able to install your package using

pip install my_package_in_wheel_file —extra-index-url http://localhost:8080/simple

4) Make sure that the dependency that you add in the package.xml is resolved in rosdep. Same as adding the dependency that we have done for above python packages

5) Now we can bundle it with colcon, you have to add arguments to pip , as follows

colcon bundle —pip-args —extra-index-url http://localhost:8080/simple

III) Install pip3 packages

This one would not work if you updated the rosdep file to pick pip3 package manager. In this case, you will have to

1) Create a python package file, and in the setup.py file, define the custom package requirements. You can see an example of it here at https://github.com/aws-robotics/aws-robomaker-sample-application-deepracer/blob/ros1/simulation_ws/src/sagemaker_rl_agent/setup.py

The file itself looks something like.

# Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
from setuptools import setup, find_packages

# Package meta-data.
NAME = 'sagemaker_rl_agent'
REQUIRES_PYTHON = '>=3.5.0'

setup(
    name=NAME,
    version='0.0.1',
    zip_safe=False,
    packages=find_packages(),
    python_requires=REQUIRES_PYTHON,
    install_requires=[
        'boto3==1.9.23',
        'futures==3.1.1',
        'gym==0.10.5',
        'kubernetes==7.0.0',
        'minio==4.0.5',
        'numpy==1.13.3',
        'pandas==0.20.2',
        'Pillow==6.2.0',
        'pygame==1.9.3',
        'PyYAML==4.2b1',
        'redis==2.10.6',
        'rospkg==1.1.7',
        'scipy==0.19.0',
        'tensorflow==1.12.2',
        'rl-coach-slim==0.11.1'
    ],
    entry_points = {
        'console_scripts': [
            'run_rollout_rl_agent=markov.rollout_worker:main',
            'run_local_rl_agent=envs.local_worker:main'
        ],
    }
)

IV) Bundle apt packages from a custom apt source list

We will lead this with an example. In my case, I had to bundle some custom packages from clearpath, the instructions from which are available at https://wiki.ros.org/ClearpathRobotics/Packages

Per the instructions present in the link above, you can install the apt packages by upading the apt source list a

wget https://packages.clearpathrobotics.com/public.key -O - | sudo apt-key add -sudo sh -c 'echo "deb https://packages.clearpathrobotics.com/stable/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/clearpath-latest.list'sudo apt-get update

On top of the above, you will have to make sure that the packages are resolved using rosdep. This is the same mechanism that you are seeing in the above two sections. An example of the rosdep yaml file is as follows:

boxer_control:
  ubuntu: ros-kinetic-boxer-control
boxer_description:
  ubuntu: ros-kinetic-boxer-description
boxer_firmware:
  ubuntu: ros-kinetic-boxer-firmware

Once you have it, make sure you add the packages in package.xml.

Now we need to update the colcon bundle souces list so that it can pickup the new apt source list.

You will need to either append to the existing source list , or point to a new one. so for the example above, you will need to create a new file which has the existing contents ( https://github.com/colcon/colcon-bundle/blob/master/colcon_bundle/installer/assets/xenial.sources.list ) and add whatever is in /etc/apt/sources.list.d/clearpath-latest.list

now do colcon bundle —apt-sources-list new_source_list.list instead of just colcon bundle

V) update GAZEBO_MODEL_PATH ‘s path

If the models are present in a rospackage, and we to update the GAZEBO_MODEL_PATH, we have the option of doing it by adding some text to the package.xml file as follows http://answers.gazebosim.org/question/6568/uri-paths-to-packages-in-the-sdf-model-file/

Which in our case would translate to adding the following at the end of the package.xml file

  <export>
    <gazebo_ros plugin_path="${prefix}/lib" gazebo_media_path="${prefix}" gazebo_model_path="${prefix}/models"/>
  </export>

colcon / colcon-bundle