[Binary] Release inside ROS main repo

liesrock commented 4 years ago

Once we fix the release versioning issue, we should try to understand how to integrate ROS End-Effector inside the ROS binaries release.

torydebra commented 4 years ago

I put here useful things for this issue, I will edit this message

Probably we will need to create metapackage: info here
I do not know anything about licences, but ROS speaks about them here
ROS package proposal. Is this what we are looking for?
Prerelease that can be done locally. Here. ROS says release are done with bloom. Before the package proposal, a prerelease test is useful to do, and it is done on local machine. Then comes the package proposal, and then the bloom release

torydebra commented 4 years ago

For Prerelase Info taken from ROS webiste : http://wiki.ros.org/regression_tests#Prerelease_Instructions

Install docker (which version? community or enterprise?).
sudo apt-get install python3-ros-buildfarm
use the website to generate the script and run it
run the prerelease.sh (docker needed)

Some problem I had below not useful for this issue

UPDATE I have generated the script as described in the pages linked before. The script prerelease.sh does not work:

./prerelease.sh 
...
Build step 1
# BEGIN SECTION: docker version
./prerelease.sh: 15: /tmp/prerelease_job/prerelease_build_underlay.sh: docker: not found

I should have docker:

sudo systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: https://docs.docker.com

UPDATE 2 I had docker badly installed, now it works

torydebra commented 4 years ago

For Prerelease test I generated the commands with the website of ROS, selecting all the rosee packages.

EDIT SEE UPDATED CODE IN THE MESSAGES BELOW

mkdir -p /tmp/prerelease_job
cd /tmp/prerelease_job
generate_prerelease_script.py \
  https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml \
  kinetic default ubuntu xenial amd64 \
  --custom-repo \
    ROSEndEffector__custom-3:git:https://github.com/ADVRHumanoids/ROSEndEffector:devel \
    rosee_gazebo_plugin__custom-4:git:https://github.com/ADVRHumanoids/rosee_gazebo_plugins:devel \
    rosee_msg__custom-6:git:https://github.com/ADVRHumanoids/rosee_msg:devel \
    rosee_gui__custom-7:git:https://github.com/ADVRHumanoids/rosee_gui:devel \
  --level 1 \
  --output-dir ./

Then the test can be run with (it takes a bit of times, like 20 min)

./prerelease.sh

After all this time, the disappointing output says that the package srdfdom is not found:

CMake Warning at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:76 (find_package):
  Could not find a package configuration file provided by "srdfdom" with any
  of the following names:

    srdfdomConfig.cmake
    srdfdom-config.cmake

[...]

So, probably we need to tell the build to install dependencies before. I am trying to find out how

liesrock commented 4 years ago

Great! I can help on the issue, but this is the right direction. @alaurenzi you can follow fyi.

torydebra commented 4 years ago

So, probably we need to tell the build to install dependencies before. I am trying to find out how

I found out that we need to put dependencies in package.xml (they were only in cmakelists, to us this did not cause issues), as I did in the releaseTest branch.

But now another dependencies problem: When compiling, ROSEndEffector is compiled as first. So:

CMake Warning at /opt/ros/kinetic/share/catkin/cmake/catkinConfig.cmake:76 (find_package):
  Could not find a package configuration file provided by "rosee_msg" with
  any of the following names:

    rosee_msgConfig.cmake
    rosee_msg-config.cmake

  Add the installation prefix of "rosee_msg" to CMAKE_PREFIX_PATH or set
  "rosee_msg_DIR" to a directory containing one of the above files.  If
  "rosee_msg" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  CMakeLists.txt:10 (find_package)

I noticed that when I compile normally (not prerelease test), rosee_msg is always the first to be compiled, so we were "lucky". This probably happens also on all the other systems (travis included) except the workspace used for prerelease test.

We need to find a method to force rosee_msg compilation as first, or something like that.

alaurenzi commented 4 years ago

I think you need to insert <build_depend>rosee_msg</build_depend> inside package.xml

torydebra commented 4 years ago

Thanks @alaurenzi. I tried that before, but probably I had done wrong something, now it works.

torydebra commented 4 years ago

Another dependencies issue :D This time is between our packages

==> cmake /tmp/ws/src/rosee_gui__custom-7 -DCATKIN_DEVEL_PREFIX=/tmp/ws/devel_isolated/rosee_gui -DCMAKE_INSTALL_PREFIX=/tmp/ws/install_isolated -DBUILD_TESTING=0 -DCATKIN_SKIP_TESTING=1 -G Unix Makefiles in '/tmp/ws/build_isolated/rosee_gui'

[...]

[ 21%] Building CXX object CMakeFiles/rosee_gui_ActionBoxesLayout.dir/src/ActionBoxesLayout.cpp.o
In file included from /tmp/ws/src/rosee_gui__custom-7/include/rosee_gui/ActionBoxesLayout.h:4:0,
                 from /tmp/ws/src/rosee_gui__custom-7/src/ActionBoxesLayout.cpp:1:
/tmp/ws/src/rosee_gui__custom-7/include/rosee_gui/ActionLayout.h:22:35: fatal error: ROSEndEffector/Action.h: No such file or directory
compilation terminated.

I double check to have all the things explained here, it seems not sufficient

alaurenzi commented 4 years ago

A possible pitfall is the following: your project name is ros_end_effector, but your include directory (the one you install) is named ROSEndEffector. If you install your headers with catkin's default cmake command, it will install to ${CMAKE_INSTALL_PREFIX}/include/${PROJECT_NAME} So, from external projects, you will need to include ros_end_effector/Action.h. This is not convenient, and you should follow the general convention that the include directory is named after the project.

torydebra commented 4 years ago

Thanks! that could be the problem. At this point I need to ask @liesrock for a final decision about the name, I opened another issue (#52) for that

torydebra commented 4 years ago

Ok for the name problem.

Some updates: now it seems a problem of system dependencies. While running, I have a lot of Investigating/broken things, for a lor of packages, like this one:

Investigating (1) ros-kinetic-rosbag [ amd64 ] < none -> 1.12.14-0xenial-20191214-075824+0000 > ( misc )
Broken ros-kinetic-rosbag:amd64 Depends on python-rospkg [ amd64 ] < none -> 1.2.6-100 > ( python )
  Considering python-rospkg:amd64 43 as a solution to ros-kinetic-rosbag:amd64 1
  Holding Back ros-kinetic-rosbag:amd64 rather than change python-rospkg:amd64

I do not know if these are real issues.

The error is:

The following packages have unmet dependencies:
 ros-kinetic-catkin : Depends: python-catkin-pkg (> 0.4.3) but it is not going to be installed
 ros-kinetic-dynamic-reconfigure : Depends: ros-kinetic-roslib but it is not going to be installed
                                   Depends: ros-kinetic-rospy but it is not going to be installed
                                   Depends: ros-kinetic-rosservice but it is not going to be installed
 ros-kinetic-gazebo-ros : Depends: ros-kinetic-roslib but it is not going to be installed
                          Depends: ros-kinetic-tf but it is not going to be installed
 ros-kinetic-kdl-parser : Depends: ros-kinetic-urdf but it is not going to be installed
 ros-kinetic-moveit : Depends: ros-kinetic-moveit-commander but it is not going to be installed
                      Depends: ros-kinetic-moveit-core but it is not going to be installed
                      Depends: ros-kinetic-moveit-planners but it is not going to be installed
                      Depends: ros-kinetic-moveit-plugins but it is not going to be installed
                      Depends: ros-kinetic-moveit-ros but it is not going to be installed
                      Depends: ros-kinetic-moveit-setup-assistant but it is not going to be installed
 ros-kinetic-moveit-ros-planning-interface : Depends: ros-kinetic-moveit-core but it is not going to be installed
                                             Depends: ros-kinetic-moveit-ros-planning but it is not going to be installed
                                             Depends: ros-kinetic-moveit-ros-warehouse but it is not going to be installed
                                             Depends: ros-kinetic-actionlib but it is not going to be installed
                                             Depends: ros-kinetic-moveit-ros-manipulation but it is not going to be installed
                                             Depends: ros-kinetic-moveit-ros-move-group but it is not going to be installed
                                             Depends: ros-kinetic-rospy but it is not going to be installed
                                             Depends: ros-kinetic-tf but it is not going to be installed
                                             Depends: ros-kinetic-tf-conversions but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

On the web, usually this implies a bad installation, that can be solved sometimes installing some libraries or some dependencies with a bit different name (e.g.apt install ros-kinetic-catkin and not apt install catkin). But I do not know how to say the prerelease test to install manually things. The only way I found is to add packages in the package.xml file, but I think we should not add all these dependency above... Maybe we need to create a rosdep file? Also the config file for test here (https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml) seems to be not suitable to add libraries to install... and they say it should be ready as it is.

liesrock commented 4 years ago

@torydebra thanks for the details: it seems a bit difficult to help you on this issue just from the issue tracking on Github.

Can you share with me the docker image you are using and list the command I have to execute to reproduce the issue?

torydebra commented 4 years ago

Do not worry, I solved yesterday. The problem was that rospy was missing in package.xml and cmakelists I also had to merge with the new updates for tests (#40). The working branch for this is prereleaseTest (for the main package) and the other remain the devel ones. If you want to run on your machine:

mkdir -p /tmp/prerelease_job
cd /tmp/prerelease_job
generate_prerelease_script.py \
  https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml \
  kinetic default ubuntu xenial amd64 \
  --custom-repo \
    ROSEndEffector__custom-3:git:https://github.com/ADVRHumanoids/ROSEndEffector:prereleaseTest \
    rosee_gazebo_plugin__custom-4:git:https://github.com/ADVRHumanoids/rosee_gazebo_plugins:devel \
    rosee_msg__custom-6:git:https://github.com/ADVRHumanoids/rosee_msg:devel \
    rosee_gui__custom-7:git:https://github.com/ADVRHumanoids/rosee_gui:devel \
  --level 1 \
  --output-dir ./

Take care that on my system it lasts like 30 minutes. Also like 3-4 gb space are needed. When it is finished, you can do docker system prune -a -f to delete ALL the docker images included this one to free the space.

torydebra commented 4 years ago

I will update the devel branch when we solve the pull request of tests #40

torydebra commented 4 years ago

I update the prereleasetest with the recent devel updates to see if the ros buildfarm is still ok, and it is :D

torydebra commented 3 years ago

Semptember 2021 Version (melodic and masters branches)

mkdir -p /tmp/prerelease_job
cd /tmp/prerelease_job
generate_prerelease_script.py \
  https://raw.githubusercontent.com/ros-infrastructure/ros_buildfarm_config/production/index.yaml \
  melodic default ubuntu bionic amd64 \
  --custom-repo \
    ROSEndEffector__custom-1:git:https://github.com/ADVRHumanoids/ROSEndEffector/:prereleaseTest \
    rosee_gazebo_plugin__custom-2:git:https://github.com/ADVRHumanoids/rosee_gazebo_plugins:master \
    rosee_msg__custom-3:git:https://github.com/ADVRHumanoids/rosee_msg:master \
    rosee_gui__custom-4:git:https://github.com/ADVRHumanoids/rosee_gui:master \
  --level 1 \
  --output-dir ./

Issues

[SOLVED] https://answers.ros.org/question/347063/error-pre-release-melodic/

torydebra commented 3 years ago

After prerelease tests, http://wiki.ros.org/bloom/Tutorials/FirstTimeRelease should be followed. In other wiki pages it says to first create the pull request to the rosdistro this should not be done since this is done automagically at the end of the bloom procedure.

Actually we are waiting for someone to accept the pull request for the rosee_msg package. Since the main ROSEndEffector depends on this, we have to release rosee_msg first. Hence, rosee_gui and rosee_gazebo_plugins will come at last.

torydebra commented 3 years ago

Ok, we are officialy among the ros packages with the _roseemsg (http://wiki.ros.org/rosee_msg?distro=melodic) and the main repo ROSEndEffector (http://wiki.ros.org/end_effector). Actually on melodic only.

It would be possible to install the package with apt install ros-melodic-... in the next months I think (I see that they do the sync usually once a month https://discourse.ros.org/c/release/melodic/38)

:tada: @liesrock @lia2790 @ntsagarakis

torydebra commented 3 years ago

Coming soon

@liesrock

liesrock commented 3 years ago

out https://discourse.ros.org/t/new-packages-for-melodic-2021-11-08/23019 ! Great job @torydebra

fyi @lia2790 @ntsagarakis

ADVRHumanoids / ROSEndEffector

[Binary] Release inside ROS main repo #48

Issues