Open hauptmech opened 8 years ago
I just tried to get kinetic packages and compile locally. Some minor changes are needed (add kinetic python versions). Moreover, it seems that PKGBUILDs should also add CMAKE_PREFIX_PATH to build succesfully.
that sounds promising
Actual CMAKE_PREFIX_PATH does not seem to be necessary. I think ros-kinetic-catkin needs to have -DCATKIN_BUILD_BINARY_PACKAGE=OFF since the PKGBUILDs are generated with this on. Is this really necessary?
Besides that, several packages require renaming dependencies. Seems a rosdep key issue. I wonder if we can get proper arch package names listed in official rosdep.
I've been updating rosdep keys in my own rosdistro fork. Managed to fix several missing dependencies problems.
So far I had this problem: urlgrabber is mapped to python2-urlgrabber, but should actually be urlgrabber in ArchLinux. The python fix logic seems to override the rosdep key (which actually performs the correct mapping).
Also, ros-kinetic-opencv3 should not be compiled as is, since now Arch supports opencv3 in stable. I wonder how to deal with this. Maybe make a metapackage providing ros-kinetic-opencv3 and which depends on opencv. I dealt with it locally by changing image_geometry's depend entry to "opencv".
Finally, I'm getting stuck with rviz since it attempts to find Eigen3 but falls back to cmake_modules version (find_package(Eigen)) which also fails for some reason. I wonder if it is due to this: https://bugs.archlinux.org/task/49234. ArchLinux package makes its own .cmake file using some patches, however a proper cmake file seems to be already included with eigen upstream.
@bchretien any ideas on these issues?
-DCATKIN_BUILD_BINARY_PACKAGE=OFF
, as it's the core package and thus provides the setup scripts. This has been changed manually so far for indigo/jade (so git checkout -p
during updates), but the generation script could handle this automatically.python-urlgrabber
does not have an entry for Arch, and thus we assume by default that the package is named python[2]-urlgrabber
.As for dependencies, we need to make PRs to rosdistro when Arch/AUR packages have been named differently than Debian's packages. For example, python-urlgrabber does not have an entry for Arch, and thus we assume by default that the package is named python[2]-urlgrabber.
I understand. The thing is that I added an entry pointing to "urlgrabber" and the PKGBUILD still got a dependency named python2-urlgrabber. Thus I believe the fix logic acts before looking for a mapping in the rosdep dictionary.
What's the conflict with the opencv3 package?
ros-kinetic-opencv3 builds opencv 3.1, which is not necessary for ArchLinux anymore
I can't test that Eigen issue right now. Could you provide an error log?
I actually "solved" it by getting the FindEigen3.cmake file and dropping it in /usr/share/cmake. I will try to reproduce it later. Anyway, I asked the archlinux bug ticket be reopened to improve eigen3 package. Maybe that fixes the problem.
Finally, I'm getting stuck with rviz since it attempts to find Eigen3 but falls back to cmake_modules version (find_package(Eigen)) which also fails for some reason. I wonder if it is due to this: https://bugs.archlinux.org/task/49234. ArchLinux package makes its own .cmake file using some patches, however a proper cmake file seems to be already included with eigen upstream.
There was a bug in eigen as packaged by archlinux that I just got fixed. Maybe that was the issue (depending on when you last synced eigen). The same issue was preventing me from building indigo-rviz.
Ah, ok. Great then.
I managed to compile ros-kinetic-perception and ros-kinetic-desktop. I had to patch rviz for gcc6 (as with Indigo). I think stage has a similar issue, but I skipped ros-kinetic-simulators.
Only remaining issues are:
I will make a pull request for these updated rosdep keys to rosdistro.
opencv3: how to deal with this?
I think the ros package should be installed. I would expect the catkin and setup.sh environments to search /opt/ros/kinetic first for their opencv. Arch packages will use the system install.
I will make a pull request for these updated rosdep keys to rosdistro.
I think the ros package should be installed. I would expect the catkin and setup.sh environments to search /opt/ros/kinetic first for their opencv. Arch packages will use the system install.
I think this would be wasteful since arch already officially supports opencv. Unless we see an incompatibility with Arch's opencv, I would avoid unnecessary recompilation.
Nice! Can you drop a link to the PR here so it's easy to find between now and when it gets accepted?
Unless we see an incompatibility with Arch's opencv, I would avoid unnecessary recompilation.
We can check the mods they made for incompatibility: http://wiki.ros.org/opencv3#Building_the_Package
BINARY_PACKAGE=OFF
is now done automatically for catkin (https://github.com/bchretien/arch-ros-stacks/commit/845a4af6bb6e13673b74dde7fd3416b53c003438).
Great!
When my rosdep pull request is accepted maybe you can see why urlgrabber is being changed into python2-urlgrabber. I've looked at the code but it seems that it uses the dictionary first as expected so I'm unsure about the problem. El jun 27, 2016 7:20 AM, "Benjamin Chrétien" notifications@github.com escribió:
BINARY_PACKAGE=OFF is now done automatically for catkin (845a4af https://github.com/bchretien/arch-ros-stacks/commit/845a4af6bb6e13673b74dde7fd3416b53c003438 ).
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-228707733, or mute the thread https://github.com/notifications/unsubscribe/AAJ3qnufABkjG2USKtweCuDD0BwlGuQDks5qP6PTgaJpZM4I-QuY .
The corrected rosdep keys were accepted in rosdistro repo. I've now submitted a PR to rosdistro for adding ArchLinux to distribution files to at least have that ready.
@bchretien While the buildfarm and bloom stuff is being solved, are you considering pushing kinetic packages to AUR to have something in the meantime? Or what do you think would be the plan?
@v01d I'm quite busy right now (PhD defense this Friday), and I don't know if I'll be actively using kinetic in the near future, so you can manage the kinetic packages in the AUR if you're interested.
Oh, ok. Good luck with your defense then =) I had mine two months ago.
I'm not sure if I would be able to maintain all these packages. It would seem that we need to call for some help of some other ROS arch users. I can push the current packages but I will probably have to orphan them if so. In principle, given your nice scripts it shouldn't be too difficult to do.
So, if anyone else is reading this, help is welcomed =)
On Mon, Jul 4, 2016 at 6:17 AM, Benjamin Chrétien notifications@github.com wrote:
@v01d https://github.com/v01d I'm quite busy right now (PhD defense this Friday), and I don't know if I'll be actively using kinetic in the near future, so you can manage the kinetic packages in the AUR if you're interested.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-230244546, or mute the thread https://github.com/notifications/unsubscribe/AAJ3qq0pyfXopVdDYRsX248_VyKFZqkAks5qSM-SgaJpZM4I-QuY .
I'm following along.
I think it's better to push the packages and then orphan them than do nothing. It creates an easy opportunity for others to join in.
I forget, is there a script to walk the ros dependencies and build locally from arch-ros-stacks? Or does in need to be pushed to AUR so yaourt or some-such can do it?
I can do a little build testing but that's it for the near future.
I've created a script called aap (github.com/v01d/aap) which can compile a pool of packages lying inside a directory, following dependency order (builds + installs). So the AUR is not actually needed and this could be an alternative to provide ROS for Arch without having to manually push packages to AUR.
On Mon, Jul 4, 2016 at 11:11 PM, Hauptmech notifications@github.com wrote:
I'm following along.
I think it's better to push the packages and then orphan them than do nothing. It creates an easy opportunity for others to join in.
I forget, is there a script to walk the ros dependencies and build locally from arch-ros-stacks? Or does in need to be pushed to AUR so yaourt or some-such can do it?
I can do a little build testing but that's it for the near future.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-230374026, or mute the thread https://github.com/notifications/unsubscribe/AAJ3qpH4tLgkiQgxWLqoYqCPY_XxKBVxks5qSb1JgaJpZM4I-QuY .
Nice! Worth adding to arch-ros-stacks I think.
I still think it's good to push the AUR packages. I think it's better if someone has a failed AUR install and comes here (a potential contributor) instead of not seeing it when searching via AUR and feeling like they have to switch to Ubuntu just to do a class project or something.
:+1: for aap, I never took the time to do something like that, and that should really help. Does it support custom commands as callbacks? I basically have a small bash functions that "processes" local AUR packages: git add -p
to review, then automatic commit (default commit message with the new version of the PKGBUILD), package building, installing and pushing to the AUR. If there's an issue at any step of the process, I stop to investigate.
So what's the status with this? Can we get the base packages built and running (if there's still issues with opencv3)?
I would be willing to help push things along, as we are having a bit of a large hardware update with the RasPi3 and it only having Xenial.
Please look at the discussion in #50 I managed to have a separate github repo with most (all?) ROS's AUR dependencies. I've built them using travis integration and they are built at the URL noted in the linked issue. I did not push to AUR since I cannot commit to maintain these in the long-term.
I realized this is not a one-man-show. If Arch is ever to be a supported platform for ROS, a critical mass of users is needed. First, you would have to at least find TU's to maintain the AUR dependencies or at least provide someone who is willing to host them somewhere else (as I did using a free server for test purposes, but which will eventually stop working). Then, significant efforts are needed in different fronts on the ROS side (mainly extending bloom and then the ros buildfarm).
Until that is achieved, I think manually pushing to AUR every time an upstream ros package is updated does not scale. Maybe it would be better to instruct users to clone the ros-arch-stacks repo and use the scripts to maintain it themselves.
The main problem, IMHO, with any non-standard solution (i.e. one where Arch is not an officially supported platform) is that a system dependency (standard archlinux package) which is updated can easily break the entire ROS installation, which nowadays requires manual handling. I've become a bit annoyed of this over time. I'm now considering using a docker container with ubuntu and ROS installed and use this for development. That is, until other users become interested in solving the aforementioned issues to allow Arch become a reasonable platform for ROS development.
In short: if you are willing to help, I think a first step would be to gather other ROS/Arch users (maybe in the forums) and gather some efforts.
@v01d Which forums? I might be able to donate a server (on linode); though I worry that the arch-ros endeavour is starting to lose the KISS aspect that makes arch such a nice tool.
The ArchLinux forums I mean. Or mailing lists.
Hosting the base packages somewhere fixed would be a great starting point to convince others that is possible. However, the only way to have this done properly (without requiring a ROS Arch user to loose the benefit of ROS's packaging system) is to support it upstream. For that, further effort is needed. That is why I suggested finding other users.
I hope that sometime in the future more ROS users look into Arch as a viable platform.
Best!
On Sat, Jul 30, 2016 at 7:51 PM, Hauptmech notifications@github.com wrote:
@v01d https://github.com/v01d Which forums? I might be able to donate a server (on linode); though I worry that the arch-ros endeavour is starting to lose the KISS aspect that makes arch such a nice tool.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-236394281, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ3qsb4D_Za4trlQN7VyI6_Ot2Aevrzks5qa9WIgaJpZM4I-QuY .
I'm not sure how many ROS users hang out on the ArchLinux forums.... I think you are more likely to find ROS users with an Arch machine or curiosity than Arch users with a ROS need. So ROS discourse might be a better place (or ros answers even). Or any other concentration of ROS users.
An Arch user coming to robotics will hit https://wiki.archlinux.org/index.php/Ros so that should be accurate.
Likewise the ROS install page.
Yes, that might be better.
Also, I can give access to the ros-arch organization on github if someone else want to maintain the ROS dependencies and leverage the travis ci build system I've made. If another server is available the URL where the package is uploaded can easily be changed.
On Sat, Jul 30, 2016 at 10:58 PM, Hauptmech notifications@github.com wrote:
I'm not sure how many ROS users hang out on the ArchLinux forums.... I think you are more likely to find ROS users with an Arch machine or curiosity than Arch users with a ROS need. So ROS discourse might be a better place (or ros answers even). Or any other concentration of ROS users.
An Arch user coming to robotics will hit https://wiki.archlinux.org/index.php/Ros so that should be accurate.
Likewise the ROS install page.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-236402871, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ3quQA_5SkteL8aUVrmXlqJwmurLbZks5qbAE1gaJpZM4I-QuY .
Hi everyone. If you are looking for Arch users of ROS, I can help with maintaining some of the pakcages in aur.
Same here. I built ROS kinetic from source this week, but it was a bit tedious.
:+1: for trying to bring people together and solve this as a community. It's clearly not a task for just one person, and we seem to have enough people to solve this situation. As @v01d said, we need either a TU or a server where we can host the packaged dependencies to really get things started.
On a side note, I just moved to a new place, so I don't have a stable Internet access, and only limited spare time right now. I'll try to follow the talks on GitHub and discourse, and I'll share my feedback when I find the time.
As for the server, I have small one at my home with pretty stable internet, so I can host some packages and I think it could be enough for a start. It is 80.252.155.196. However, I don't have registered DNS right now. Edit: It could be also possible to involve Moscow Institute of Physics and Technology's resources for that if we can estimate what we need.
If we can get some estimates of data usage, ports needed, and specs of a machine that would fit the requirements for the server, I might be able to have a poke at some of the official channels at the Queensland University of Technology. They may probably be pretty happy to say that they are doing a part to support "ROS".
Hi everyone. I finally have an access to yet almost unused server with pretty good resources (8-core i7, 16G RAM), so I can set up a virtual machine for hosting and/or even building some packages. It would be pretty good to have any advice on how much resources to allocate for such machine.
Assuming you have to run virtual machine (because the server is in use for something else), then I'd do a lightweight/minimal image for serving the files and second image that uses maximum available resources for builds.
I was about to add support for Arch in ROS buildfarm. I have deployed a local buildfarm server for that.
The buildfarm needs at least 3 virtual machines (a Jenkins master, a Jenkins slave and a repository for binary packages). Exact requirements are listed here to replicate the full build system but that includes dozen of Ubuntu versions. It’s probably much lighter if you focus only on Arch. I don’t have the resources to maintain a publicly available buildfarm but if @rolling-robot can do it, it would be great. Ideally, it would be nice to add Arch to the official buildfarm for the next release.
I created a pull request https://github.com/ros-infrastructure/bloom/pull/409 to Bloom to add Arch support. In the meantime, I maintain a fork of all ROS release repositories https://github.com/gdlg-rosarch where I have generated the PKGBUILD for Kinetic.
I also wrote a quick-and-dirty script which can fetch, build and installed the packages from the release repositories.
Hi! I was trying to do a full installation of ROS Indigo but I have faced many issues concerning Qt4 (Rviz...), OpenCV2, GCC 6... So after patching and updating some packages, I have given up. @gdlg Is there a mean to use you work to install Kinetic on my machine ?
@romainreignier Have you tried to install Jade? It should be more up-to-date than Indigo and straightforward to install from AUR.
You can try to install Kinetic using my repositories. For that, you can use my installation script
To use it, you need to define the environment variable ROSDISTRO_INDEX_URL=https://raw.githubusercontent.com/gdlg-rosarch/rosdistro/master/index.yaml
beforehand.
However this script was built for testing and isn’t production-ready. I don’t update the repositories very often either.
For some Kinetic packages, you also need urdfdom 0.4.2 which is a little newer that the AUR version.
@gdlg Yes, you are right, Kinetic is more up-to-date so I have installed it with @bchretien arch-ros-stacks
and aap
from @v01d. It is a quite nice solution.
Everything went fine instead of the opencv3
build with GCC 6. I had to use the patch of the official Arch opencv3
package. And, for the building, I did not have any issue with urdfdom
but I have only installed desktop
so no Gazebo and did not tried yet to run anything yet.
I had launched the aap script before your answer that's why I did not use your solution but I will try later. Thanks.
@v01d With your nice aap
script, I had to use the option --all
, writing desktop
was not working (after running python2 import_catkin_packages.py --distro=kinetic --output-dir=kinetic/ -r desktop
in arch-ros-stacks
). The 198 packages were found but no target:
Searching for packages... 198 found
Building dependency list... done
Building packages...
Target desktop not found among detected packages
Hi, Aap expects package names, not directores, so you should use ros-kinetic-desktop
El 26/11/2016 7:15, "Romain Reignier" notifications@github.com escribió:
@gdlg https://github.com/gdlg Yes, you are right, Kinetic is more up-to-date so I have installed it with @bchretien https://github.com/bchretien arch-ros-stacks and aap from @v01d https://github.com/v01d. It is a quite nice solution. Everything went fine instead of the opencv3 build with GCC 6. I had to use the patch of the official Arch opencv3 package. And, for the building, I did not have any issue with urdfdom but I have only installed desktop so no Gazebo and did not tried yet to run anything yet. I had launched the aap script before your answer that's why I did not use your solution but I will try later. Thanks.
@v01d https://github.com/v01d With your nice aap script, I had to use the option --all, writing desktop was not working (after running python2 import_catkin_packages.py --distro=kinetic --output-dir=kinetic/ -r desktop in arch-ros-stacks). The 198 packages were found but no target:
Searching for packages... 198 found Building dependency list... done Building packages... Target desktop not found among detected packages
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-263055471, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ3qrIj_E_3h59Qb2jfLERlngZ5mtcfks5rCAbegaJpZM4I-QuY .
@v01d Ah, thanks for the clarification. It works now. You should update the Readme that mention:
./aap build < path to arch-ros-stacks clone >/kinetic desktop
Fixed!
On Sat, Nov 26, 2016 at 10:20 AM, Romain Reignier notifications@github.com wrote:
@v01d https://github.com/v01d Ah, thanks for the clarification. It works now. You should update the Readme that mention:
./aap build /kinetic desktop
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-263063171, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ3qh1yEMyDz-EXSTb32MSjxIeNYbxKks5rCDI5gaJpZM4I-QuY .
@v01d I have managed to install ros-kinetic-desktop. But I have given up with desktop-full because of some issue with libstage if I remember.
@gdlg I have tried your script. It works quite well. I had to install few dependencies manually and modify the rosconsole sources to add #include <vector>
in the file include/ros/console.h
as reported here: https://github.com/ros/ros_comm/pull/911
So thank you and keep working to deliver an easy to install Ros distribution to the Arch users!
@romainreignier Thank you very much for trying both script. I didn’t know about Aap before and it looks much better written than my script.
I updated my release repositories because in the latest version of ros_comm, the vector issue has been fixed. The whole update process is currently bit fiddly and those repositories are more for testing purposes than production.
Hi everyone. I dived a little bit into buildfarm deployment and set up a couple of VMs with jenkins instances. Yet, its porting from ubuntu is still not straightforward for me.
So I have established a repo intended mainly for ROS dependencies and packages that I or someone else can update manually. It is located here: https://mukhachev.me/archlinux/
and normally should be online 24/7.
If someone wants to keep it up to date with any of the packages while I am dealing with jenkins please feel free to email me at rolling.robot at gmail.com and I'll give you ssh access to the machine.
That’s great. I have also started adapting the buildfarm for Arch on a test server. My code is a bit of a mess but perhaps we can share the development work.
Currently, my version is able to:
However I have been struggling with AUR dependencies.
The buildfarm relies on a few dependencies such as python3-empy to generate Dockerfiles, python3-rosdistro and python3-rosdep and probably a few others. If we want to build those packages using Jenkins, we need a separate codepath because most of the buildfarm code is tied to empy, rosdistro and rosdep.
It makes sense to build those packages from the buildfarm because some dependencies such as pcl breaks when boost is updated. However a separate repo is a lot easier to create.
Can we use @v01d ’s https://github.com/ros-archlinux/ to store the sources? We would need to add python3-{rosdep,rosdistro,empy,rospkg,catkin-pkg} to it as well as bumping the version of urdfdom and urdfdom-headers to 0.4.2. Also the python2 and python3 packages conflict with each other and need to be built in a separate environment.
We also need to extend the buildfarm config to support Arch repository in addition to Ubuntu/Debian. My first approach was to add a field archlinux_repositories and archlinux_target_repository to the config. e.g. https://github.com/gdlg/ros_buildfarm_config/blob/rosarchfarm/kinetic/release-build.yaml but perhaps, we should have something more generic with a list of OSes.
Regarding using ros-archlinux, just let me know and I can transfer the organization to someone else. I'm currently unable to pursue this project, but I'm glad we got the ball rolling =)
On Fri, Dec 9, 2016 at 6:16 PM, Grégoire Payen de La Garanderie < notifications@github.com> wrote:
That’s great. I have also started adapting the buildfarm for Arch on a test server. My code is a bit of a mess but perhaps we can share the development work. My current work
Currently, my version is able to:
- Create Jenkins jobs for Arch
- Start Docker, add pacman repo to pacman.conf and keys and update the environment
- I have also some work on fetching sources and installing package dependencies
AUR dependencies
However I have been struggling with AUR dependencies.
- We can do what @rolling-robot https://github.com/rolling-robot suggest and have a separate repo
- or build them with Jenkins
The buildfarm relies on a few dependencies such as python3-empy to generate Dockerfiles, python3-rosdistro and python3-rosdep and probably a few others. If we want to build those packages using Jenkins, we need a separate codepath because most of the buildfarm code is tied to empy, rosdistro and rosdep.
It makes sense to build those packages from the buildfarm because some dependencies such as pcl breaks when boost is updated. However a separate repo is a lot easier to create.
Can we use @v01d https://github.com/v01d ’s https://github.com/ros- archlinux/ to store the sources? We would need to add python3-{rosdep,rosdistro,empy,rospkg,catkin-pkg} to it as well as bumping the version of urdfdom and urdfdom-headers to 0.4.2. Also the python2 and python3 packages conflict with each other and need to be built in a separate environment. Config file
We also need to extend the buildfarm config to support Arch repository in addition to Ubuntu/Debian. My first approach was to add a field archlinux_repositories and archlinux_target_repository to the config. e.g. https://github.com/gdlg/ros_buildfarm_config/blob/ rosarchfarm/kinetic/release-build.yaml but perhaps, we should have something more generic with a list of OSes.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bchretien/arch-ros-stacks/issues/57#issuecomment-266125462, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJ3qhP5XcV8coEV4e7OQXF55wJmS1Wsks5rGcUZgaJpZM4I-QuY .
@v01d Could you add @rolling-robot and myself to the organization?
I have added python-* packages to github.com/ros-archlinux and updated console-bridge, urdfdom and urdfdom-headers to the version required by Kinetic. I haven’t updated the rosarch branch of those repo and Travis.
A simple way of getting the dependencies working would be to modify the Travis script to upload packages to https://mukhachev.me/archlinux/ . Still, I think that it would be better to build the dependencies inside the buildfarm.
The ROS buildfarm build process is rather complex for Ubuntu.
I think that we could simplify the process for Arch by creating a Docker and then building inside a chroot and let makepkg install the dependencies inside the chroot. https://wiki.archlinux.org/index.php/DeveloperWiki:Building_in_a_Clean_Chroot
On the repo side, we need a main and a staging repository.
Hey, all. It's been a while, but I'm back to using ROS again. Props to @bchretien for making these wonderful helper scripts, they made things super easy.
I'm currently in the process of uploading all the kinetic packages I'm using to the AUR. Please feel free to test them, report any issues, and suggest any other packages that you'd like to see. My email for AUR-related stuff is aur
For the OpenCV thing, I simply replaced any instances of "ros-kinetic-opencv3" with "opencv" and that seemed to work fine. I'm not uploading ros-kinetic-opencv3 to the AUR. Please let me know if this breaks anything.
Anyone have time to start on this?