Closed ivan1993br closed 3 years ago
tfoote
commented
4 years ago For history on the imported version see: https://github.com/ros-perception/perception_pcl/pull/124 and https://github.com/ros-infrastructure/reprepro-updater/issues/32
The changelog looks like exactly what we patched in and was contributed by @kyrofa and was the same minimal change thus I would have hoped that the new upstream package would be a good replacement.
kyrofa
commented
4 years ago As far as I understand it, it may not quite be the same. I believe you followed the Debian reversion back to the vtk-vendored libproj, right? We can't do that in Ubuntu as it would have broken ABI, so I tried to fix it by instead pushing all the way through to using the system libproj by adding the dep and rebuilding libpcl to have the proper library in its cmake config files. In a way I kinda broke your ABI by... not breaking ABI :thinking: . And then the pcl rebuild is the cherry on top. More info here. @tfoote we can set up a time to chat tomorrow if you like.
ivan1993br
commented
4 years ago So, if I understood correctly, we used a version of the pcl release that used the vtk-vendored libproj, the recent update on PCL fix the ABI and uses the system libproj. I was able to "fix" just setting the vtk6, using the system libpcl, because the vtk-vendored libproj is dangling there and not being used by pcl, just linked and not used on catkin packages.
We have two options for now: new release on vtk6 and pcl on top of the new one, or rebuild all ros packages dependent on PCL and their dependencies.
What about just releasing a new vtk6, but leave pcl as it is, leaving vtk-vendored libproj dangling? Too fragile? This would allow to update the packages libraries as we go. Does it make sense?
kyrofa
commented
4 years ago Yeah, to clarify, the state of Ubuntu before the bugfix was:
As a result, anyone using libvtk6 ended up with errors because they were using a library that required libproj to be installed, but it didn't depend on it (bug 1). Also, anyone using cmake to link to pcl ended up with build errors because they were trying to link to a library that didn't exist (bug 2). It was just all-around broken.
The fix that Open Robotics got into their Debian archive was to modify libvtk6 to start building and using its vendored libproj again, which made the libpcl in Xenial work because it could find that vtkproj4 library again. However, the vendored libproj is not ABI compatible with the system libproj, so that's not something I could safely make happen in Xenial. I instead made libvtk6 properly depend upon libproj, and then rebuilt libpcl so it didn't have a library that didn't exist in its cmake config.
We have two options for now: new release on vtk6 and pcl on top of the new one, or rebuild all ros packages dependent on PCL and their dependencies.
That's my thinking anyway, yes. Right now things like pcl_rosConfig.cmake and costmap_2dConfig.cmake contain vtkproj4. The possible fixes are:
What about just releasing a new vtk6, but leave pcl as it is, leaving vtk-vendored libproj dangling? Too fragile? This would allow to update the packages libraries as we go. Does it make sense?
No, I think that will break. It's exactly the opposite situation of the breakage that was in Xenial before this happened:
I suspect that will be one big segfault for anyone trying to use libpcl.
tfoote
commented
4 years ago I just talked with @kyrofa and since the upstream libvtk and libpcl are both now consistent the simplest solution is to do the general rebuild of Kinetic to use the consistent libproj. Although it's not an ABI break, the solution is similar, we will rebuild all packages in Kinetic depending on libvtk6 and anyone with compiled binary distributions will have to do the same. This will take some server time but will provide the simplest solution in the long run.
Process:
[x] rebuild all kinetic packages on top of the new libvtk6
Dependencies of libvtk6-dev
Dependencies of libpcl-dev
Selection listings:
nuclearsandwich
commented
4 years ago I'm in favor of the above plan. Do we want to scrub the vendored vtk from our repositories once it is no longer needed or leave it up for the forensics of others?
ivan1993br
commented
4 years ago Thank @tfoote!
For those in need of a temporary "fix" now (for those who pinning will no be functional) ppa. (It may be removed after the binary distribution)
Currently only with vtk6, I executed integration tests on the robot (it has some custom layers, sonar pcl), and did not break, but as pointed @kyrofa the ABI is not compatible. I do not know if we are missing something.
The ideas was just to do some tests with this partial fix, and then push the PCL, but launchpad today was rather slow for some reason.
tfoote
commented
4 years ago As a temporary fix for now I would recommend just pinning back the version of vtk as outlined on this issue: https://github.com/ros-perception/perception_pcl/issues/318
Do we want to scrub the vendored vtk from our repositories once it is no longer needed or leave it up for the forensics of others?
After this patch is out it would no longer be necessary for anything rebuilt, but I'd suggest leaving it in case someone needs to pin it for a distributed binary from before the patch.
kyrofa
commented
4 years ago I'd suggest leaving it in case someone needs to pin it for a distributed binary from before the patch.
Seconded. No harm in leaving other than space, but definitely a potential benefit, there.
tfoote
commented
4 years ago I'm seeing a problem of the rebuild on armhf: http://build.ros.org/view/Ksrc_uX/job/Kbin_uxhf_uXhf__pcl_ros__ubuntu_xenial_armhf__binary/16/console
Console error Error: *** No rule to make target '/usr/lib/arm-linux-gnueabihf/hdf5/openmpi/lib/libhdf5.so', needed by 'devel/lib/libpcl_ros_tf.so'. Stop.
kyrofa
commented
4 years ago That's from perception_pcl/pcl_ros? It builds locally, it sounds like a dep hasn't been rebuilt yet. I think it's related to the dependency your patched version had on libhdf5-mpi-dev (which pulls in libhdf5-openmpi-dev) that the one in Xenial does not have:
$ apt-cache policy libvtk6.2
libvtk6.2:
Installed: 6.2.0+dfsg1-10ubuntu0.1
Candidate: 6.2.0+dfsg1-10ubuntu0.1
Version table:
*** 6.2.0+dfsg1-10ubuntu0.1 500 # <- Xenial's updated version
500 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages
100 /var/lib/dpkg/status
6.2.0+dfsg1-10build1+debian11.1+osrf1 500 # <- OR's version
500 http://packages.ros.org/ros/ubuntu xenial/main amd64 Packages
6.2.0+dfsg1-10build1 500 # <- Old, broken version
500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
# Old version:
$ apt-cache depends libvtk6.2=6.2.0+dfsg1-10build1 | grep mpi
Depends: libopenmpi1.10
Suggests: mpi-default-bin
$ apt-cache depends libvtk6-dev=6.2.0+dfsg1-10build1 | grep mpi
Depends: mpi-default-dev
# Xenial's (updated) version:
$ apt-cache depends libvtk6.2 | grep mpi
Depends: libopenmpi1.10
Suggests: mpi-default-bin
$ apt-cache depends libvtk6-dev | grep mpi
Depends: mpi-default-dev
# OR's version:
$ apt-cache depends libvtk6.2=6.2.0+dfsg1-10build1+debian11.1+osrf1 | grep mpi
Depends: libhdf5-openmpi-10
Depends: libopenmpi1.10
Suggests: mpi-default-bin
$ apt-cache depends libvtk6-dev=6.2.0+dfsg1-10build1+debian11.1+osrf1 | grep mpi
Depends: libhdf5-mpi-dev
Depends: mpi-default-devIf pcl_ros depends upon it, it should be explicit, but I suspect it's coming from another cmake config file somewhere.
ros-discourse
commented
3 years ago This issue has been mentioned on ROS Discourse. There might be relevant details there:
https://discourse.ros.org/t/preparing-for-kinetic-sync-2020-12-03/17739/1
tfoote
commented
3 years ago Ahh after some investigation we are hosting a backport of libpcl for armhf only that has the dependency on vtk6.2 and thus needs to be rebuilt to clear the cached cmake config files passing through the vtk dependencies. http://repositories.ros.org/ubuntu/building/dists/xenial/main/binary-armhf/Packages
The backports are pulled here https://launchpad.net/~tully.foote/+archive/ubuntu/backports/+packages
Which was a workaround for #11583
It looks like the backports will need to be rebuilt and reimported.
kyrofa
commented
3 years ago Which was a workaround for #11583
@tfoote that's https://bugs.launchpad.net/ubuntu/+source/pcl/+bug/1906277 . It's fixed in -proposed and is in the 7-day waiting period, should be released to -updates on the 10th. A rebuild of your patched version should solve this, but I wanted to make sure you knew.
tfoote
commented
3 years ago Thanks for the heads up. I'd much rather use the upstream one. With everything broken right now I don't want to wait a full week, or the remainder of the week. We should be able to import the proposed packages by updating this url:
So we don't diverge again from upstream and require ongoing maintenance.
I see the packages here: http://ports.ubuntu.com/dists/xenial-proposed/universe/binary-armhf/
kyrofa
commented
3 years ago Yeah sounds perfect.
tfoote
commented
3 years ago With the xenial-proposed packages for armhf we're now approaching a complete rebuild. I expect it will be complete in the morning.
ros-discourse
commented
3 years ago This issue has been mentioned on ROS Discourse. There might be relevant details there:
https://discourse.ros.org/t/new-packages-for-kinetic-kame-2020-12-07/17786/1
tfoote
commented
3 years ago
Hi guys,
There was a new release of
libvtk6-devon 2020-11-30 on ubuntu xenial, that conflicts with the osrf one, and in consequence withros-kinetic-pcl-rospackage.The osrf release for vtk6 is
6.2.0+dfsg1-10build1+debian11.1+osrf1.As the most updated" version now, is the
ubuntu updatesone we are not installing the osrf version.Can we have an update on this lib?
Related pcl issue
Thanks!