search

ros / rosdistro

This repo maintains a lists of repositories for each ROS distribution
Other
913 stars 2.54k forks source link

Mass bug: architecture_independent tag verification #4037

Closed cottsay closed 3 years ago

cottsay commented 10 years ago

With the addition of noarch support for RPMs in bloom (https://github.com/ros-infrastructure/bloom/pull/270), I found a couple of packages that incorrectly marked architecture_independent when they were, in fact, not so. These packages were notified and the appropriate package.xmls updated.

Inversely, there are MANY packages that do not mark architecture_independent when they should. This is mostly because the current debian buildfarm is unable to leverage this.

My experimental RPM buildfarm (http://csc.mcs.sdsmt.edu/jenkins/) has the capability to both utilize this and to check when it is missing, so I'm compiling a list of packages that should be updated.

Hopefully this will act as a first step in implementing this functionality on the debian buildfarm, and to produce smaller, more accurate builds on the RPM buildfarm.

Thanks!

Originally https://github.com/ros/catkin/issues/623

Package List (work-in-progress):

After compiling the list, I've decided that there are just too many packages to target all of Hydro. Instead, I'll try to catch Indigo packages as they come in.

This is a preliminary list. Some of these may be false positives.

tfoote commented 10 years ago

@cottsay Is there a way we could work this sort of check into roslint to make it easier for maintainers to detect? Can we run the tool in the installation directory instead of an rpm? I'm assuming it's applying some heuristic for file types and locations?

cottsay commented 10 years ago

Actually @tfoote, the way I checked this was by looking at the generated debuginfo RPM. If it was empty, then there were no ELFs to extract debugging info from, usually indicating that the package is noarch. The alternative is that the package wasn't compiled with the flags provided by rpmbuild, but I passed those along in the .spec file, so it shouldn't be the case.

I'm not sure how to tell beyond that, but you're right, this would be a great thing to include in roslint.

Since there are quite a few packages on this list, I'll probably invoke the help of some minions here at school to help with the testing, forking, and PR-ing process.

jack-oquin commented 10 years ago

@tfoote: roslint checks C++ and Python files. This would be more in the nature of catkin_lint.

Glad to see this is finally being used.

jack-oquin commented 10 years ago

I presume you want this tag added to metapackages, also.

Are you packaging them?

cottsay commented 10 years ago

@jack-oquin I've been assuming that metapackages are noarch regardless of if they mark architecture_independent, so no, I don't think metapackages need to be marked.

By them do you mean metapackages? If so, yes: http://csc.mcs.sdsmt.edu/ros/rpmbuild/hydro.html http://csc.mcs.sdsmt.edu/ros/rpmbuild/indigo.html

EDIT: In the end, I'd like to remove the metapackages and start using RPM groups instead, but I'm not sure how things would work with metapackages depending on metapackages...

wjwwood commented 10 years ago

Thanks for all your work on this @cottsay, this is definitely a big effort item.

jack-oquin commented 10 years ago

released ackermann_msgs 0.9.1 (jack-oquin/ackermann_msgs#3)

jack-oquin commented 10 years ago
mikaelarguedas commented 7 years ago

@cottsay Is this still of interest ? Is the aforementioned fedora buildfarm still active (for testing, if we keep adding the architecture independant tag to packages) ?

cottsay commented 7 years ago

The Fedora buildfarm is not currently active. Maybe I'll have cycles to get things running again some day, but not now.

jack-oquin commented 7 years ago

I recently noticed a ROS2 pull request to remove <architecture-independent/> from one of the unique_identifier packages.

Is there opposite motion going on?

mikaelarguedas commented 7 years ago

While this tag is used only in the RPM metadata generated by bloom, I think it's valuable information and maybe the ROS buildfarm will leverage it some day. So we do not encourage users to get rid of it in ROS2.

One thing to keep in mind though is that ROS2 message packages do compile code and libraries so cannot be considered architecture-independent anymore. Only pure python packages (build type ament_python in the ROS2 world) and header-only packages can be considered architecture independent.

tfoote commented 5 years ago

@cottsay Maybe this would be worth returning to briefly with your current work. I audited all the ones with PRs open and was able to check most of them off as merged.

cottsay commented 5 years ago

I'll start by working through the list as it was 4 years ago, but I expect there have been new packages which need to be discovered. Since we don't have an RPM buildfarm active at the moment, I may have to come up with an alternate bulk discovery method.

Since the ROS2 package list is more containable right now, I might go after those to get ahead of the problem.

tfoote commented 3 years ago

@cottsay We've turned on RPM builds on ROS 2 rolling and we've shipped Noetic already which is the last ROS 1. To that end this seems like it should be caught now in rolling and we're not likely to turn on for Noetic so can we close out this long running ticket as enough for now?

cottsay commented 3 years ago

this seems like it should be caught now in rolling

The new buildfarm code doesn't have support for RPM linting yet, so we won't catch packages that are missing the tag but should have it. We'll still catch packages that claim to be architecture-independent but actually aren't.

I have a hunch that ROS 2 is in pretty rough shape as far as this flag is concerned. There are advantages to doing noarch right, but we still don't have any capability in the buildfarm to build noarch packages only once, so we can't take full advantage of it even if it's declared correctly in the packages.

I think we should close this bug now - it was intended for tracking the feature in ROS 1, and like you said, it's unlikely that we'll ever see Noetic RPMs on the buildfarm. We can always re-open it if things change.