noetic outdated package

[AchmadFathoni]

Hi I have made python script that compare the noetic package version from installed AUR package and from ROS repo.

import requests
import gzip
import io
import subprocess
from debian_parser import PackagesParser #https://aur.archlinux.org/packages/python-debian-parser/
from pkg_resources import packaging

#Download data from ROS repository
url = 'http://repositories.ros.org/ubuntu/main/dists/focal/main/binary-amd64/Packages.gz'
file_gz = io.BytesIO(requests.get(url).content)
data = gzip.GzipFile(fileobj=file_gz).read().decode()

#Parse data from ROS repository
parser = PackagesParser(data)
raw = parser.parse()
ros_db = []
for package in raw:
    name = package[0].get('value')
    if 'dbgsym' not in name and 'noetic' in name:
        for field in package:
            if field.get('tag') == 'Version':
                version = field.get('value').split('-')[0]
                ros_db.append([name,version])
                break

#Get ros packages data from local pacman
proc1 = subprocess.Popen(['pacman', '-Qm'], stdout=subprocess.PIPE)
proc2 = subprocess.Popen(
    ['grep', 'ros-noetic'], stdin=proc1.stdout,
    stdout=subprocess.PIPE, stderr=subprocess.STDOUT
)
proc1.stdout.close()
out, err = proc2.communicate()
pacman_db = []
for package in out.decode().split('\n')[:-1]:
    name, version = package.split()
    version = version.split('-')[0]
    pacman_db.append([name,version])

#Compare both data
last_i=0
for name,current_version in pacman_db:
    for i in range(last_i, len(ros_db)):
        if ros_db[i][0] == name:
            current_version = packaging.version.parse(current_version)
            new_version = packaging.version.parse(ros_db[i][1])
            if(current_version < new_version):
                print(name, current_version, new_version)
            last_i = i
            break

This is output of that program. I hope you update your ROS noetic package that you maintained in AUR. I don't mind if you let me be the AUR maintainer.

ros-noetic-actionlib-msgs 1.13.0 1.13.1
ros-noetic-actionlib-tutorials 0.1.12 0.2.0
ros-noetic-bond 1.8.5 1.8.6
ros-noetic-bond-core 1.8.5 1.8.6
ros-noetic-bondcpp 1.8.5 1.8.6
ros-noetic-bondpy 1.8.5 1.8.6
ros-noetic-camera-calibration 1.15.2 1.15.3
ros-noetic-catkin 0.8.8 0.8.9
ros-noetic-common-msgs 1.13.0 1.13.1
ros-noetic-common-tutorials 0.1.11 0.2.0
ros-noetic-control-msgs 1.5.0 1.5.2
ros-noetic-controller-interface 0.19.1 0.19.4
ros-noetic-controller-manager 0.19.1 0.19.4
ros-noetic-controller-manager-msgs 0.19.1 0.19.4
ros-noetic-cpp-common 0.7.1 0.7.2
ros-noetic-depth-image-proc 1.12.23 1.15.3
ros-noetic-diagnostic-aggregator 1.10.2 1.10.3
ros-noetic-diagnostic-analysis 1.10.2 1.10.3
ros-noetic-diagnostic-common-diagnostics 1.9.3 1.10.3
ros-noetic-diagnostic-msgs 1.13.0 1.13.1
ros-noetic-diagnostic-updater 1.10.2 1.10.3
ros-noetic-diagnostics 1.10.2 1.10.3
ros-noetic-diff-drive-controller 0.15.0 0.18.1
ros-noetic-dynamic-reconfigure 1.7.0 1.7.1
ros-noetic-executive-smach 2.0.1 2.5.0
ros-noetic-filters 1.8.1 1.9.1
ros-noetic-forward-command-controller 0.15.0 0.18.1
ros-noetic-gennodejs 2.0.1 2.0.2
ros-noetic-genpy 0.6.10 0.6.14
ros-noetic-geometry 1.12.0 1.13.2
ros-noetic-geometry-msgs 1.13.0 1.13.1
ros-noetic-geometry-tutorials 0.2.2 0.2.3
ros-noetic-gl-dependency 1.1.0 1.1.2
ros-noetic-hardware-interface 0.19.1 0.19.4
ros-noetic-image-pipeline 1.14.0 1.15.3
ros-noetic-image-proc 1.14.0 1.15.3
ros-noetic-image-publisher 1.15.2 1.15.3
ros-noetic-image-rotate 1.12.23 1.15.3
ros-noetic-image-view 1.15.2 1.15.3
ros-noetic-interactive-marker-tutorials 0.10.3 0.11.0
ros-noetic-joint-limits-interface 0.15.1 0.19.4
ros-noetic-joint-state-controller 0.15.0 0.18.1
ros-noetic-kdl-parser 1.14.0 1.14.1
ros-noetic-kdl-parser-py 1.14.0 1.14.1
ros-noetic-laser-geometry 1.6.4 1.6.5
ros-noetic-laser-pipeline 1.6.3 1.6.4
ros-noetic-librviz-tutorial 0.10.3 0.11.0
ros-noetic-map-msgs 1.13.0 1.14.1
ros-noetic-media-export 0.2.0 0.3.0
ros-noetic-message-filters 1.15.4 1.15.9
ros-noetic-mk 1.15.2 1.15.7
ros-noetic-nav-msgs 1.13.0 1.13.1
ros-noetic-nodelet-tutorial-math 0.1.11 0.2.0
ros-noetic-pcl-conversions 1.7.0 1.7.2
ros-noetic-pcl-msgs 0.2.0 0.3.0
ros-noetic-perception 1.4.1 1.5.0
ros-noetic-perception-pcl 1.7.0 1.7.2
ros-noetic-position-controllers 0.15.0 0.18.1
ros-noetic-qt-dotgraph 0.3.16 0.4.2
ros-noetic-qt-gui 0.4.1 0.4.2
ros-noetic-qt-gui-py-common 0.3.16 0.4.2
ros-noetic-qwt-dependency 1.1.0 1.1.1
ros-noetic-realtime-tools 1.16.0 1.16.1
ros-noetic-resource-retriever 1.12.5 1.12.6
ros-noetic-robot 1.4.1 1.5.0
ros-noetic-robot-state-publisher 1.14.0 1.15.0
ros-noetic-ros 1.15.2 1.15.7
ros-noetic-ros-base 1.4.1 1.5.0
ros-noetic-ros-comm 1.15.4 1.15.9
ros-noetic-ros-core 1.4.1 1.5.0
ros-noetic-ros-environment 1.3.1 1.3.2
ros-noetic-rosbag 1.15.7 1.15.9
ros-noetic-rosbag-storage 1.15.7 1.15.9
ros-noetic-rosbash 1.15.6 1.15.7
ros-noetic-rosboost-cfg 1.15.2 1.15.7
ros-noetic-rosbuild 1.15.2 1.15.7
ros-noetic-rosclean 1.15.6 1.15.7
ros-noetic-rosconsole 1.14.1 1.14.3
ros-noetic-roscpp-core 0.7.1 0.7.2
ros-noetic-roscpp-serialization 0.7.1 0.7.2
ros-noetic-roscpp-traits 0.7.1 0.7.2
ros-noetic-roscpp-tutorials 0.9.1 0.10.2
ros-noetic-roscreate 1.15.6 1.15.7
ros-noetic-rosgraph 1.15.4 1.15.9
ros-noetic-roslang 1.15.6 1.15.7
ros-noetic-roslaunch 1.15.4 1.15.9
ros-noetic-roslib 1.15.2 1.15.7
ros-noetic-roslint 0.11.2 0.12.0
ros-noetic-roslz4 1.15.7 1.15.9
ros-noetic-rosmake 1.15.2 1.15.7
ros-noetic-rosmaster 1.15.4 1.15.9
ros-noetic-rosmsg 1.15.7 1.15.9
ros-noetic-rosnode 1.15.4 1.15.9
ros-noetic-rosout 1.15.4 1.15.9
ros-noetic-rospack 2.6.1 2.6.2
ros-noetic-rosparam 1.15.4 1.15.9
ros-noetic-rospy-tutorials 0.9.1 0.10.2
ros-noetic-rosservice 1.15.4 1.15.9
ros-noetic-rostest 1.15.7 1.15.9
ros-noetic-rostime 0.7.1 0.7.2
ros-noetic-rostopic 1.15.4 1.15.9
ros-noetic-rosunit 1.15.2 1.15.7
ros-noetic-roswtf 1.15.7 1.15.9
ros-noetic-rqt-bag 0.4.13 0.5.0
ros-noetic-rqt-bag-plugins 0.4.12 0.5.0
ros-noetic-rqt-common-plugins 0.4.8 0.4.9
ros-noetic-rqt-console 0.4.8 0.4.11
ros-noetic-rqt-dep 0.4.9 0.4.10
ros-noetic-rqt-gui 0.5.0 0.5.2
ros-noetic-rqt-gui-cpp 0.5.0 0.5.2
ros-noetic-rqt-gui-py 0.5.0 0.5.2
ros-noetic-rqt-image-view 0.4.15 0.4.16
ros-noetic-rqt-launch 0.4.8 0.4.9
ros-noetic-rqt-logger-level 0.4.8 0.4.11
ros-noetic-rqt-moveit 0.5.7 0.5.9
ros-noetic-rqt-msg 0.4.8 0.4.9
ros-noetic-rqt-plot 0.4.9 0.4.12
ros-noetic-rqt-pose-view 0.5.8 0.5.10
ros-noetic-rqt-publisher 0.4.8 0.4.9
ros-noetic-rqt-py-common 0.5.0 0.5.2
ros-noetic-rqt-py-console 0.4.8 0.4.9
ros-noetic-rqt-robot-dashboard 0.5.7 0.5.8
ros-noetic-rqt-robot-monitor 0.5.9 0.5.13
ros-noetic-rqt-robot-plugins 0.5.7 0.5.8
ros-noetic-rqt-robot-steering 0.5.9 0.5.12
ros-noetic-rqt-runtime-monitor 0.5.7 0.5.8
ros-noetic-rqt-rviz 0.6.0 0.6.1
ros-noetic-rqt-service-caller 0.4.8 0.4.9
ros-noetic-rqt-shell 0.4.9 0.4.10
ros-noetic-rqt-tf-tree 0.6.0 0.6.2
ros-noetic-rqt-top 0.4.8 0.4.9
ros-noetic-rqt-web 0.4.8 0.4.9
ros-noetic-rviz-plugin-tutorials 0.10.3 0.11.0
ros-noetic-rviz-python-tutorial 0.10.3 0.11.0
ros-noetic-self-test 1.10.2 1.10.3
ros-noetic-sensor-msgs 1.13.0 1.13.1
ros-noetic-shape-msgs 1.13.0 1.13.1
ros-noetic-simulators 1.4.1 1.5.0
ros-noetic-smach 2.0.1 2.5.0
ros-noetic-smach-msgs 2.0.1 2.5.0
ros-noetic-smach-ros 2.0.1 2.5.0
ros-noetic-smclib 1.8.5 1.8.6
ros-noetic-std-srvs 1.11.2 1.11.3
ros-noetic-stereo-image-proc 1.15.2 1.15.3
ros-noetic-stereo-msgs 1.13.0 1.13.1
ros-noetic-tf 1.13.1 1.13.2
ros-noetic-tf2 0.7.1 0.7.5
ros-noetic-tf2-eigen 0.7.2 0.7.5
ros-noetic-topic-tools 1.15.4 1.15.9
ros-noetic-trajectory-msgs 1.13.0 1.13.1
ros-noetic-transmission-interface 0.15.1 0.19.4
ros-noetic-turtle-actionlib 0.1.11 0.2.0
ros-noetic-turtle-tf 0.2.2 0.2.3
ros-noetic-turtle-tf2 0.2.2 0.2.3
ros-noetic-turtlesim 0.9.1 0.10.2
ros-noetic-urdf 1.13.1 1.13.2
ros-noetic-urdf-parser-plugin 1.13.1 1.13.2
ros-noetic-urdf-sim-tutorial 0.4.0 0.5.1
ros-noetic-urdf-tutorial 0.4.0 0.5.0
ros-noetic-urdfdom-py 0.4.0 0.4.5
ros-noetic-visualization-marker-tutorials 0.10.3 0.11.0
ros-noetic-visualization-msgs 1.13.0 1.13.1
ros-noetic-visualization-tutorials 0.10.3 0.11.0
ros-noetic-viz 1.4.1 1.5.0
ros-noetic-webkit-dependency 1.1.0 1.1.2
ros-noetic-xmlrpcpp 1.15.8 1.15.9

[acxz]

Hey @AchmadFathoni glad you are interested in this!

Hi I have made python script that compare the noetic package version from installed AUR package and from ROS repo.

We actually already have scripts that do the same thing over at the ros-arch github org. This github org is specific for ros-noetic on arch while the other one is for all ros related scripts, discussions, etc. Check it out here: https://github.com/ros-arch. This might the most similar to what you have just created: https://github.com/ros-arch/arch_ros_package_monitor

I hope you update your ROS noetic package that you maintained in AUR.

Its a lot of work and there is actually some work on automating the update path. See: https://github.com/ros-arch/ros-aur-helpers I haven't look at this closely yet so haven't gotten around to doing it.

I don't mind if you let me be the AUR maintainer.

Right now our development process is create PRs to github and then syncing to the AUR. Before I would transfer ownership of all ros-noetic AUR to you, I would like to see you create PRs, get familiar, and help out the project. Since our dev process includes github, I actually don't need to give you AUR maintainership for you to help out in updating these packages.

I'll still keep this issue open as technically this is an issue for updating out of date noetic packages. Feel free to submit PRs to the respecitive repos upgrading them, and I'll sync them to the AUR in a jiffy!

[AchmadFathoni]

Thank you. I did not know that ros-arch project exist. Anyway what method do you use to bump the package version on AUR? Using script or manually by hand?

[acxz]

You should be able to do it automatically with: https://github.com/ros-arch/ros-aur-helpers

I've been doing it manually for the small ones, but I'll script for a big update (been a long time since I've done that)

[AchmadFathoni]

A month have passed, Everytime I rebuild my AUR package with yay's --rebuildtree, it downloads your outdated repository. Do you find any difficulty in upgrading those packages?

[acxz]

Sorry man, things have just been piling on me and I need to catch up on quite a few stuff. I can't really promise on when I'll get to this (at least before the semester ends for me)

[AchmadFathoni]

OK, good luck on your study. For me, I will graduate on 11 April.

[AchmadFathoni]

How about I give you 5 pull request of package update per day?

[AchmadFathoni]

I have made all the pull request today by using python script. If you really don't have time to maintain this then I suggest to orphan the AUR repo.

[bionade24]

@acxz I now merged als PRs, you still have to sync them. Don't be that mad and try to merge them all manually. After I looked at some and was sure they're made by rosaur update <package>, I grabbed a Jupyter terminal and used PyGithub to merge them.

[AchmadFathoni]

@acxz can I just be a co-maintainer and sync them myself? I am on long vacation and have too much free time.

[bionade24]

@AchmadFathoni Syncing everything shouldn't be mre work than adding you as co-maintainer in AUR. If you really want to make ROS on Archlinux sustainable better, please consider enhacing the official ROS buildfarm with Arch Linux, so that Arch can apply as an officially supported Distro.

[AchmadFathoni]

@AchmadFathoni Syncing everything shouldn't be mre work than adding you as co-maintainer in AUR. If you really want to make ROS on Archlinux sustainable better, please consider enhacing the official ROS buildfarm with Arch Linux, so that Arch can apply as an officially supported Distro.

I will consider it for ROS2. Unrealistic to think they will officially support a bleeding edge rolling release distro.

[AchmadFathoni]

@AchmadFathoni Syncing everything shouldn't be mre work than adding you as co-maintainer in AUR.

Do you mean syncing is easier than adding me as co-maintainer? If it's that easy then why it's not done already?

[bionade24]

I will consider it for ROS2. Unrealistic to think they will officially support a bleeding edge rolling release distro.

That's what OSRF said: https://discourse.ros.org/t/adding-archlinux-to-the-buildfarm/16500

Do you mean syncing is easier than adding me as co-maintainer? If it's that easy then why it's not done already?

It both needs some simple scripting with the AUR ssh interface, so it's likely equal.

[AchmadFathoni]

It both needs some simple scripting with the AUR ssh interface, so it's likely equal.

Is that script has never been made? What the maintainer of kinetic, melodic, and noetic use to sync their AUR before?

[bionade24]

Several tools exist.

Am Donnerstag, 22. April 2021 schrieb Achmad Fathoni:

It both needs some simple scripting with the AUR ssh interface, so it's likely equal.

Is that script has never been made? What the maintainer of kinetic, melodic, and noetic use to sync their AUR before?

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/ros-noetic-arch/ros-noetic-desktop-full/issues/20#issuecomment-82456713

[bionade24]

You could have at least noted that you took maintainership of the outdated packages. I hope we can find a sustainable, future-proof solution for noetic, like we have with melodic with our CI mirror successfull builds automatically to AUR, being accessable to more than 1 person.

[AchmadFathoni]

What prevents you from apply that CI to noetic before this issue?