search

roboticslab-uc3m / questions-and-answers

A place for general debate and question&answer
https://robots.uc3m.es/developer-manual/appendix/repository-index.html
2 stars 0 forks source link

Set a minimum version policy for dependencies at a global or project-wide level #7

Closed PeterBowman closed 2 years ago

PeterBowman commented 7 years ago

There seems to be an unwritten rule about always developing against the latest version of YARP. After having read robotology/QA#173, I concluded that we may benefit from defining such policies as projects grow larger and more people get involved in their development. I perceive this as a bit of a pressing matter since several of my past commits were mostly aimed at dealing with deprecations, code no longer working due to certain upgrade and achieving backwards compatibility where possible. For example, I'm unsure about the minimum version of CMake I should be targeting at and in consequence I will miss some of the newer features. Additionally, it's not clear to which extent hacks (sometimes dirty ones) must be applied to support outdated software.

The YARP team has a nice page that covers all dependencies: http://wiki.icub.org/wiki/YARP_Supported_Distributions

Seeing this from another perspective, care should also be taken to conform to the latest releases of upstream software - in case such policy applies, too (as stated in the first sentence, YARP could be a candidate in this respect). To list some of these issues, developers should be aware of incoming deprecations (see roboticslab-uc3m/teo-body@efb8962) and keep their documentation up-to-date (roboticslab-uc3m/installation-guides#1).

PeterBowman commented 7 years ago

An example of a breaking change in YARP, should we conform to the always-latest-release-policy: the new EXTRA_CONFIG parameter in 2.3.68.

jgvictores commented 7 years ago

I agree on the fact that this is an important issue.

Where would this go? Perhaps on an individual file in the root of installation-guides?

Alternatives would include https://github.com/roboticslab-uc3m/best-practices and perhaps a developer manual as suggested in https://github.com/roboticslab-uc3m/QA/issues/12, but maybe there is where it can have most visibility.

David-Estevez commented 7 years ago

If I understood correctly, installation-guides only specifies how a dependency should be installed, and then each repo specifies somewhere within that repo which dependencies are needed for that concrete repo.

In that case, the minimum version should be specified along each dependency within that repo. Note that several entries for the same dependency might be required in installation-guides, one per version.

PeterBowman commented 7 years ago

Another YARP breaking change: roboticslab-uc3m/teo-body@b342044.

PeterBowman commented 7 years ago

In a recent meeting, @jgvictores and I concluded that we could just specify the minimum version for each dependency along its name on each repo's installation guide (see David's comment), dropping the idea of a master table on a separate .md file that would hold this information. That is, for teo-body/doc/teo_body_install_on_debian_6.md#install-the-software, this would simply result in:

(version numbers were chosen only for illustrative purposes) The final format may vary (I'd rather use parentheses and no v prefix, but in this case it just didn't fit otherwise).

jgvictores commented 7 years ago

Okay, I've been populating teo_body_install_on_debian_6.md.

Regarding YARP, here are the facts:

PS: I'll be working with 2.3.68 this morning to see if the analogServer is working correctly.

David-Estevez commented 7 years ago

I think we should be working with the gitflow in our minds:

This, of course, has the problem of maintaining two software versions with their dependencies in the robot. There are many choices to solve this problem, although all of them might not be suitable for Teo:

PeterBowman commented 7 years ago

I'm with option 3, too. @David-Estevez please keep in mind that the ASIBOT's PC is actually a Roboard (RB-110, https://www.robot-r-us.com/vmchk/microcontroller/roboard-rb-110-kit.html). We are nearly out of space ATM for installing additional software (Docker/WMs weigh too much), and I'm not sure yet we can go past the current 8 GB memory size for the Micro-SD card.

David-Estevez commented 7 years ago

@PeterBowman What about two Micro-SD cards, one for develop and other for master? Is that feasible or too crazy?

jgvictores commented 7 years ago

@David-Estevez We've tried to implement this on TEO (see https://github.com/roboticslab-uc3m/teo-body/issues/43) but haven't been doing this, it just turns out not to be very practical.

PS: Regarding YARP version on teo-body, I've started https://github.com/roboticslab-uc3m/teo-body/issues/89.

PeterBowman commented 7 years ago

@David-Estevez there is only one slot, and the USB is already used by the WiFi adapter. I also forgot to mention that ASIBOT's casing is quite hard to remove, that means that any hardware change (even as simple as changing the SD) would require some work and attention to:

  1. undock the robot arm
  2. unscrew the casing, do whatever we have to do
  3. take the utmost care for not leaving any loose wire connections
David-Estevez commented 7 years ago

I see, it seems that working with robots is a little bit more difficult than working with PCs :sweat_smile:

jgvictores commented 7 years ago

A quite complex repo in terms of installation and versions that has been updated is the current teo-body.

Can we conclude this is the accepted way to document dependencies and their versions, and can close the issue once the following two steps are done?

David-Estevez commented 7 years ago

@jgvictores automated installation tools means apt-get, pip, or similar, right?

jgvictores commented 7 years ago

@David-Estevez Yes, I meant apt-get etc

David-Estevez commented 7 years ago

So, if I have understood correctly:

jgvictores commented 7 years ago

The installation-guides are a fallback for cases where a dependency cannot be installed/configured with a simple apt-get install or pip install.

Please note there was a misunderstanding here. I was referring to the case where the entire repository or all its dependencies could be simultaneously compiled/installed with a single line (via some automated script, PPA or similar).

However, having an automated script should not exclude a best practice of documenting how to install dependencies and compile/install from source.

For dependencies, I still support creating a new page on roboticslab-uc3m/installation-guides. There are several reasons for this:

  1. Even looking at different versions of a specific distro, a package can disappear or its name can change.
  2. We can easily end up recommending different ways to install the same package, ending up in multiple possibly conflicting installations. Giving visibility to what each project uses on a shared page can really help avoid this bad situation.

Each repo can contain the dependencies expressed as dependency_name=version (with = or >=), a command to install them all in case an automated tool is available and some links to the guides in the case the installation process is more complex than that.

  1. We were looking a slight different syntax: [CMake v2.8.12+ (Debian 6.0.10)]()
  2. A >= is dangerous, given that maintainers too often introduce changes that break bw-compat. I'd support the assumption that it's a use at your own risk if using a != version.

Other than being tedious (and the trapped in a tunnel, OMG I forgot to download uc3m/installation-guides scenario in which case you can't install the dependencies anyway), do you think this is reasonable?

PeterBowman commented 7 years ago

@David-Estevez I was told that the textiles project is particularly hard to document because of a huge number of dependencies. I wonder if said documenting process could be, at least partially, supplanted by the requirements.txt file. I stumbled upon a more complex case at the pywikibot framework which might interest you, especially concerning dependency version ranges and python 2.x/3.x distinction:

flickrapi>=1.4.5,<2 ; python_version < '2.7'

BTW another important change in YARP, I wonder if it could break anything on our end: robotology/QA#187.

David-Estevez commented 7 years ago

@PeterBowman That is something I am still considering. But there are some dependencies that require certain development version or compiling one specific commit of the required library to have some cutting-edge functionality. The project also involves C++, so CMake is required at some point to compile the C++ parts, so this has to be taken into account.

We are in a bit of a hurry now since we have an approaching deadline, so probably I won't be dealing with textiles dependencies / installation until next month. Then I will deal with the Spring cleaning project in textiles.

PeterBowman commented 7 years ago

@PeterBowman That is something I am still considering. But there are some dependencies that require certain development version or compiling one specific commit of the required library to have some cutting-edge functionality. [...]

See https://pip.pypa.io/en/stable/reference/pip_install/#git.

David-Estevez commented 7 years ago

Unfortunately, in my case it is a CMake based repository split into two different repositories (OpenCV 3 with contrib, Python 3 support and fix for SVM_load, see https://github.com/roboticslab-uc3m/installation-guides/blob/develop/install-opencv.md).

Nevertheless that reference page is an excellent resource for all my incoming work on simplifying installation, thanks!

PeterBowman commented 7 years ago

As a side note: @jgvictores and I had concluded that all CMake list files may bump the required minimum version to 2.8.11 (now using 2.8.9 due to YCM or even 2.6 due to old project generators). This is guaranteed to work on Trusty onwards, even Precise systems may find appropriate packages in the kalakris PPA (sudo add-apt-repository ppa:kalakris/cmake).

For reference, see also: https://cmake.org/Wiki/CMake_Version_Compatibility_Matrix.

jgvictores commented 7 years ago

Bump https://packages.ubuntu.com/trusty/cmake3 ?

PeterBowman commented 6 years ago

Idea: set 3.5 (3.5.1) as the minimum CMake version everywhere. Thanks to:

PeterBowman commented 6 years ago

YARP is going to require CMake 3.5, soon (https://github.com/robotology/QA/issues/280).

PeterBowman commented 6 years ago

Check this out (kinematics-dynamics-install.md):

First install the dependencies:

  • Install CMake 3.5+
  • Install YCM 0.8+
  • Install YARP 2.3.70+
  • Install KDL 1.4+
  • Install color-debug

At the time of writing, the first three requirements apply now to several repos in our org.

PeterBowman commented 2 years ago

@jgvictores are you happy with https://github.com/roboticslab-uc3m/questions-and-answers/issues/7#issuecomment-398225484? Can we finally close this ticket?

jgvictores commented 2 years ago

Perfect, yes!