Potential dependency conflicts between invenio-rdm-records and invenio-records-files

[NeolithEra]

Hi, as shown in the following full dependency graph of invenio-rdm-records, invenio-rdm-records requires invenio-records-files >=1.2.1,<1.3.0, invenio-rdm-records requires invenio-records-permissions >=0.7.0 (invenio-records-permissions 0.7.0 will be installed, i.e., the newest version satisfying the version constraint), and directed dependency invenio-records-permissions 0.7.0 transitively introduces invenio-records-files >=1.2.0,<1.3.0.

Obviously, there are multiple version constraints set for invenio-records-files in this project. However, according to pip's “first found wins” installation strategy, invenio-records-files 1.2.1 (i.e., the newest version satisfying constraint >=1.2.1,<1.3.0) is the actually installed version.

Although the first found package version invenio-records-files 1.2.1 just satisfies the later dependency constraint （invenio-records-files >=1.2.1,<1.3.0), such installed version is very close to the upper bound of the version constraint of invenio-records-files specified by invenio-records-permissions 0.7.0.

Once invenio-records-permissions upgrades，its newest version will be installed, as invenio-rdm-records does not specify the upper bound of version constraint for invenio-records-permissions. Therefore, it will easily cause a dependency conflict (build failure), if the upgraded invenio-records-permissions version introduces a higher version of invenio-records-files, violating its another version constraint >=1.2.1,<1.3.0.

According to the release history of invenio-records-permissions, it habitually upgrates Invenio-records-files in its recent releases. For instance, invenio-records-permissions 0.7.0 upgrated Invenio-records-files’s constraint from * to ==1.1. , and invenio-records-permissions 0.7.1 upgrated Invenio-records-files’s constraint from ==1.1.1 to >=1.2.0,<1.3.0.

As such, it is a warm warning of a potential dependency conflict issue for invenio-rdm-records.

Dependency tree

invenio-rdm-records - 2.0.1
| +- jsonmerge(install version:1.7.0 version range:>=1.4.0,<2.0)
| | +- jsonschema(install version:2.6.0 version range:*)
| +- jsonschema(install version:2.6.0 version range:<3.0.0)
| +- invenio-records-permissions(install version:0.7.0 version range:>=0.7.0)
| | +- invenio-records-files(install version:1.2.1 version range:>=1.2.0,<1.3.0)
| | +- pyserial(install version:3.4 version range:==3.)
| | +- six(install version:1.14.0version range:==1.)
| +- invenio-records-files(install version:1.2.1 version range:>=1.2.1,<1.3.0)
| | +- mbed-os-tools(install version:0.0.13 version range:>=0.0.9,<0.1.0)
| | +- prettytable(install version:0.7.2 version range:>=0.7.2)
| +- prettytable(install version:0.7.2 version range:<1.0)
| +- pydash(install version:4.7.6 version range:<5.0,>=4.0)
| +- pyserial(install version:3.4 version range:>2.5)
| +- requests(install version:2.23.0 version range:*)
| | +- certifi(install version:2020.4.5.1 version range:>=2017.4.17)
| | +- chardet(install version:3.0.4 version range:>=3.0.2,<4)
| | +- idna(install version:2.9 version range:>=2.5,<3)
| | +- urllib3(install version:1.25.9 version range:>=1.21.1,<1.26)
| +- semver(install version:2.10.1 version range:<3.0,>=2.0)
| +- six(install version:1.14.0 version range:>=1.0,<2.0)
| +- transitions(install version:0.8.1 version range:<1.0)
| | +- six(install version:1.14.0 version range:*)
| +- yattag(install version:1.13.2 version range:>=1.0,<2.0)

Thanks for your help. Best, Neolith

[NeolithEra]

Suggested Solution

1. Loosen the version range of invenio-records-files to be >=1.2.1.
2. Remove your direct dependency invenio-records-files, and use the invenio-records-files transitively introduced by invenio-records-permissions.
3. Change your direct dependency invenio-records-permissions to be <=0.7.0. @ppanero Which solution do you prefer, 1 ,2or 3? Please let me know your choice. May I pull a request to solve this issue?

[lnielsen]

Thanks @NeolithEra for reporting. All the modules you mention are under our control, and when we're out of the development phase, InvenioRDM will control the dependencies via Invenio releases, which locks all related modules to their patch-level release. Thus, this is not something we will fix at this stage of the development. Thanks though for the offer to make a PR, it's much appreciated!