Problem: It isn't easy to find versions and dependencies without accessing the operating system itself

Please describe the problem you'd like to be solved.

The tickets listed as 'related' below are all open and all identify some form of issue around dependencies and versioning.

Versioning is complex in digital preservation tools.

JHOVE has an overall version, and it has modules that are also versioned. Knowing the module version might be the difference between understanding a validation failure or not.
Siegfried and Fido have overall versions, but then have specific versions of 'signature file', Both version in concert determine whether a format identification will work or not.
Individual tools used for characterization and normalization, if they sit at the wrong version might not contain important fixes which determine their output success, e.g. Ghostscript's ability to read and output various versions of PDF, or ffmpeg's ability to output a valid AV file.
Archivematica as an open source tool, and adopting widely-used open source standards enables its deployment on many different operating environments, CentOS/RHEL, Ubuntu Bionic and Xenial, and also offers a docker-compose environment which uses different base images per module.
In CentOS/RHEL Python which is a big part of Archivematica is 2.7.5 where on Bionic it is 2.7.15 and Xenial is 2.7.12.
Also, depending on how these servers are maintained, kernel versions might be upgraded and output different behavior than perhaps ones that we are used to testing on.

While one approach is to make the control of versioning much tighter, sometimes as a tester, or a developer, devops, or a user in the wild, coming up against issues we're finding to be 'inconsistent' the first piece of information we should be able to access easily is the version information of the environment Archivematica is running on.

Before testing begins, testers and developers should know of the version differences between what is installed, and what is 'ideal'. This might identify issues to be resolved by devs or ops immediately.
Before recording PREMIS events and other information in the METS microservices and the Archivematica database should have access to a dynamic live-list of versions to report back to the user so that it is as accurate as possible.

Archivematica's scripts have access to all of this information at varying degrees, the question for this question, if taken up, is how much do we report, or how do we slowly build up a report that is useful.

Describe the solution you'd like to see implemented.

Collect statistics at service startup and persist them somewhere in the system to be made available as a an API endpoint or download.
Give microservices more interface/API-like behavior where once consistent call that could be made to them would be to return version information relevant to the tools/modules crucial to their operation, metsrw or lxml might be two good examples.
Teach the FPR to report specifically on tool version, there might be more than one approach to this, but certainly allowing the system to report accurately on the version of the tooling it has available, even if the tooling can't always be controlled, would make the service more dynamic. (I imagine this is being done in the PAR project).
Included in this, it would be nice if Archivematica and the Storage Service both reported on which source-control 'commit' they were running as well as their version number.

Describe alternatives you've considered.

I think looking at controlling versions, vs. providing information on versions, this would be a great first step to maintaining Archivematica, as a tester/developer as well as a user running it live. I think it would also enable the issues above to be resolved more easily, and on a more consistent basis, e.g. when versions do fall out of synch which they might.

Additional context

Other than wrestling this weekend with versions!! :wink: The reason this hits home, is that currently, to access this information, we often have to access a distribution directly, accessing kernel version information, or tool information directly. In the case of what Python might be doing, we might have to access virtual environments which themselves might be configured differently to what vanilla-python might be running on the server itself. This is especially difficult during release testing if the version of a tool might be what is causing inconsistencies between versions of Archivematica.

See Also

This study of the tools in the FPR and the versions appearing across all AM deployments.

For Artefactual use: Please make sure these steps are taken before moving this issue from Review to Verified in Waffle:

All PRs related to this issue are properly linked 👍
All PRs related to this issue have been merged 👍
Test plan for this issue has been implemented and passed 👍
Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍

Title	Milestone	Assignees	Stage
problem: signature file version is not captured during format identification #1053	N/A	N/A	Open
Problem: Storage Service Locations "Usage" does not update when AIPs are moved #495	N/A	N/A	Open
Problem: Tesseract is out of 4.0.0-beta and 4.1.1 available #1045	N/A	N/A	Closed
Archivematica should pin exact package versions in requirements.txt for predictable deployments #634	1.10.0	N/A	Closed
Problem: There are JHOVE inconsistencies across deployments and in the FPR/PREMIS output #524	1.10.0	ablwr, mamedin	Closed
Problem: Fido is being reported as 1.3.12 from FPR Database where only 1.3.10 is on the system through PIP requirements #520	1.11.0	N/A	Closed
Problem: FPR "related tool" options are duplicated and with incorrect versions #394	N/A	ablwr	Closed
Problem: the versions shown for fpr tools are outdated #191	1.10.0	sallain	Closed

archivematica / Issues

Problem: It isn't easy to find versions and dependencies without accessing the operating system itself #522

do-not-edit-start-codetree-epic-issues

Issues in this epic:

do-not-edit-end-codetree-epic-issues