Status report of background services for software

[dmijatovic]

Status report of background services

To test background services you will need to have properly defined service items in the .env file. Some might require secret to enable service to scrape information using "less restricted" quota.

Changes proposed in this pull request:

• The software maintainer can view status of the background services running for the software item
• The statuses reported are:
  • scheduled: if the url is provided and last_error or last_run value is not present
  • not_active: if the repository url or package manager url value is not present
  • error: if the last_error value is present
  • success: if the last_run value is present and the last_error value is NOT present. We show date/time of the last successful run
• Short description about the background services is added in the right panel of the page

How to test:

• make start to build app and generate test data
• login as user, if you use dev environment variable first user will become admin.
• navigate to first software item in the overview and go to edit software page.
• initially you should see background report with scheduled status for all items as in the image 1
• start the scrapers services by restarting app: docker compose down && docker compose up
• navigate to second software item. Remove repository URL and all package managers. The background service reports should like ones in the image 2
• login as rsd_amin (if not already), navigate to error logs page of administration section. From there use link to open the software page with an service error. Confirm that error is shown in the background services too (image 3)
• from the software overview page select the item that has programming languages in the card. Navigate to edit section and confirm that, at least, programming languages service was runned successfully (image 4)
• read the background services info on the right side of the page. If not clear please provide suggested changes to the explanation

Background services (scheduled - initial state)

Background services (not active state)

Background services (error state)

Background services (multiple package managers)

PR Checklist:

• [ ] Increase version numbers in docker-compose.yml
• [ ] Link to a GitHub issue
• [ ] Update documentation
• [ ] Tests

[jmaassen]

I tested this PR and have some comments. Unsure if this originates in the scrapers of frontend. My setup is an RSD with a single software and the following info:

Repo: https://github.com/GooglingTheCancerGenome/sv-callers

Package managers: https://anaconda.org/conda-forge/esmvaltool https://pypi.org/project/ESMValTool/ https://hub.docker.com/r/esmvalgroup/esmvaltool https://cran.r-project.org/web/packages/GGIR https://mvnrepository.com/artifact/nl.junglecomputing/sbbi-upnplib https://www.npmjs.com/package/aws-sdk

I see the many of the scrapers running successfully after leaving the RSD for some time. Some never seem to run however:

As a user, it is difficult to figure out if the scrapers hasn't run because of some mistake I have made, or because of some external reason (waiting for my turn, scraper did too many requests, scraper only runs once a day, a missing scraper, etc.). Would there be a way to make this more clear?

Unrelated: I noticed that when making a mistake when cutting and pasting a package manager URL, the PM detection sometimes fails. For example:

• add package manager
• paste "ttps://hub.docker.com/r/esmvalgroup/esmvaltool" (note the first "h" missing)
• PM detection says "other"
• fix by moving to start of line and typing "h"
• PM detection still says "other"
• move to end of line and type
• PM detection now works correctly

[dmijatovic]

Thanks for the feedback @jmaassen. I have fixed repository stats flag (was not correct) and removed package manager services that are not implemented (yet). I made force push, please remove old local branch and pull again.

[jmaassen]

The feedback provided about the services is now much better. A few other comments:

I would change the text in this text box:

to:

No information about package managers is provided for this software

And this one:

to:

The RSD collects additional information about your software using the links you
have provided. For example, the software repository URL is used to collect 
information about the commit history and programming languages, and a 
package manager URL can be used to retrieve download numbers.  

This section provides feedback if the RSD could succesfully retrieve 
this information or if it has encountered an error. These may be cause by 
a URL that is missing or incorrect. 

Please contact us at rsd@esciencecenter.nl in case you need our 
assistance in solving any of the errors.

There is also a typo here:

"Commit hisitory" should be "Commit history". I would also use the full "Repository statistics" instead of "Repository stats"

For these two:

I think that the "public api" bit in the subtitle is not that useful for the average user? Maybe its better to say:

Additional information extracted using the source code repository URL

and

Additional information extracted using package manager URLs 

[dmijatovic]

@jmaassen Thanks for the feedback. I improved the messages and rebased this PR to the latest main (to include updates of Next and Node).

[sonarcloud[bot]]

Quality Gate passed for 'rsd-frontend'

The SonarCloud Quality Gate passed, but some issues were introduced.

2 New issues
0 Security Hotspots
0.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud