Closed dmijatovic closed 8 months ago
I tested this PR and have some comments. Unsure if this originates in the scrapers of frontend. My setup is an RSD with a single software and the following info:
Repo: https://github.com/GooglingTheCancerGenome/sv-callers
Package managers: https://anaconda.org/conda-forge/esmvaltool https://pypi.org/project/ESMValTool/ https://hub.docker.com/r/esmvalgroup/esmvaltool https://cran.r-project.org/web/packages/GGIR https://mvnrepository.com/artifact/nl.junglecomputing/sbbi-upnplib https://www.npmjs.com/package/aws-sdk
I see the many of the scrapers running successfully after leaving the RSD for some time. Some never seem to run however:
As a user, it is difficult to figure out if the scrapers hasn't run because of some mistake I have made, or because of some external reason (waiting for my turn, scraper did too many requests, scraper only runs once a day, a missing scraper, etc.). Would there be a way to make this more clear?
Unrelated: I noticed that when making a mistake when cutting and pasting a package manager URL, the PM detection sometimes fails. For example:
Thanks for the feedback @jmaassen. I have fixed repository stats flag (was not correct) and removed package manager services that are not implemented (yet). I made force push, please remove old local branch and pull again.
The feedback provided about the services is now much better. A few other comments:
I would change the text in this text box:
to:
No information about package managers is provided for this software
And this one:
to:
The RSD collects additional information about your software using the links you
have provided. For example, the software repository URL is used to collect
information about the commit history and programming languages, and a
package manager URL can be used to retrieve download numbers.
This section provides feedback if the RSD could succesfully retrieve
this information or if it has encountered an error. These may be cause by
a URL that is missing or incorrect.
Please contact us at rsd@esciencecenter.nl in case you need our
assistance in solving any of the errors.
There is also a typo here:
"Commit hisitory" should be "Commit history". I would also use the full "Repository statistics" instead of "Repository stats"
For these two:
I think that the "public api" bit in the subtitle is not that useful for the average user? Maybe its better to say:
Additional information extracted using the source code repository URL
and
Additional information extracted using package manager URLs
@jmaassen Thanks for the feedback. I improved the messages and rebased this PR to the latest main (to include updates of Next and Node).
The SonarCloud Quality Gate passed, but some issues were introduced.
2 New issues
0 Security Hotspots
0.6% Coverage on New Code
0.0% Duplication on New Code
Status report of background services
To test background services you will need to have properly defined service items in the .env file. Some might require secret to enable service to scrape information using "less restricted" quota.
Changes proposed in this pull request:
scheduled
: if the url is provided and last_error or last_run value is not presentnot_active
: if the repository url or package manager url value is not presenterror
: if the last_error value is presentsuccess
: if the last_run value is present and the last_error value is NOT present. We show date/time of the last successful runHow to test:
make start
to build app and generate test datadocker compose down && docker compose up
Background services (scheduled - initial state)
Background services (not active state)
Background services (error state)
Background services (multiple package managers)
PR Checklist:
docker-compose.yml