dockstore / dockstore

Our VM/Docker sharing infrastructure and management component
https://dockstore.org/
Apache License 2.0
116 stars 27 forks source link

DOCK-2312: Add "is this Entry automatically syncing?" endpoint #5865

Closed svonworl closed 2 months ago

svonworl commented 2 months ago

Description This PR adds an isSyncing endpoint to the webservice that indicates whether or not the specified Entry is being automatically synced (currently, via our GitHub app).

Regarding the design and functionality of this endpoint, there were a number of different options as to complexity and effectiveness:

In the end, I decided to go with the most basic approach, which is to conclude that an entry is not syncing iff the latest LambdaEvent for the entry's repo is an UNINSTALL. All the other approaches were much more involved. Sure, they would give us more visibility for older entries, but everyone I consulted with seemed to [slightly grudgingly] agree that the basic solution is good enough. I would love it to be perfect and give us useful information about preexisting uninstalls, but I don't think the engineering cost justifies the payoff. As it is, it's a quick little addition that will help us diagnose new issues.

Access to the endpoint is limited to entry owners and admins.

Note that since LambdaEvents can be delivered to the webservice out of order, and we don't log the GitHub creation date (because in general, the webhook events don't include one), it's theoretically possible that an UNINSTALL will get logged after events that were actually created later, and cause the endpoint to report that an entry is not syncing. Similar issues can happen if we fail to log an event that directly follows an UNINSTALL for whatever reason. Such cases should be fairly rare, and will tend to heal on their own, because when the next non-UNINSTALL event gets logged, the endpoint will report that the entry is syncing.

There will be a companion UI PR, coming soon. Currently, the plan is to display some sort of warning on the private .dockstore.yml-based entry page if the entry is not syncing, and leave the UI as-is otherwise.

Review Instructions Register an entry in a test GitHub repository via the GitHub App / .dockstore.yml path. Query this endpoint and confirm that it indicates that the entry is syncing. Uninstall the GitHub App from the repo and wait five minutes. Query this endpoint and confirm that it indicates that the entry is not syncing. Reinstall the GitHub App on the repo and wait five minutes. Query this endpoint and confirm that it indicates the the entry is syncing.

Issue https://ucsc-cgl.atlassian.net/browse/DOCK-2312 https://github.com/dockstore/dockstore/issues/5331

Security and Privacy

No concerns.

e.g. Does this change...

Please make sure that you've checked the following before submitting your pull request. Thanks!

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 76.47059% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 72.88%. Comparing base (7347da6) to head (6cec07d).

Files Patch % Lines
...n/java/io/dockstore/webservice/api/SyncStatus.java 66.66% 2 Missing :warning:
.../dockstore/webservice/resources/EntryResource.java 77.77% 0 Missing and 2 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #5865 +/- ## ============================================= - Coverage 74.49% 72.88% -1.62% + Complexity 5244 5110 -134 ============================================= Files 368 369 +1 Lines 18945 18960 +15 Branches 1992 1993 +1 ============================================= - Hits 14114 13819 -295 - Misses 3880 4191 +311 + Partials 951 950 -1 ``` | [Flag](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | Coverage Δ | | |---|---|---| | [bitbuckettests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `27.22% <5.88%> (-0.02%)` | :arrow_down: | | [integrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `58.92% <76.47%> (+0.01%)` | :arrow_up: | | [languageparsingtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `11.08% <5.88%> (-0.01%)` | :arrow_down: | | [localstacktests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `21.70% <5.88%> (-0.02%)` | :arrow_down: | | [toolintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `30.60% <5.88%> (-0.02%)` | :arrow_down: | | [unit-tests_and_non-confidential-tests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `23.41% <5.88%> (-5.17%)` | :arrow_down: | | [workflowintegrationtests](https://app.codecov.io/gh/dockstore/dockstore/pull/5865/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore) | `38.86% <5.88%> (-0.03%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=dockstore#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

svonworl commented 2 months ago

Lots of conflicting feedback, took a crack at getting closer to consensus. In the end, I:

sonarcloud[bot] commented 2 months ago

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
86.2% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud