search

codecov / engineering-team

This is a general repo to use with GH Projects
1 stars 1 forks source link

Using Shelter for CLI commands can lead to confusing user experience #2634

Open JerrySentry opened 3 weeks ago

JerrySentry commented 3 weeks ago

CLI commands that gets redirected to Shelter always yields a 202 "queued" status whereas if the request is handled by the API it would get processed on the spot and return whatever its supposed (eg 404, 200, etc). The API would return useful information to the CLI caller, however with Shelter the user would always see 202 and would not know the true response of the call ,potentially leading to incorrect workflow in the CI or lead to confusions for the user.

For example, this user did not know why their workflow was not working, turns out they forgot to do create-commit. In the logs all they saw was Process Upload Completion complete when in reality the upload-complete command should have returned "Commit SHA not found" which the user would have been able to self diagnose fix instead of raising a support ticket.

https://sentry.zendesk.com/agent/tickets/129874

Notice the initial responses to upload-complete command was 202, then a few log lines later those were processed and showed an error that is now lost. https://cloudlogging.app.goo.gl/UG5kfb5uVax3MKyZ8

Related thought: Could the use of Shelter for CLI commands also cause race conditions with a user's typical "create-commit -> ... -> upload" workflow?

  1. "create-commit" -> queued/202
  2. "upload" (some variant of upload command) -> queued/202
  3. CI shows all good
  4. upload message gets consumed first -> errors out with commit not found
  5. create commit message gets consumed -> a commit created
  6. the system ends in an expected state

Not sure if this can happen with the current setup (topics, subscribers, FIFO policy, etc)

katia-sentry commented 2 weeks ago

@trent-codecov this didn't end up being a bug, but would be good to improve the experience here