Fix the e2e cleanup job

pixiebrix / pixiebrix-extension

PixieBrix browser extension

https://www.pixiebrix.com

GNU Affero General Public License v3.0

83 stars 22 forks source link

Fix the e2e cleanup job #8881

Closed fungairino closed 1 month ago

fungairino commented 1 month ago

Description

Add metrics / log queries to view how regularly the job is running and if it throws errors
Figure out why the job is not deleting the expected packages / resources related to the e2e test user.

A potential related issue is how the created date is being tracked in packages.

mnholtz commented 1 month ago

Figure out why the job is not deleting the expected packages / resources related to the e2e test user.

From manually executing the management command, it seems like it is actually being successful at deleting resources.

fungairino commented 1 month ago

Related log for the scheduler job running on sumologic: https://pixiebrix.sumologic.com/ui/#/search/44a46e5f_7208_9104_6c7e_41809af5a6f6

I'm not sure if we have any other logs than this, so we might want to make sure that we log some message about how many resources were deleted if the job succeeded.

We also should try to find a way to add some attribute that identifies the specific scheduler job that is emitting the log.

fungairino commented 1 month ago

See this slack thread: https://pixiebrix.slack.com/archives/C03HUH15EJX/p1721320059338279

It seems like the job was just never running.

We should still follow-up and make sure that we have some monitoring in place to ensure that we get notified if the job fails to run. (and maybe add some more logging metadata)

fungairino commented 1 month ago

Moved into https://github.com/pixiebrix/pixiebrix-extension/issues/8910