cronitorio / cronitor-python

Python client for Cronitor
MIT License
73 stars 9 forks source link

Stucked environments when deleted #23

Closed soragXYZ closed 2 years ago

soragXYZ commented 2 years ago

Hi!

I'm working on a project with multiple environments (prod, dev, staging, etc)

At first, everything went smooth, I updated my code to send telemetry if an error occurs, depending on the environment.

I was testing your product when something strange happened: when you delete an environment in the dashboard, this environment seems "stucked".

After being deleted, the environment isn't automatically recreated even if an event is sent (but according to the doc: "if the environment doesn't exist it will be created automatically on the first ping" )

The deleted environment can't receive any events anymore, even if it is manually recreated in the dashboard later on.

Additionally, in my jobs, in active environment, there is an empty field, which is probably the environment that was deleted, then unsucessfully recreated after. But I can still see the environment name on the API page (https://cronitor.io/api/monitors/?env=&time=24h )

Does someone have any idea about what is going on ?

Steps to reproduce:

cronitor.api_key = <apiKey>
cronitor.environment = <env>

monitor = cronitor.Monitor(<monitorName>)

...

monitorVerifyTx.ping(
    state='fail',
)
aflanagan commented 2 years ago

Hi @soragXYZ - thanks for this report! I believe this is a caching bug. In our telemetry ingest pipeline we cache the environment look up for a number of hours, and it looks like this cache key was not being invalidated when deleting the environment from the dashboard.

I am deploying a fix for that now, and you should see this working as expected shortly!