prha
commented
1 month ago @mdhatrika are you talking about the jobs table? Starting in 0.14.10, we migrated off of the jobs table to start reading from the instigators table. The main difference here is that the rows in the jobs table were keyed off of specific things in the code location origin, such as python path and version. The rows in the instigators table are durable across python versions, so you shouldn't lose schedule/sensor state when you update those parts of your code locations.
We've been double-writing entries to both tables, but I'll create a task to clean up the jobs table to avoid confusion in the future.
Dagster version
1.7.7
What's the issue?
Hello,
We have a Dagster DBT PostgreSQL database that stores all the metadata related to our jobs, schedules, assets, etc. Recently, we updated some schedules by modifying their cron expressions and removed a few assets. While these changes appear correctly in the Dagster UI, the jobs table in the Dagster database still shows the old cron expressions and assets.
Do we need to perform any migration or take any specific actions to ensure that the database reflects these updates? We're unsure why the changes haven't been picked up.
What did you expect to happen?
All new changes that we made should be reflected in the Dagster database metadata
How to reproduce?
Make some updates in asset selections to jobs and cron expression being changed from UTC to EST and you should see the changes in UI, but not in the Dagster metadata especially the Jobs tables which has the cron data
Deployment type
Dagster Helm chart
Deployment details
Tools Docker Helm Charts
Additional information
The modified date seems to be updated fine, just some set of data is not updating
Message from the maintainers
👍 We are highly impacted by this issue, as we have a downstream reports which are relayed on this data, which is broken currently