I tried exploring chronon for the past one week with one of the production usecase. Faced few issues in understanding the following
How ts_ds is used in joins ? Looks like this field is created implicitly.
There are many places where date argument is used. Confused on how to use different date args.
Source has date param,
GroupBy has backfill_start_date
run.py has ds, start_date, end_date.
Which partition does backfill starts filling. May be documenting different scenarios might be helpful.
Example scenario. I have a groupBy with backfill_start_date : 2024-01-01 and I gave run.py --ds=2024-05-01 . If I have groupBy output for 2024-03-31 (but do not have 2024-01-01 to 2024-03-30) . From where my backfill will start populating partitions.
I tried exploring chronon for the past one week with one of the production usecase. Faced few issues in understanding the following
ts_dsis used in joins ? Looks like this field is created implicitly.Example scenario. I have a groupBy with backfill_start_date : 2024-01-01 and I gave
run.py --ds=2024-05-01. If I have groupBy output for 2024-03-31 (but do not have 2024-01-01 to 2024-03-30) . From where my backfill will start populating partitions.