Fix dashboard view

[hancush]

The dashboard view is currently broken! 🙀

https://la-metro-dashboard.datamade.us/dashboard/

Let's fix this.

[hancush]

@antidipyramid Ok, so I cannot reproduce this locally. See if you can use the stack trace in the error on the live site to identify where and why the code is breaking. Leave a short summary of the problem in this issue, and we can chat about how best to resolve.

[antidipyramid]

After reading the stack trace, it looks like one of the DAGs is failing at this line in Dashboard.get_dag_info():

d.following_schedule() seems to be returning None. The source for following_schedule():

Clearly Airflow wants us to move away from using this function but I'm not sure that that's the problem. I guess the first possibility is that a DAG is returning None for next_dagrun_info().

Did the problem first start when I pushed up the analytics DAG? If so, it's likely that I configured it incorrectly.

[hancush]

@antidipyramid Love the instinct to look for the last change (analytics DAG). Unfortunately, this page was broken before that release. It seems like we have a DAG that does not have a next scheduled run. Can you figure out (based on our DAG files) which one doesn't have a schedule? (Hint: This is definitely my fault!!! 😬)

[antidipyramid]

@hancush Hm, well it seems like the only DAG with no schedule_interval is make_packet.py. That one is explicitly set to None.

If that's it, why can't we reproduce the error locally?

[hancush]

@antidipyramid Try running the make_packet DAG in your local instance, then see if the error occurs.

[antidipyramid]

@hancush Ah ok, I see now! The Dashboard error doesn't appear until Airflow attempts to run make_packet. I thought Dashboard would throw an error even if it hadn't been run yet. What do we want to set the interval as?

[hancush]

make_packet intentionally doesn't have a schedule, because it's triggered by the Metro application. Rather than giving it an interval, I think we need the handle the case where a DAG doesn't have a next scheduled run. Make sense?

[antidipyramid]

@hancush Got it. Adding a simple conditional seems to remove the error:

I thought this portion of the templates/dashboard.html would throw an error for trying to access pst_time when dag.next_scheduled_date is None but it doesn't on my end. I'm thinking of adding an extra check here just in case.

[hancush]

Interesting, I would expect that, too. I like the idea of a check so it's explicit that there might not be a next run.