Add possibility to refresh list of DAGs

[mnojek]

Description

Add new button to DAGs list view for refreshing DAGs, e.g. when new DAG is uploaded to Airflow, using button will refresh the list without a need to refresh the whole page.

This can look like this (or it can be separated from the group of buttons):

Use case/motivation

I want to be able to refresh the DAGs list without a need to refresh the whole page. Especially, when I upload a new DAG and it's not displayed automatically on the DAGs list.

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[BasPH]

I'd rather get the main page to also autorefresh. Adding yet another button without reducing the number of clicks required to refresh doesn't feel like a valuable addition IMO.

[potiuk]

Agree with @BasPH . It's pretty "normal" for the page to get refreshed with refresh button when pretty much all content is refreshed. There is a very (unless it's autor-refresh which is good to avoid flickering) there is very little saving by just refreshing the list - all the other components are already in cache.

[mnojek]

So what actually we can do about it? Somehow implement autorefresh when new DAG is added or just refresh the list with some time interval? I don't fully get your message @potiuk (please review it)

[potiuk]

So what actually we can do about it? Somehow implement autorefresh when new DAG is added or just refresh the list with some time interval? I don't fully get your message @potiuk (please review it)

Just to clarify - I think (and I think @BasPH thinks the same) this "list-only refreshing" is only going to be useful if we implement it similarly to http://airflow.apache.org/docs/apache-airflow/stable/ui.html#graph-view with auto-refresh. Otherwise it's fine for the users to "Refresh" using Browser's built in refresh button.

There are two cases:

1) When users want to manually refresh the list - > this is perfectly fine to hit the browser "Refresh" button. There will be a bit flickering when the whole page reloads, but this is perfectly fine - when someone hits refresh button, this is natural and expected. And the way caching works in the browser refreshing the "whole page" will not be much slower than "just refresh the list" - the time between the user clicks the "refresh" button and gets the page rendered will be about the same whether you use refresh browser button or "mini-refresh" list button - simply because most of the time will be reading the content of the list and rendering the content of it. And Hitting "refresh" in browser has an additional advantage in this case, because you know (browser tells you) when the refresh finished (it shows progress bar and visually indicates refresh is in progress) . So there is little value to it.

If you would like to do a "list only" refresh button, you have to make it a bit more complex than what refresh of browser is doing - you also need to include some kind of progress information - i.e. indicate to the user that refreshing is in-progress and that it finished. You need to replace the DOM Content in the way that flickering is minimized (likely with atomic replacement of the whole new list content in one go). And once you did it, that it's very close to being able to run auto-refresh like we have in http://airflow.apache.org/docs/apache-airflow/stable/ui.html#graph-view

And this leads us to second case

2) Automatically refresh the list with "auto-refresh" feature like we have in graph view. This one has MUCH bigger value, as this means you have all the content fresh without your action.

So my point is - if youd add just "refresh" button the value added is minimal, but the value is much bigger if you also make "auto-refresh" working at the same time for that page.

[uranusjr]

Considering the complexity of the current index page, re-implementing it to support in-page refresh like the graph view is not worth it IMO, unless we can do it in a way that can be reused for AIP-38. An htmx/Turbo-like approach (not necessaily using either of those implementations since we only need very minimal features) may be more viable—we still do a complete fetch of the index page, but instead of reloading the entire page, only replace the DAG table with new content in-place.

[bbovenzi]

Yeah, that was the reason that I didn't do auto-refresh for the home page when I added it to tree view. I did have a basic live-update working in the demo for AIP-38. After the New Tree View (#18675) is added we can start thinking about other parts of the UI that are worth refactoring.

[potiuk]

BTW, @bbovenzi - will the approach with Tree view be replicated and we will gradualy replace screen by screen with new reactive approach rather then have a separate UI? Seems that is doable and actually very nice way to approach it if we can combine old and new way and replace piece-by-piece

[bbovenzi]

@potiuk Yes and no. I do want to refactor other parts of the app. But many are too tightly linked to FAB that it would be so much more effort to build vs from scratch. So we will get some piece-by-piece changes but we will still need to do a hard break to a separate UI at some point.

[potiuk]

Understood. Makes perfect sense

[nathadfield]

Auto-refresh of the DAGs page is now a feature of Airflow.