yohamta
commented
1 day ago Thank you very much for submitting the issue and showing interest in contributing a PR! Your input is really helpful.
Currently, when a large number of DAG files are created, performance issues arise due to the need to load a significant amount of data. To address this, I plan to change the storage structure. With the upcoming release of the v2.0 alpha version, I would like to ensure that all DAG execution data for the current day is reflected on the dashboard.
As a temporary workaround, I think that allowing configuration for the page limit might be an idea. If you're interested in contributing a PR, it would be very welcome.
To add config key in the configuration, the following files would need to be modified:
- Add a
perPagekey andDAGU_PER_PAGEenvironment variable to the config (to change the default upper limit): https://github.com/dagu-org/dagu/blob/10b0c690acbeadec527d3d3bc407779e86be1549/internal/config/config.go#L34-L62 - Add configuration items to the frontend: https://github.com/dagu-org/dagu/blob/10b0c690acbeadec527d3d3bc407779e86be1549/internal/frontend/server/templates.go#L65-L93
- Modify base HTML template https://github.com/dagu-org/dagu/blob/10b0c690acbeadec527d3d3bc407779e86be1549/internal/frontend/templates/base.gohtml#L10-L19
- Related frontend files
I also welcome any alternative ideas for the workaround. :)
The dashboard page only shows 100 DAGs, even if there are more than 100 DAGs in the system
It looks like there is a default limit of 100 here (which is reasonable) https://github.com/dagu-org/dagu/blob/678dd21a05aeb968ffbe9aac01a5b6d46026a232/internal/client/client.go#L311
I can see the API has support for increasing the limit
api/v1/dags?limit=xxxxThoughts on this? I appreciate we may not want to get all the DAGs in case there are a lot in the system.
What is restricted it to only DAGs executed within the day? Or maybe this time period can be controlled?
Happy to help with a PR, but keen to see how you might want to approach a possible solution.