search

spotify / luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Apache License 2.0
17.85k stars 2.39k forks source link

See task workflow for indefinite time #3245

Open MrityunjoyS opened 1 year ago

MrityunjoyS commented 1 year ago

I've seen lot of issues and codes but was not able to find a definite solution. I've setup db, created by custom luigi.cfg files. I'm able to see all task in /history window. I'm using central scheduler. But what I need is to see the workflow indefinitely in the UI window. Is there any way to achieve it. Any kind of work around will be highly appreciated. Thanking all in advance.

MrityunjoyS commented 1 year ago

I tried using two kind of config

[scheduler] remove_delay=-1

and

[scheduler] remove_delay=99999

but still UI data is not visible for more than a day,