search

mljar / mercury

Convert Jupyter Notebooks to Web Apps
https://RunMercury.com
GNU Affero General Public License v3.0
4.04k stars 256 forks source link

ETA for Scheduled Notebooks? #322

Open kapily opened 1 year ago

kapily commented 1 year ago

Hi - do you have an ETA for scheduled notebooks?

In the meantime, is there a hackaround to just have a Notebook get executed in the background every day? A hackaround could be something like: run this command to inject a task into the queue and have the worker run it.

Use-case: there's a notebook that is expensive to compute for the first time daily, but subsequent runs are fast due to cacheing. If I can have the notebook get auto-scheduled, that reduces user-preceived latency.

pplonski commented 1 year ago

Hi @kapily,

No ETA for scheduling notebooks ... Scheduling code for V1 is ready ... probably it could be ported to V2. In V1 we keep scheduling details in YAML. The question is, how to save scheduling information for V2. Should it be in Python API? or it can be in the UI in the Mercury App?

Are you going to produce some files with results from scheduled notebooks? How would you like to cache results between runs? I'm very interested in your use-case Could you please provide more details?

andreytaboola commented 7 months ago

Hi @pplonski , we are also super interested in scheduled notebooks and waiting it to be released. Our case is to use it for scheduling and generating hourly/daily reports based on python notebooks our Algo/DS teams develop. If it something that exists but need adoption to newer version I can make a try and help with some guidance on where & how

pplonski commented 7 months ago

Hi @andreytaboola,

we would like to add this as well, we will plan it with @aplonska @apjanusz and will let you know about ETA.

andreytaboola commented 6 months ago

Hi again @pplonski Any news? Just curios, thank you