Running jupyter notebooks in public dashboard in automatic mode

[swastis10]

@shankari I went through the entire project and realized that we are using Crontab.

In docker-compose.dev.yml, CRON_MODE= is empty by default, manually opening the notebooks. We can make changes in the notebook parameters manually. If I set, CRON_MODE=TRUE it will launch the container in cron mode. In the crontab file, I set */2 * * * * python bin/generate_plots.py mode_purpose_share.ipynb default &> /var/log/log_file.log which will recalculate the files every 2 mins.

To test if the notebook is recalculated every 2 mins, I did docker-compose -f docker-compose.dev.yml up --build with CRON_MODE=TRUE but nothing happens.

Debugging steps:

1. run docker exec -ti em-public-dashboard-notebook-server-1 bash
2. run source setup/activate.sh && conda env update --name emission --file setup/environment36.notebook.additions.yml to download all pip dependencies.
3. run /usr/src/app/.docker/start_notebook.sh. The logs for the same are:

   
   (emission) root@24ae37d05a34:/usr/src/app# /usr/src/app/.docker/start_notebook.sh
   /usr/src/app/e-mission-server /usr/src/app
   DB host = db
   /usr/src/app
   {
   "timeseries": {
       "url": "db",
       "result_limit": 250000
   }
   }
   Web host = 0.0.0.0
   /usr/src/app/e-mission-server /usr/src/app
   /usr/src/app/e-mission-server
   # conda environments:
   #
   base                     /root/miniconda-4.12.0
   emission              *  /root/miniconda-4.12.0/envs/emission

/usr/src/app Running crontab without user interaction, setting python path

which means that crontab is enabled.
4. Now, we want to see the crontab logs: The code generates crontab logs at `devcron ../crontab >> /var/log/cron.console.stdinout 2>&1` so, I run `vim /var/log/cron.console.stdinout` and see the following errors:

Traceback (most recent call last): File "/root/miniconda-4.12.0/envs/emission/bin/devcron", line 8, in sys.exit(main()) File "/root/miniconda-4.12.0/envs/emission/bin/devcron.py", line 47, in main cron.run() File "/root/miniconda-4.12.0/envs/emission/bin/devcron.py", line 259, in run time.sleep(dt) KeyboardInterrupt Traceback (most recent call last): File "/root/miniconda-4.12.0/envs/emission/bin/devcron", line 8, in sys.exit(main()) File "/root/miniconda-4.12.0/envs/emission/bin/devcron.py", line 39, in main crontab_data = open(args[0]).read() FileNotFoundError: [Errno 2] No such file or directory: 'crontab'

[shankari]

In the crontab file, I set /2 * python bin/generate_plots.py mode_purpose_share.ipynb default &> /var/log/log_file.log which will recalculate the files every 2 mins.

In which crontab file? the error seems to suggest that there is no crontab file present

FileNotFoundError: [Errno 2] No such file or directory: 'crontab'

[shankari]

Closing this since @swastis10 has "graduated"