Fix issue with logging FileHandler in unsupported random writes file system

itsjafer / jupyterlab-sparkmonitor

JupyterLab extension that enables monitoring launched Apache Spark jobs from within a notebook

https://krishnan-r.github.io/sparkmonitor/

Apache License 2.0

92 stars 23 forks source link

Closed DougTrajano closed 2 years ago

DougTrajano commented 2 years ago

In this PR, we are adding some env vars to control logging settings.

I needed it because the FileHandler causes an error with file systems that does not support random writes (e.g. dbfs).

SPARKMONITOR_LOG_LEVEL: Logging Level.
- DEBUG, INFO (default value), WARNING, ERROR, CRITICAL
SPARKMONITOR_LOG_HANDLER: Use FileHandler or StreamHandler
- "file" (default value) use FileHandler('sparkmonitor_kernelextension.log', mode='w')
- "anything else" use StreamHandler(sys.stdout)

itsjafer commented 2 years ago

LGTM