TresAmigosSD / SMV

Spark Modularized View
Apache License 2.0
42 stars 22 forks source link

I1512 jvmless logging #1516

Closed laneb closed 5 years ago

laneb commented 5 years ago

Resolves #1512.

Replaces log4j logger with Python standard logger for all logging in SMV:

Removing the cost of py4j from logging improves the latency of get_graph_json for a 1000 module project by over 50%. For now logging levels and other configuration will need to be adjusted through code (e.g. smv.logger.setLevel("DEBUG")). logging supports configuration via yaml, so I created #1517 to load a custom config provided by users.

laneb commented 5 years ago

Missed that smv.dsm.TX was logging through log4j using a different strategy. Will migrate that to use smv.logger as well.

laneb commented 5 years ago
AliTajeldin commented 5 years ago

Got it, makes sense. merging.