tugraz-isds / systemds

An open source ML system for the end-to-end data science lifecycle
Apache License 2.0
37 stars 20 forks source link

[WIP] Python SystemDSContext instead of shutdown() #131

Open kev-inn opened 4 years ago

kev-inn commented 4 years ago

Switch to using SystemDSContext as a ContextManager instead of purely automatic startup and manual shutdown. This makes performance impact more obvious for the user because the expensive startup of a java process happens during SystemDSContext creation and allows for us to ensure the closing of the py4j connection and shutdown of subprocess if the user decides to use with SystemDSContext() as sds:. Manual shutdown with .close() is also possible for cases where the user wants to keep the SystemDSContext around longer, for example as an member variable of his class.