search

mapr-demos / python-bindings

Python bindings for MapR DB JSON API
Apache License 2.0
6 stars 15 forks source link

The handling of the JVM life-cycle should be better #3

Open tdunning opened 9 years ago

tdunning commented 9 years ago

There are several places where the JVM life-cycle is exposed to the user. This is bad because it means that programs may behave differently when we eventually move to a pure C interface. Some of the issues that I observe are:

1) The JVM loading should not be tied one-to-one with making a connection. The JVM loading should be a pre-requisite for establishing a connection that should be transparently satisfied if necessary, but making a second connection should be just fine.

2) Objects like Conditions should be possible to instantiate without actually opening a connection. That may mean that they too cause the JVM to load, or it may mean that they defer any calls to Java until absolutely necessary. It should not be an error to create a Condition before a connection is created.

3) If there is code that would fail if the JVM is not loaded, then that code should cause the JVM to be loaded. This is a generalization of the issue in (1), but should be moderated by the fact that conditions are not dependent on connections or tables.

4) it should be possible to connect to multiple clusters at the same time

jblazecybervision commented 9 years ago

1-3) Then how we pass arguments to JVM (mapr.home.dir)? now we do it in connect() method. If we instantiate JVM lazily then we should define argument in the environment variables for example? 4) Since Java library gets mapr.home.dir from the JVM parameters, to connect to different cluster we need to start another JVM process. Unforunately, starting multiple JVMs is not possible because JVM is like a singleton in libraries as pyjnius and jpype, and cannot be started twice. Another limitation is in the requirements of the interface, since requirement is to start JVM lazily (not binded to specific connection), I'm not sure how to understand which action should be done in which machine.

tdunning commented 9 years ago

I don't think that it is necessary to have a connection for a condition variable. The only time that it is necessary is when you need to use a condition in a query. At that point, a jvm is already available.

I agree with you that the jvm should only be constructed in the connect call. But I don't agree that connect has to fail.

Sent from my iPhone

On Sep 22, 2015, at 9:34, Johny Blaze notifications@github.com wrote:

1-3) Then how we pass arguments to JVM (mapr.home.dir)? now we do it in connect() method. If we instantiate JVM lazily then we should define argument in the environment variables for example? 4) Since Java library gets mapr.home.dir from the JVM parameters, to connect to different cluster we need to start another JVM process. Unforunately, starting multiple JVMs is not possible because JVM is like a singleton in libraries as pyjnius and jpype, and cannot be started twice. Another limitation is in the requirements of the interface, since requirement is to start JVM lazily (not binded to specific connection), I'm not sure how to understand which action should be done in which machine.

— Reply to this email directly or view it on GitHub.

jblazecybervision commented 9 years ago

Replaced exception with warning, so no error will appear now if connect called twice, or some action that requires the JVM is called error will not be thrown.

For Condition actual building of the java object is lazy now - will be done in place where it is required (on calling condition.java_condition).

Connection object also will be constructed if it's not yet constructed via connect() in places where it is needed with default parameters.

About 4) - as I said before, it's unfortunately not possible so far, because of specifics of java library and because of that fact that connection parameters are passed as JVM arguments, not in runtime.

Reopen please if you disagree with some item.

jblazecybervision commented 8 years ago

@tdunning can you please comment what should be done for this ticket?