Pooling Implemented in Pyorientdb OMG/Client?

[jbrow70]

Hi,

Can someone confirm or deny that pyorient OGM and/or the base client have the ability to do connection pooling? It doesn't appear so. I see relations in https://orientdb.com/docs/2.2/Graph-Factory.html that implements pooling with OrientDB but I don't think this is in pyorient client either. I've searched the source code of pyorient client and OGM and found no pooling anywhere in the code. I also see some threads on folks trying to do concurrency: https://github.com/mogui/pyorient/issues/145 and https://github.com/mogui/pyorient/issues/51. One said "connection pool is not implemented in the driver." I realize one could build their own pooling but was hoping the OGM or at least the driver had some sort of pool implemented.

If it's not implemented, here's my next question: running concurrent queries against the same graph connection will really run serially in the DB, correct? We were going to test that but haven't done it yet. It sounds like that is the case though.

It would really be useful to pool and recycle connections especially for use cases where application is used to process asynchronous events. I know there's OrientDB Batch that's implemented in OGM and client but we have found limitations with this that doesn't look to work in our case. E.g., if one process in the batch fails once commit is run, all processes in the batch fail.

Any knowledge around this would be appreciated.

[jbrow70]

Hi,

If someone could confirm the capabilities of the python driver (pyorient) currently with pooling and concurrency that would be great!

We're thinking of using orientdb with an application/web service that will take concurrent requests. We're wondering if sharing one connection is scalable. Will it be able to handle multiple queries at a time or would it be serial.

The GraphFactory I mentioned earlier was for Java so that's not relevant for pyorient for sure. We're trying to test concurrency now with the driver on one connection, but it's not been easy to make a query slow enough to where if we issue multiple queries the query planner will show them running concurrently or serially, at least definitively. We're trying though to answer our own question.

Anyway would like to give a definitive answer on this before we move forward with our decision making of what direction we take. What we found so far has been pretty gray. I understand we could build our own pooling if say we wanted to use pyorient but maybe this is unnecessary with the capabilities of OrientDB. We can't answer that yet. Anyone?

[Ostico]

Hi @jbrow70 , please refer to this thread:

https://github.com/mogui/pyorient/issues/145

[jbrow70]

Hi @Ostico,

Thanks for referring to that thread again. I took a look at it one more time. I think pooling would be required in our use case but all of our parallel queries would be on same database. So with that said, here's what you said in that thread:

"Otherwise you can create a pool of clients, one for database on which the client must operate and switch over them programmatically."

We'd like to do that but was hoping the pyorient driver had pooling included so we wouldn't have to build that out. If you or anyone could give us a starting point to build that pool of clients off one database that would be great or if anyone had a working example of a pool of pyorient clients on the same db available that they're willing to share, we'd appreciate it.

Rather not reinvent the wheel if it's been done.

[Ostico]

Hi @jbrow70 ,

a client by design is not thought to have a pooling embedded in my opinion, it's better to have a library than provide such functionality, something like apache https://github.com/apache/commons-dbcp

There is another discussion for that: https://github.com/mogui/pyorient/issues/51 https://github.com/mogui/pyorient/issues/64

I hope this can be useful.

[jbrow70]

Hi @Ostico,

Eventlit looks interesting, on thread #51. Will take a look... #64 has some good information too. Thanks! I do agree that a library should manage a pool of connections. However, I see many drivers/clients/etc. include pooling option, especially for an enterprise ready driver. Even the Java orientdb libraries include a pool, Graph-Factory. We were hoping pyorient had something similar, but it appears not, and this type of functionality would have to be built by us it seems if we wanted to go this route.

Thanks for your help in this manner. We'll continue to investigate our options.

-Jordan