More than 2 seconds data retrieval time

wuuuaaaa commented 3 years ago

Hi,

I am trying to retrieve data from Neo4j using this lib. A simple test code as below, generally it took 2 secs to get the result. If using neo4j driver, it will take up to 5 secs. The MATCH query running in neo4j browser completed in few milliseconds. Is there anything missing in my php library configuration?

Neo4j version 4.2.1 Php 7.4 Laravel Framework

$client = ClientBuilder::create()
->withDriver('bolt', 'bolt://neo4j:password@localhost?database=test') // creates a bolt driver
->withDriver('http', 'http://localhost:7474?database=test', Authenticate::basic('neo4j', 'password')) 
->withDriver('neo4j', 'neo4j://localhost:7687?database=test', Authenticate::basic('neo4j', 'password')) 
->withDefaultDriver('bolt')
->build();

$client->runStatement(new Statement('MATCH (d:Data {created_by: $created_by}) RETURN d LIMIT 1',  ['created_by' => Auth::id()]), 'bolt');

regards,

fbiville commented 3 years ago

Hi @worldpeach, thanks for the report. Could you also share how you measure the execution time?

wuuuaaaa commented 3 years ago

Hi thanks for your reply.

i do a diff by getting time before and after the query run.

$start_date = new \DateTime("now");  

$client = ClientBuilder::create().....
$result = $client->runStatement(.....)

$since_start = $start_date->diff(new \DateTime("now"));
\Log::debug('neo4j data retrieval time:: '. print_r($since_start, true));

Output for time difference

neo4j data retrieval time::  DateInterval Object
(
    [y] => 0
    [m] => 0
    [d] => 0
    [h] => 0
    [i] => 0
    [s] => 2
    [f] => 0.120213
    [weekday] => 0
    [weekday_behavior] => 0
    [first_last_day_of] => 0
    [invert] => 0
    [days] => 0
    [special_type] => 0
    [special_amount] => 0
    [have_weekday_relative] => 0
    [have_special_relative] => 0
)

transistive commented 3 years ago

Hello @worldpeach,

Thank you for reaching out regarding this issue. Can you confirm the same query is fast when using the web browser? If you have a standard deployment of neo4j locally, you should be able to reach it through the web browser when going to localhost:7474.

Thank you,

Ghlen

wuuuaaaa commented 3 years ago

Yup, by using neo4j browser, it take less than 10ms to complete the query.

transistive commented 3 years ago

Interesting! Can you also check if the HTTP driver also takes this long?

wuuuaaaa commented 3 years ago

Time taken for HTTP and Neo4j scheme took longer. About 5 secs

additional i have enable ext-ds ds version 1.3.0

fbiville commented 3 years ago

It might be worth running the program with a profiler. 10 years ago, I was using XDebug for this, there may be better alternatives now, I'm not sure. WDYT @transistive?

transistive commented 3 years ago

Hmm, the fact that all drivers work slowly makes me think it is a networking issue, not a driver issue. The architecture of the HTTP and bolt driver is very different, so it would mean they both have the same bug. This is the simplest piece of code I can think of to check this:

$client = \Http\Discovery\Psr18ClientDiscovery::find();
$factory = \Http\Discovery\Psr17FactoryDiscovery::findRequestFactory();

$url = 'http://localhost:7474'; // change the url to the one relevant for your setup

$response = $client->sendRequest($factory->createRequest('GET', $url));

echo $response->getBody();

The output should be:

{
  "bolt_routing" : "neo4j://neo4j:7687",
  "transaction" : "http://neo4j:7474/db/{databaseName}/tx",
  "bolt_direct" : "bolt://neo4j:7687",
  "neo4j_version" : "4.3.0",
  "neo4j_edition" : "enterprise"
}

If this code runs quickly, the bug is most likely on the driver side. If it runs slowly, it is a driver issue. In that case, can you give me more information about your setup @worldpeach?

wuuuaaaa commented 3 years ago

seems like Psr client is the root cause. it took about 2 secs to get the response. while i tried with HTTP client come with Laravel, it took about 0.3 secs to get response. https://laravel.com/docs/8.x/http-client

$response = \Illuminate\Support\Facades\Http::get('http://localhost:7474/');
echo $response->body();

transistive commented 3 years ago

This is very strange. The only thing I can think of is that the ext-streams is not behaving properly. Can you send me the output when you run php -i from your environment?

Thank you,

Ghlen

wuuuaaaa commented 3 years ago

Here you are.

transistive commented 3 years ago

I am going to be honest, this problem has gotten me completely stumped. I will need to be able to completely reproduce the setup.

If your setup is dockerized, you can post the docker/docker-compose files here. If not, it might be easiest to just schedule a call where you share your screen. You can send me available time slots to ghlen@pm.me

My apologies for the long wait!

tim-hanssen commented 3 years ago

Hi @transistive we spend all day debugging why our new app (the former one was build on Graphaware plugin) was so extremely slow. Especially if there was much data to respond.

We have profiled the application using xdebug. The most of our run time is spent in the /DS/* and the client code.

(DS\Map->keyAreEqual) alone is about 8% of our runtime.

If you want, we can do a screen share or something, since you are also speaking Dutch :)

milanchheda commented 3 years ago

Interested to know if you were to able to fix as I m facing a similar problem where cypher query is taking more time when executed through run() and is taking milliseconds when executed directly in neo4j from browser.

transistive commented 3 years ago

Closed due to inactivity and a considerable increase in performance because of referenced PRs.

neo4j-php / neo4j-php-client

More than 2 seconds data retrieval time #66