search

amazon-archives / dynamodb-janusgraph-storage-backend

The Amazon DynamoDB Storage Backend for JanusGraph
Apache License 2.0
446 stars 99 forks source link

Parallel requests multiplies query time #197

Open brianfoody opened 7 years ago

brianfoody commented 7 years ago

Hi. Loving everything about this so far until I actually hit production load. I'm getting really slow load times. First off my initial query is running at 9-10 seconds which is a little slow for 415 vertices and 409 edges.

However the real problem comes in when I issue parallel requests. This basically multiplies the time by 10 seconds for each request which means;

2 requests in parallel = 20 second load times for both queries. 3 requests in parallel = 30 second load times for both queries.

Is there a way to get around this? This happens for both use cases of the gremlin server - websocket connection or HTTP.

amcp commented 7 years ago

Try playing with this parameter: https://github.com/awslabs/dynamodb-janusgraph-storage-backend/blob/master/src/main/java/com/amazon/janusgraph/diskstorage/dynamodb/Constants.java#L253

amcp commented 7 years ago

@brianfoody That option is only for full graph scans. Probably not what you are doing but I can't know for sure without seeing a test case. Can you add a scenario test that duplicates this behavior with sanitized data, load and traversal code?