AQL: too many collections/shards

[kuzeko]

I'm trying to load a graph with tinkerpop, and got this error:

Caused by: com.arangodb.ArangoDBException: Response: 400, Error: 1522 - AQL: too many collections/shards (while parsing)

what does it mean?

The graph I'm loading has some many thousands node labels and edge types

This is where it happens:

g.V()
        .has(ds.uid_field, edge.source)
    .outE(edge.label)
    .where(__.inV().has(ds.uid_field, edge.target))
    .next().id()

[dothebart]

the number of collections outdone for this error message is 2048. @arcanefoam may know more about the other implementation details in this driver.

[kuzeko]

It means arango cannot load more than 2048 edge or node labels?

[dothebart]

if you have each of them in your own collection - then you can't do that. Collections are what Tables are in SQL solutions, so that is quiet a lot.

[arcanefoam]

Seems more like an Arango exception due to limits mentioned by Wilfred than the tinker pop implementation.

On Fri, 21 Aug 2020, 09:14 Wilfried Goesgens, notifications@github.com wrote:

if you have each of them in your own collection - then you can't do that. Collections are what Tables are in SQL solutions, so that is quiet a lot.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/63#issuecomment-678108889, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3MVAJRXJIGM42BXGRDSBYUGJANCNFSM4QGIW6GA .

[kuzeko]

It seems I have no direct control on where Arango wants to store edge/node labels...

[arcanefoam]

How are you providing the graph schema? The Arango tinkerpop provider does expect one collection per label. One option would be to group similar labels in a super-label and then use a node/edge property for the specific label

On Fri, 21 Aug 2020, 09:48 M. Lissandrini, notifications@github.com wrote:

It seems I have no direct control on where Arango wants to store edge/node labels...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/63#issuecomment-678123317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3JSEVVJCFTB5W6S2N3SBYYF7ANCNFSM4QGIW6GA .

[kuzeko]

Then I need to write ad-hoc queries because I cannot query with common edge/node label query construct.

If arango cannot handle more than 2048 collections and if there is no way to handle node/edge labels without declaring a collection for each one, it means that this tinkerpop implementation requires a new feature then?

[arcanefoam]

A new feature can be a solution; implementatuon wise this would use my suggestion to 'mock' infinite labels. Note that this might have a performance impact as all node/edge search by label will now have an additional property filer query.

Can you add a separate issue to request the feature? We could sort ou the details there. Also, it would be very helpful if you can attempt a fix and provide a PR :).

On Fri, 21 Aug 2020, 11:46 M. Lissandrini, notifications@github.com wrote:

Then I need to write ad-hoc queries because I cannot query with common edge/node label query construct.

If arango cannot handle more than 2048 collections and if there is no way to handle node/edge labels without declaring a collection for each one, it means that this tinkerpop implementation requires a new feature then?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ArangoDB-Community/arangodb-tinkerpop-provider/issues/63#issuecomment-678222996, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQOU3OUZMESRP37OOG67N3SBZGAZANCNFSM4QGIW6GA .

[arcanefoam]

Closing this as bug for feature request was opened.

[kuzeko]

I would suggest this as well to convert to a feature that catches that error and outputs a more meaningful error message