Query not returning expected data after upgrade from 3.5.16 community to 3.5.19 enterprise

achantavy commented 4 years ago

Hi there, I observed some unexpected query behaviors. Please see this forum thread for full context.

Neo4j version

3.5.19 Enterprise

Operating system: (for example Windows 95/Ubuntu 16.04)

Ubuntu 14.04.6 LTS

API/Driver: (for example Cypher/Java API/Python driver vX.Y.Z)

The script that fills my DB uses 1.7.6 of the Python driver

Description

I recently upgraded my DB from 3.5.16 community to 3.5.19 with the following procedure:

service stop neo4j
Uninstall Neo4j Community 3.5.16, keeping the old data
apt-get install neo4j-enterprise=3.5.19
service start neo4j

This query works as expected:

match (n:AWSTag) 
where n.key contains "aws:auto" 
return n.key, n.value 
order by n.key limit 300

as it returns a list of nodes with n.key = aws:autoscaling:groupName:

However when I just add an "s" to the key filter by running

match (n:AWSTag)
where n.key contains "aws:autos"
return n.key, n.value
order by n.key limit 300

I get 0 results:

I have a second neo4j server that was also upgraded using the same steps, running similar data and it shows this exact problem as well.

Things I've tried:

I have tried deleting all indexes on this :AWSTag node but that didn't work.
I then tried turning off the server, deleting /var/lib/neo4j/data/databases/graph.db/schema/*, and turning it on again. All other queries except for those related to :AWSTags work as expected (I think).
PROFILE of the successful query:
PROFILE of the failing query:

Here are some other observations.

Observation 1

Expected behavior

match (n:AWSTag) 
where n.key contains "aws:autos" 
return n.key, n.value 
order by n.key limit 300

should return a table that looks like

n.key	n.value
aws:autoscaling:groupName	some group name
aws:autoscaling:groupName	some other group name

Actual behavior

The above query returns zero results.

Observation 2

Expected

match (a:AWSTag)
where a.key starts with "aws:auto"
return distinct a.key

should return only keys that actually start with "aws:auto".

Actual

The above query has these results:

Observation 3

Expected

If I do an exact field search like

match (a:AWSTag{key:"aws:autoscaling:groupName"}) return a

I should get back nodes with key='aws:autoscaling:groupName, as shown in the "Description" above.

Actual

I get no results.

Additional information

Partial output of CALL db.stats.retrieve("GRAPH COUNTS")

{
  "relationships": [
    {
      "count": 8882403
    },

    {
      "relationshipType": "TAGGED",
      "count": 97277,
      "endLabel": "AWSTag"
    },
    {
      "relationshipType": "TAGGED",
      "count": 97277,
      "endLabel": "Tag"
    },

  ],
  "nodes": [
    {
      "count": 2855520
    },

    {
      "count": 66181,
      "label": "AWSTag"
    },
    {
      "count": 66181,
      "label": "Tag"
    },

  ],
  "indexes": [

    {
      "updatesSinceEstimation": 2134,
      "totalSize": 64047,
      "properties": [
        "id"
      ],
      "labels": [
        "AWSTag"
      ],
      "estimatedUniqueSize": 64047
    },

  ],
  "constraints": []
}

achantavy commented 4 years ago

Just to add another data point, please see this post: https://community.neo4j.com/t/really-weird-query-not-returning-expected-data-after-upgrade-from-3-5-16-community-to-3-5-19-enterprise/21672/15?u=alexchantavy

Could this issue be related to full text indexing or something? Anyway, hope I'm not sending over a red herring, but thought it was worth noting the difference.

OliviaYtterbrink commented 4 years ago

Hello @achantavy, I've looked into this a bit and think it's a browser/javascript-driver bug. Could you try running the queries via cypher-shell?

achantavy commented 4 years ago

Hi @OliviaYtterbrink, wow you're right, the query works in cypher-shell but not in the browser!

neo4j> match(n:AWSTag{key:"aws:autoscaling:groupName"}) return n.key, n.value limit 50;
+------------------------------------------------------------+
| n.key                       | n.value                      |
+------------------------------------------------------------+
| "aws:autoscaling:groupName" | "{value1}-staging"           |
| "aws:autoscaling:groupName" | "{value2}-production"        |
| "aws:autoscaling:groupName" | "{value3}-production"        |
| "aws:autoscaling:groupName" | "{value4}"                   |
| "aws:autoscaling:groupName" | "{value5}-production"        |
| "aws:autoscaling:groupName" | "{value6}"                   |
... etc ...

achantavy commented 4 years ago

Hi, is there any update on this? We're still seeing inconsistent query behavior on the web interface vs via cypher-shell.

OskarDamkjaer commented 4 years ago

Thanks for the detailed report and sorry about the delay. I've got time to look into this next week, so I'll have a new update some time next week

OskarDamkjaer commented 4 years ago

I looked into this just now and found the reason for the odd behaviour. We allow cypher queries to be prefixed with :auto for automatic transactions, the current implementation simplly removes all instances of :auto in the query. I'll fix it today, thanks again for your detailed report!

neo4j / neo4j-browser