vesoft-inc / nebula

A distributed, fast open-source graph database featuring horizontal scalability and high availability
https://nebula-graph.io
Apache License 2.0
10.61k stars 1.19k forks source link

The problem of using skip and limit at the same time #5254

Closed foesa-yang closed 1 year ago

foesa-yang commented 1 year ago

Please check the FAQ documentation before raising an issue

Describe the bug (required) First of all, there is no problem using limit in match. But when limit and skip are used at the same time, the returned data is limited first and then skipped. I think the returned data should be skip first and then limit.

(root@nebula) [basketballplayer]> match ()-[e]->() return e limit 20
+------------------------------------------------------------------------------+
| e                                                                            |
+------------------------------------------------------------------------------+
| [:serve "player102"->"team203" @0 {end_year: 2015, start_year: 2006}]        |
| [:serve "player102"->"team204" @0 {end_year: 2019, start_year: 2015}]        |
| [:follow "player102"->"player100" @0 {degree: 75}]                           |
| [:follow "player102"->"player101" @0 {degree: 75}]                           |
| [:serve "player106"->"team204" @0 {end_year: 2018, start_year: 2014}]        |
| [:serve "player106"->"team212" @0 {end_year: 2019, start_year: 2018}]        |
| [:serve "player115"->"team210" @0 {end_year: 2016, start_year: 1996}]        |
| [:serve "player129"->"team216" @0 {end_year: 2018, start_year: 2017}]        |
| [:serve "player129"->"team221" @0 {end_year: 2017, start_year: 2016}]        |
| [:serve "player129"->"team229" @20032016 {end_year: 2016, start_year: 2003}] |
| [:serve "player129"->"team229" @20182019 {end_year: 2019, start_year: 2018}] |
| [:follow "player129"->"player116" @0 {degree: 90}]                           |
| [:follow "player129"->"player121" @0 {degree: 90}]                           |
| [:follow "player129"->"player128" @0 {degree: 90}]                           |
| [:serve "player138"->"team204" @0 {end_year: 2019, start_year: 2016}]        |
| [:serve "player138"->"team210" @0 {end_year: 2014, start_year: 2008}]        |
| [:serve "player138"->"team212" @0 {end_year: 2008, start_year: 2001}]        |
| [:serve "player138"->"team221" @0 {end_year: 2016, start_year: 2014}]        |
| [:serve "player138"->"team225" @0 {end_year: 2020, start_year: 2019}]        |
| [:follow "player138"->"player115" @0 {degree: 90}]                           |
+------------------------------------------------------------------------------+
Got 20 rows (time spent 3067/3956 us)

I should return 10 rows of data except the first one instead of 9 rows.

(root@nebula) [basketballplayer]> match ()-[e]->() return e skip 1 limit 10
+------------------------------------------------------------------------------+
| e                                                                            |
+------------------------------------------------------------------------------+
| [:serve "player102"->"team204" @0 {end_year: 2019, start_year: 2015}]        |
| [:follow "player102"->"player100" @0 {degree: 75}]                           |
| [:follow "player102"->"player101" @0 {degree: 75}]                           |
| [:serve "player106"->"team204" @0 {end_year: 2018, start_year: 2014}]        |
| [:serve "player106"->"team212" @0 {end_year: 2019, start_year: 2018}]        |
| [:serve "player115"->"team210" @0 {end_year: 2016, start_year: 1996}]        |
| [:serve "player129"->"team216" @0 {end_year: 2018, start_year: 2017}]        |
| [:serve "player129"->"team221" @0 {end_year: 2017, start_year: 2016}]        |
| [:serve "player129"->"team229" @20032016 {end_year: 2016, start_year: 2003}] |
+------------------------------------------------------------------------------+
Got 9 rows (time spent 5040/6795 us)

Your Environments (required)

nebula: v3.3

How To Reproduce(required)

Steps to reproduce the behavior:

  1. Step 1
  2. Step 2
  3. Step 3

Expected behavior

Additional context

Hester-Gu commented 1 year ago
(root@nebula) [nba]> profile match ()-[e]->() return e skip 1 limit 10
+---------------------------------------------------------------------------+
| e                                                                         |
+---------------------------------------------------------------------------+
| [:like "Dwyane Wade"->"Chris Paul" @0 {likeness: 90}]                     |
| [:like "Dwyane Wade"->"LeBron James" @0 {likeness: 90}]                   |
| [:serve "Dwyane Wade"->"Bulls" @0 {end_year: 2017, start_year: 2016}]     |
| [:serve "Dwyane Wade"->"Cavaliers" @0 {end_year: 2018, start_year: 2017}] |
| [:serve "Dwyane Wade"->"Heat" @0 {end_year: 2016, start_year: 2003}]      |
| [:serve "Dwyane Wade"->"Heat" @1 {end_year: 2019, start_year: 2018}]      |
| [:like "Tony Parker"->"LaMarcus Aldridge" @0 {likeness: 90}]              |
| [:like "Tony Parker"->"Manu Ginobili" @0 {likeness: 95}]                  |
| [:like "Tony Parker"->"Tim Duncan" @0 {likeness: 95}]                     |
+---------------------------------------------------------------------------+
Got 9 rows (time spent 2019/3462 us)

Execution Plan (optimize time 277 us)

-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| id | name           | dependencies | profiling data                                      | operator info                   |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| 13 | Project        | 18           | ver: 0, rows: 9, execTime: 8us, totalTime: 8us      | outputVar: {                    |
|    |                |              |                                                     |   "colNames": [                 |
|    |                |              |                                                     |     "e"                         |
|    |                |              |                                                     |   ],                            |
|    |                |              |                                                     |   "type": "DATASET",            |
|    |                |              |                                                     |   "name": "__Limit_8"           |
|    |                |              |                                                     | }                               |
|    |                |              |                                                     | inputVar: __Limit_14            |
|    |                |              |                                                     | columns: [                      |
|    |                |              |                                                     |   "$-.e[0] AS e"                |
|    |                |              |                                                     | ]                               |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| 18 | Limit          | 19           | ver: 0, rows: 9, execTime: 3us, totalTime: 4us      | outputVar: {                    |
|    |                |              |                                                     |   "colNames": [                 |
|    |                |              |                                                     |     "__VAR_0",                  |
|    |                |              |                                                     |     "e",                        |
|    |                |              |                                                     |     "__VAR_1"                   |
|    |                |              |                                                     |   ],                            |
|    |                |              |                                                     |   "type": "DATASET",            |
|    |                |              |                                                     |   "name": "__Limit_14"          |
|    |                |              |                                                     | }                               |
|    |                |              |                                                     | inputVar: __AppendVertices_19   |
|    |                |              |                                                     | offset: 1                       |
|    |                |              |                                                     | count: 10                       |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| 19 | AppendVertices | 17           | {                                                   | outputVar: {                    |
|    |                |              | ver: 0, rows: 10, execTime: 122us, totalTime: 420us |   "colNames": [                 |
|    |                |              | resp[0]: {                                          |     "e",                        |
|    |                |              |   "exec": "86(us)",                                 |     "__VAR_1"                   |
|    |                |              |   "host": "storage-0.storage:9779",                 |   ],                            |
|    |                |              |   "total": "303(us)"                                |   "type": "DATASET",            |
|    |                |              | }                                                   |   "name": "__AppendVertices_19" |
|    |                |              | total_rpc: 352(us)                                  | }                               |
|    |                |              | }                                                   | inputVar: __Project_17          |
|    |                |              |                                                     | space: 4                        |
|    |                |              |                                                     | dedup: true                     |
|    |                |              |                                                     | limit: 11                       |
|    |                |              |                                                     | filter:                         |
|    |                |              |                                                     | orderBy: []                     |
|    |                |              |                                                     | src: none_direct_dst($-.e)      |
|    |                |              |                                                     | props: [                        |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_tag"                    |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "tagId": 5                  |
|    |                |              |                                                     |   },                            |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_tag"                    |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "tagId": 7                  |
|    |                |              |                                                     |   },                            |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_tag"                    |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "tagId": 6                  |
|    |                |              |                                                     |   }                             |
|    |                |              |                                                     | ]                               |
|    |                |              |                                                     | exprs:                          |
|    |                |              |                                                     | vertex_filter:                  |
|    |                |              |                                                     | if_track_previous_path: true    |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| 17 | Project        | 16           | ver: 0, rows: 10, execTime: 37us, totalTime: 38us   | outputVar: {                    |
|    |                |              |                                                     |   "colNames": [                 |
|    |                |              |                                                     |     "e"                         |
|    |                |              |                                                     |   ],                            |
|    |                |              |                                                     |   "type": "DATASET",            |
|    |                |              |                                                     |   "name": "__Project_17"        |
|    |                |              |                                                     | }                               |
|    |                |              |                                                     | inputVar: __ScanEdges_16        |
|    |                |              |                                                     | columns: [                      |
|    |                |              |                                                     |   "[EDGE] AS e"                 |
|    |                |              |                                                     | ]                               |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
| 16 | ScanEdges      | 3            | {                                                   | outputVar: {                    |
|    |                |              | ver: 0, rows: 10, execTime: 171us, totalTime: 927us |   "colNames": [                 |
|    |                |              | resp[0]: {                                          |     "e"                         |
|    |                |              |   "exec": "478(us)",                                |   ],                            |
|    |                |              |   "host": "storage-0.storage:9779",                 |   "type": "DATASET",            |
|    |                |              |   "total": "815(us)"                                |   "name": "__ScanEdges_16"      |
|    |                |              | }                                                   | }                               |
|    |                |              | total_rpc: 884(us)                                  | inputVar:                       |
|    |                |              | }                                                   | space: 4                        |
|    |                |              |                                                     | dedup: true                     |
|    |                |              |                                                     | limit: 10                       |
|    |                |              |                                                     | filter:                         |
|    |                |              |                                                     | orderBy: []                     |
|    |                |              |                                                     | props: [                        |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_src",                   |
|    |                |              |                                                     |       "_type",                  |
|    |                |              |                                                     |       "_rank",                  |
|    |                |              |                                                     |       "_dst",                   |
|    |                |              |                                                     |       "start_year",             |
|    |                |              |                                                     |       "end_year"                |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "type": 10                  |
|    |                |              |                                                     |   },                            |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_src",                   |
|    |                |              |                                                     |       "_type",                  |
|    |                |              |                                                     |       "_rank",                  |
|    |                |              |                                                     |       "_dst",                   |
|    |                |              |                                                     |       "likeness"                |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "type": 8                   |
|    |                |              |                                                     |   },                            |
|    |                |              |                                                     |   {                             |
|    |                |              |                                                     |     "props": [                  |
|    |                |              |                                                     |       "_src",                   |
|    |                |              |                                                     |       "_type",                  |
|    |                |              |                                                     |       "_rank",                  |
|    |                |              |                                                     |       "_dst",                   |
|    |                |              |                                                     |       "start_year",             |
|    |                |              |                                                     |       "end_year"                |
|    |                |              |                                                     |     ],                          |
|    |                |              |                                                     |     "type": 9                   |
|    |                |              |                                                     |   }                             |
|    |                |              |                                                     | ]                               |
|    |                |              |                                                     | exprs:                          |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------
|  3 | Start          |              | ver: 0, rows: 0, execTime: 0us, totalTime: 9us      | outputVar: {                    |
|    |                |              |                                                     |   "colNames": [],               |
|    |                |              |                                                     |   "type": "DATASET",            |
|    |                |              |                                                     |   "name": "__Start_3"           |
|    |                |              |                                                     | }                               |
-----+----------------+--------------+-----------------------------------------------------+----------------------------------

Fri, 13 Jan 2023 09:17:28 UTC
Shylock-Hg commented 1 year ago

I can't reproduce this

(root@nebula) [nba]> profile match ()-[e]->() return e skip 1 limit 10
+------------------------------------------------------------------------------+
| e                                                                            |
+------------------------------------------------------------------------------+
| [:serve "Amar'e Stoudemire"->"Heat" @0 {end_year: 2016, start_year: 2015}]   |
| [:serve "Amar'e Stoudemire"->"Knicks" @0 {end_year: 2015, start_year: 2010}] |
| [:serve "Amar'e Stoudemire"->"Suns" @0 {end_year: 2010, start_year: 2002}]   |
| [:like "Aron Baynes"->"Tim Duncan" @0 {likeness: 80}]                        |
| [:serve "Aron Baynes"->"Celtics" @0 {end_year: 2019, start_year: 2017}]      |
| [:serve "Aron Baynes"->"Pistons" @0 {end_year: 2017, start_year: 2015}]      |
| [:serve "Aron Baynes"->"Spurs" @0 {end_year: 2015, start_year: 2013}]        |
| [:like "Ben Simmons"->"Joel Embiid" @0 {likeness: 80}]                       |
| [:serve "Ben Simmons"->"76ers" @0 {end_year: 2019, start_year: 2016}]        |
| [:like "Blake Griffin"->"Chris Paul" @0 {likeness: -1}]                      |
+------------------------------------------------------------------------------+
Got 10 rows (time spent 24226/26290 us)

Execution Plan (optimize time 8058 us)

-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| id | name           | dependencies | profiling data                                        | operator info                   |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| 13 | Project        | 14           | ver: 0, rows: 10, execTime: 59us, totalTime: 62us     | outputVar: {                    |
|    |                |              |                                                       |   "colNames": [                 |
|    |                |              |                                                       |     "e"                         |
|    |                |              |                                                       |   ],                            |
|    |                |              |                                                       |   "type": "DATASET",            |
|    |                |              |                                                       |   "name": "__Limit_8"           |
|    |                |              |                                                       | }                               |
|    |                |              |                                                       | inputVar: __Limit_14            |
|    |                |              |                                                       | columns: [                      |
|    |                |              |                                                       |   "$-.e[0] AS e"                |
|    |                |              |                                                       | ]                               |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| 14 | Limit          | 15           | ver: 0, rows: 10, execTime: 36us, totalTime: 43us     | outputVar: {                    |
|    |                |              |                                                       |   "colNames": [                 |
|    |                |              |                                                       |     "__VAR_0",                  |
|    |                |              |                                                       |     "e",                        |
|    |                |              |                                                       |     "__VAR_1"                   |
|    |                |              |                                                       |   ],                            |
|    |                |              |                                                       |   "type": "DATASET",            |
|    |                |              |                                                       |   "name": "__Limit_14"          |
|    |                |              |                                                       | }                               |
|    |                |              |                                                       | inputVar: __AppendVertices_15   |
|    |                |              |                                                       | offset: 1                       |
|    |                |              |                                                       | count: 10                       |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| 15 | AppendVertices | 17           | {                                                     | outputVar: {                    |
|    |                |              | ver: 0, rows: 11, execTime: 2443us, totalTime: 5145us |   "colNames": [                 |
|    |                |              | resp[0]: {                                            |     "e",                        |
|    |                |              |   "exec": "1737(us)",                                 |     "__VAR_1"                   |
|    |                |              |   "host": "127.0.0.1:18841",                          |   ],                            |
|    |                |              |   "total": "3935(us)"                                 |   "type": "DATASET",            |
|    |                |              | }                                                     |   "name": "__AppendVertices_15" |
|    |                |              | total_rpc: 4580(us)                                   | }                               |
|    |                |              | }                                                     | inputVar: __Project_17          |
|    |                |              |                                                       | space: 2                        |
|    |                |              |                                                       | dedup: true                     |
|    |                |              |                                                       | limit: -1                       |
|    |                |              |                                                       | filter:                         |
|    |                |              |                                                       | orderBy: []                     |
|    |                |              |                                                       | src: none_direct_dst($-.e)      |
|    |                |              |                                                       | props: [                        |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_tag"                    |
|    |                |              |                                                       |     ],                          |
|    |                |              |                                                       |     "tagId": 3                  |
|    |                |              |                                                       |   },                            |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "tagId": 5,                 |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_tag"                    |
|    |                |              |                                                       |     ]                           |
|    |                |              |                                                       |   },                            |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_tag"                    |
|    |                |              |                                                       |     ],                          |
|    |                |              |                                                       |     "tagId": 4                  |
|    |                |              |                                                       |   }                             |
|    |                |              |                                                       | ]                               |
|    |                |              |                                                       | exprs:                          |
|    |                |              |                                                       | vertex_filter:                  |
|    |                |              |                                                       | if_track_previous_path: true    |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| 17 | Project        | 16           | ver: 0, rows: 11, execTime: 411us, totalTime: 439us   | outputVar: {                    |
|    |                |              |                                                       |   "colNames": [                 |
|    |                |              |                                                       |     "e"                         |
|    |                |              |                                                       |   ],                            |
|    |                |              |                                                       |   "type": "DATASET",            |
|    |                |              |                                                       |   "name": "__Project_17"        |
|    |                |              |                                                       | }                               |
|    |                |              |                                                       | inputVar: __ScanEdges_16        |
|    |                |              |                                                       | columns: [                      |
|    |                |              |                                                       |   "[EDGE] AS e"                 |
|    |                |              |                                                       | ]                               |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
| 16 | ScanEdges      | 3            | {                                                     | outputVar: {                    |
|    |                |              | ver: 0, rows: 11, execTime: 1278us, totalTime: 6855us |   "colNames": [                 |
|    |                |              | resp[0]: {                                            |     "e"                         |
|    |                |              |   "exec": "1659(us)",                                 |   ],                            |
|    |                |              |   "host": "127.0.0.1:18841",                          |   "name": "__ScanEdges_16",     |
|    |                |              |   "total": "5380(us)"                                 |   "type": "DATASET"             |
|    |                |              | }                                                     | }                               |
|    |                |              | total_rpc: 6450(us)                                   | inputVar:                       |
|    |                |              | }                                                     | space: 2                        |
|    |                |              |                                                       | dedup: true                     |
|    |                |              |                                                       | limit: 11                       |
|    |                |              |                                                       | filter:                         |
|    |                |              |                                                       | orderBy: []                     |
|    |                |              |                                                       | props: [                        |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_src",                   |
|    |                |              |                                                       |       "_type",                  |
|    |                |              |                                                       |       "_rank",                  |
|    |                |              |                                                       |       "_dst",                   |
|    |                |              |                                                       |       "start_year",             |
|    |                |              |                                                       |       "end_year"                |
|    |                |              |                                                       |     ],                          |
|    |                |              |                                                       |     "type": 8                   |
|    |                |              |                                                       |   },                            |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "type": 6,                  |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_src",                   |
|    |                |              |                                                       |       "_type",                  |
|    |                |              |                                                       |       "_rank",                  |
|    |                |              |                                                       |       "_dst",                   |
|    |                |              |                                                       |       "likeness"                |
|    |                |              |                                                       |     ]                           |
|    |                |              |                                                       |   },                            |
|    |                |              |                                                       |   {                             |
|    |                |              |                                                       |     "type": 7,                  |
|    |                |              |                                                       |     "props": [                  |
|    |                |              |                                                       |       "_src",                   |
|    |                |              |                                                       |       "_type",                  |
|    |                |              |                                                       |       "_rank",                  |
|    |                |              |                                                       |       "_dst",                   |
|    |                |              |                                                       |       "start_year",             |
|    |                |              |                                                       |       "end_year"                |
|    |                |              |                                                       |     ]                           |
|    |                |              |                                                       |   }                             |
|    |                |              |                                                       | ]                               |
|    |                |              |                                                       | exprs:                          |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------
|  3 | Start          |              | ver: 0, rows: 0, execTime: 1us, totalTime: 26us       | outputVar: {                    |
|    |                |              |                                                       |   "colNames": [],               |
|    |                |              |                                                       |   "type": "DATASET",            |
|    |                |              |                                                       |   "name": "__Start_3"           |
|    |                |              |                                                       | }                               |
-----+----------------+--------------+-------------------------------------------------------+----------------------------------

Mon, 16 Jan 2023 09:59:40 CST
Shylock-Hg commented 1 year ago

Please run query desc space nba

Shylock-Hg commented 1 year ago

run nebula-graphd --version

Shylock-Hg commented 1 year ago

It's ok in latest version.

Hester-Gu commented 1 year ago

new rc is can't reproduce, use the last rc pkg, maybe it will be fixed