penberg / limbo

Limbo is a work-in-progress, in-process OLTP database management system, compatible with SQLite.
MIT License
960 stars 53 forks source link

Refactor join processing / support multiway joins #204

Closed jussisaurio closed 1 month ago

jussisaurio commented 1 month ago

Features:

Architectural changes:


Example query: explain select u.first_name, p.name, p2.name from users u left join products p on p.name = u.first_name join products p2 on length(p2.name) > 8 where u.first_name = 'Franklin';

Codegen (limbo):

> explain select u.first_name, p.name, p2.name from users u left join products p on p.name = u.first_name join products p2 on length(p2.name) > 8 where u.first_name = 'Franklin';
addr  opcode             p1    p2    p3    p4             p5  comment
----  -----------------  ----  ----  ----  -------------  --  -------
0     Init               0     37    0                    0   Start at 37
1     OpenReadAsync      0     2     0                    0   root=2
2     OpenReadAwait      0     0     0                    0
3     OpenReadAsync      1     3     0                    0   root=3
4     OpenReadAwait      0     0     0                    0
5     OpenReadAsync      2     3     0                    0   root=3
6     OpenReadAwait      0     0     0                    0
7     RewindAsync        0     0     0                    0
8     RewindAwait        0     36    0                    0
9       Column           0     1     2                    0   r[2]=users.first_name
10      Ne               2     3     34                   0   if r[2]!=r[3] goto 34
11      Integer          0     1     0                    0   r[1]=0
12      RewindAsync      1     0     0                    0
13      RewindAwait      1     34    0                    0
14        Column         1     1     4                    0   r[4]=products.name
15        Column         0     1     5                    0   r[5]=users.first_name
16        Ne             4     5     29                   0   if r[4]!=r[5] goto 29
17        Integer        1     1     0                    0   r[1]=1
18        RewindAsync    2     0     0                    0
19        RewindAwait    2     29    0                    0
20          Column       2     1     8                    0   r[8]=products.name
21          Function     1     8     6     length         0   r[6]=func(r[8..])
22          Le           6     7     27                   0   if r[6]<=r[7] goto 27
23          Column       0     1     9                    0   r[9]=users.first_name
24          Column       1     1     10                   0   r[10]=products.name
25          Column       2     1     11                   0   r[11]=products.name
26          ResultRow    9     3     0                    0   output=r[9..11]
27        NextAsync      2     0     0                    0
28        NextAwait      2     19    0                    0
29      NextAsync        1     0     0                    0
30      NextAwait        1     13    0                    0
31      IfPos            1     34    0                    0   r[1]>0 -> r[1]-=0, goto 34
32      NullRow          1     0     0                    0   Set cursor 1 to a (pseudo) NULL row
33      Goto             0     17    0                    0
34    NextAsync          0     0     0                    0
35    NextAwait          0     8     0                    0
36    Halt               0     0     0                    0
37    Transaction        0     0     0                    0
38    String8            0     3     0     Franklin       0   r[3]='Franklin'
39    Integer            8     7     0                    0   r[7]=8
40    Goto               0     1     0                    0

Codegen (sqlite):

sqlite> pragma automatic_index = 0;
sqlite> explain select u.first_name, p.name, p2.name from users u left join products p on p.name = u.first_name join products p2 on length(p2.name) > 8 where u.first_name = 'Franklin';
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     28    0                    0   Start at 28
1     OpenRead       0     2     0     2              0   root=2 iDb=0; users
2     OpenRead       1     3     0     2              0   root=3 iDb=0; products
3     OpenRead       2     3     0     2              0   root=3 iDb=0; products
4     Rewind         0     27    0                    0
5       Column         0     1     1                    0   r[1]= cursor 0 column 1
6       Ne             2     26    1     BINARY-8       82  if r[1]!=r[2] goto 26
7       Integer        0     3     0                    0   r[3]=0; init LEFT JOIN match flag
8       Rewind         1     23    0                    0
9         Column         1     1     1                    0   r[1]= cursor 1 column 1
10        Column         0     1     4                    0   r[4]= cursor 0 column 1
11        Ne             4     22    1     BINARY-8       81  if r[1]!=r[4] goto 22
12        Integer        1     3     0                    0   r[3]=1; record LEFT JOIN hit
13        Rewind         2     23    0                    0
14          Column         2     1     1                    64  r[1]= cursor 2 column 1
15          Function       0     1     4     length(1)      0   r[4]=func(r[1])
16          Le             5     21    4                    80  if r[4]<=r[5] goto 21
17          Column         0     1     6                    0   r[6]= cursor 0 column 1
18          Column         1     1     7                    0   r[7]= cursor 1 column 1
19          Column         2     1     8                    0   r[8]= cursor 2 column 1
20          ResultRow      6     3     0                    0   output=r[6..8]
21        Next           2     14    0                    1
22      Next           1     9     0                    1
23      IfPos          3     26    0                    0   if r[3]>0 then r[3]-=0, goto 26
24      NullRow        1     0     0                    0
25      Goto           0     12    0                    0
26    Next           0     5     0                    1
27    Halt           0     0     0                    0
28    Transaction    0     0     2     0              1   usesStmtJournal=0
29    String8        0     2     0     Franklin       0   r[2]='Franklin'
30    Integer        8     5     0                    0   r[5]=8
31    Goto           0     1     0                    0