Open an3l opened 5 years ago
Getting started with mysql
from source:
git clone https://github.com/mysql/mysql-server.git
Download boost - not needed see bellow
# This step is not needed, since we are using -DDOWNLOAD_BOOST=1, but make sure you have
libssl-dev, and pkg-config installed
$ sudo apt-get install libssl-dev apt install pkg-config
$ wget https://dl.bintray.com/boostorg/release/1.71.0/source/boost_1_71_0.tar.gz
&& tar xzvf boost_1_71_0.tar.gz && rm -rf boost_1_71_0.tar.gz
&& cd ~/boost_1_71_0/tools/build && ./boostrap.sh
&& ./b2 install --prefix=/home/anel/boost1
&& PATH=$PATH:~/boost1/bin/
$ mkdir build-mysql $ cd build-mysql/ && cmake ../mysql-server/ -DCMAKE_BUILD_TYPE=DEbug -DDOWNLOAD_BOOST=1 -DWITH_BOOST=/home/anel/boost1 $ make -j8 # haven't tried with ninja
Starting and stopping mysqld
https://dev.mysql.com/doc/refman/5.7/en/automatic-start.html
- This is the 2 step process:
1. initialize datadir to get the temporary password
$mkdir ~/mysql-datadir $ cd build-mysql/sql $ ./mysqld --initialize --basedir=. --datadir=/home/anel/mysql-datadir
A temporary password is generated for root@localhost: DIr/mZ+k4Y8&
2. start the server
$ cd build-mysql/sql/ $ ./mysqld --no-defaults --basedir=. --datadir=/home/anel/mysql-datadir --lc_messages_dir=./share --console
Start the client:
$ cd build-mysql/client/
$ ./mysql -u root -p
paste the password from above DIr/mZ+k4Y8&
$ ALTER USER 'root'@'localhost' IDENTIFIED BY 'anel123'; # test this (close mysql and open again)
$ Create table json (t json) engine=myisam # for our patch myisam tables are used !
$ insert into json values ('{"key1":"val1", "key2":"val2"}');
#### Debug mysql
```bash
>> gdb
build-mysql/sql$ gdb mysqld
r --no-defaults --basedir=. --datadir=/home/anel/workspace/mysql-datadir --lc_messages_dir=./share --console
b Value::element(size_t pos) -> sql/json_binary.cc:1184
>> client
build-mysql/client$ ./mysql -u root -p (anel123)
mysql> select * from simple_ints_table_json_object;
Test case in mysql
, note that date
is currently not working
/* ~/workspace/build-mysql/my_scripts/json_test.sql */
SET NAMES utf8;
CREATE TABLE t1 (i INT, j JSON) ENGINE=MyISAM;
SHOW CREATE TABLE t1;
INSERT INTO t1 VALUES (0, NULL);
INSERT INTO t1 VALUES (1, '{"a": 2}');
INSERT INTO t1 VALUES (2, '[1,2]');
INSERT INTO t1 VALUES (3, '{"a":"b", "c":"d","ab":"abc", "bc": ["x", "y"]}');
INSERT INTO t1 VALUES (4, '["here", ["I", "am"], "!!!"]');
INSERT INTO t1 VALUES (5, '"scalar string"');
INSERT INTO t1 VALUES (6, 'true');
INSERT INTO t1 VALUES (7, 'false');
INSERT INTO t1 VALUES (8, 'null');
INSERT INTO t1 VALUES (9, '-1');
INSERT INTO t1 VALUES (10, CAST(CAST(1 AS UNSIGNED) AS JSON));
INSERT INTO t1 VALUES (11, '32767');
INSERT INTO t1 VALUES (12, '32768');
INSERT INTO t1 VALUES (13, '-32768');
INSERT INTO t1 VALUES (14, '-32769');
INSERT INTO t1 VALUES (15, '2147483647');
INSERT INTO t1 VALUES (16, '2147483648');
INSERT INTO t1 VALUES (17, '-2147483648');
INSERT INTO t1 VALUES (18, '-2147483649');
INSERT INTO t1 VALUES (19, '18446744073709551615');
INSERT INTO t1 VALUES (20, '18446744073709551616');
INSERT INTO t1 VALUES (21, '3.14');
INSERT INTO t1 VALUES (22, '{}');
INSERT INTO t1 VALUES (23, '[]');
INSERT INTO t1 VALUES (24, CAST(CAST('2015-01-15 23:24:25' AS DATETIME) AS JSON));
INSERT INTO t1 VALUES (25, CAST(CAST('23:24:25' AS TIME) AS JSON));
# INSERT INTO t1 VALUES (26, CAST(CAST('2015-01-15' AS DATE) AS JSON)); # not working in mariadb
INSERT INTO t1 VALUES (27, CAST(TIMESTAMP'2015-01-15 23:24:25' AS JSON));
#INSERT INTO t1 VALUES (28, CAST(ST_GeomFromText('POINT(1 1)') AS JSON));
# auto-convert to utf8mb4
INSERT INTO t1 VALUES (29, CAST('[]' AS CHAR CHARACTER SET 'ascii'));
INSERT INTO t1 VALUES (30, CAST(x'cafe' AS JSON));
INSERT INTO t1 VALUES (31, CAST(x'cafebabe' AS JSON));
Use it in mysql
:
>> use test;
>> source ../my_scripts/json_test.sql
Compiling issue - >aside of above: Did have some problems while compiling from source:
1. git clone and in the source directory check 10.4, configure and compile it.
2. git checkout 10.3 git clean -dffx, git reset --hard HEAD, it will work cmake and compiling.
But if we after 1. checkout to 10.3 in source and build files in build-10.3
directory error will be raised because of libmariadb
:
nel@ubuntu:~/workspace/build-10.3$ ninja
[867/1257] Building C object tests/CMakeFiles/mysql_client_test.dir/mysql_client_test.c.o
FAILED: tests/CMakeFiles/mysql_client_test.dir/mysql_client_test.c.o
/usr/bin/cc -DHAVE_CONFIG_H -DMYSQL_CLIENT -D_FILE_OFFSET_BITS=64 -I/home/anel/workspace/server/libmariadb/include -Ilibmariadb/include -Iinclude -I/home/anel/workspace/server/include -I/home/anel/workspace/server/client -pie -fPIC -Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4 -fPIC -g -DENABLED_DEBUG_SYNC -ggdb3 -DSAFE_MUTEX -Wall -Wdeclaration-after-statement -Wextra -Wformat-security -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare -Wno-unused-parameter -Wvla -Wwrite-strings -Werror -MD -MT tests/CMakeFiles/mysql_client_test.dir/mysql_client_test.c.o -MF tests/CMakeFiles/mysql_client_test.dir/mysql_client_test.c.o.d -o tests/CMakeFiles/mysql_client_test.dir/mysql_client_test.c.o -c /home/anel/workspace/server/tests/mysql_client_test.c
In file included from /home/anel/workspace/server/tests/mysql_client_fw.c:24:0,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
include/mysql_version.h:14:0: error: "MYSQL_SERVER_VERSION" redefined [-Werror]
#define MYSQL_SERVER_VERSION "10.3.15-MariaDB"
In file included from /home/anel/workspace/server/libmariadb/include/mysql.h:60:0,
from /home/anel/workspace/server/tests/mysql_client_fw.c:18,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
/home/anel/workspace/server/libmariadb/include/mariadb_version.h:21:0: note: this is the location of the previous definition
#define MYSQL_SERVER_VERSION "10.4.5-MariaDB"
In file included from /home/anel/workspace/server/tests/mysql_client_fw.c:24:0,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
include/mysql_version.h:16:0: error: "MARIADB_BASE_VERSION" redefined [-Werror]
#define MARIADB_BASE_VERSION "mariadb-10.3"
In file included from /home/anel/workspace/server/libmariadb/include/mysql.h:60:0,
from /home/anel/workspace/server/tests/mysql_client_fw.c:18,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
/home/anel/workspace/server/libmariadb/include/mariadb_version.h:14:0: note: this is the location of the previous definition
#define MARIADB_BASE_VERSION "mariadb-10.4"
In file included from /home/anel/workspace/server/tests/mysql_client_fw.c:24:0,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
include/mysql_version.h:19:0: error: "MYSQL_VERSION_ID" redefined [-Werror]
#define MYSQL_VERSION_ID 100315
In file included from /home/anel/workspace/server/libmariadb/include/mysql.h:60:0,
from /home/anel/workspace/server/tests/mysql_client_fw.c:18,
from /home/anel/workspace/server/tests/mysql_client_test.c:38:
/home/anel/workspace/server/libmariadb/include/mariadb_version.h:20:0: note: this is the location of the previous definition
#define MYSQL_VERSION_ID 100405
cc1: all warnings being treated as errors
[872/1257] Building CXX object plugin/feedback/CMakeFiles/feedback.dir/url_base.cc.o
ninja: build stopped: subcommand failed.
git change specific commit message : git rebase -i HEAD~3
[edit the commit of interest]. You will be stopped at the specific commit and will have to use git commit --amend
. After that run git rebase --continue
init_from_frm_image()
is shared in DDls so I think that we should use mysql_upgrade
in order to enable table to be altered.
Select backtrace:
#0 TABLE_SHARE::init_from_binary_frm_image (this=0x7fffe0026700, thd=0x7fffe0000cf8, write=false,
frm_image=0x7fffe0026c58 "\376\001\t\t\003", frm_length=8578)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table.cc:1868
#1 0x0000555555d35cfa in open_table_def (thd=0x7fffe0000cf8, share=0x7fffe0026700, flags=11)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table.cc:677
#2 0x0000555555e52f43 in tdc_acquire_share (thd=0x7fffe0000cf8, tl=0x7fffe00161b0, flags=3,
out_table=0x7ffff09fc2c8) at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table_cache.cc:840
#3 0x0000555555b9b885 in open_table (thd=0x7fffe0000cf8, table_list=0x7fffe00161b0, ot_ctx=0x7ffff09fc6c0)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:1831
#4 0x0000555555b9ef6d in open_and_process_table (thd=0x7fffe0000cf8, lex=0x7fffe0004af8,
tables=0x7fffe00161b0, counter=0x7ffff09fc754, flags=0, prelocking_strategy=0x7ffff09fc7d8,
has_prelocking_list=false, ot_ctx=0x7ffff09fc6c0)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:3619
#5 0x0000555555ba01e0 in open_tables (thd=0x7fffe0000cf8, options=..., start=0x7ffff09fc738,
counter=0x7ffff09fc754, flags=0, prelocking_strategy=0x7ffff09fc7d8)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:4142
#6 0x0000555555ba1f14 in open_and_lock_tables (thd=0x7fffe0000cf8, options=..., tables=0x7fffe00161b0,
derived=true, flags=0, prelocking_strategy=0x7ffff09fc7d8)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:5019
#7 0x0000555555b62309 in open_and_lock_tables (thd=0x7fffe0000cf8, tables=0x7fffe00161b0, derived=true,
flags=0) at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.h:502
#8 0x0000555555c352fc in execute_sqlcom_select (thd=0x7fffe0000cf8, all_tables=0x7fffe00161b0)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_parse.cc:6469
#9 0x0000555555c2bdf3 in mysql_execute_command (thd=0x7fffe0000cf8)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_parse.cc:3821
#10 0x0000555555c3974b in mysql_parse (thd=0x7fffe0000cf8, rawbuf=0x7fffe0015fb0 "select * from test.t1",
length=21, parser_state=0x7ffff09fd5d0, is_com_multi=false, is_next_command=false)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_parse.cc:8091
#11 0x0000555555c2676b in dispatch_command (command=COM_QUERY, thd=0x7fffe0000cf8,
packet=0x7fffe000aab9 "select * from test.t1", packet_length=21, is_com_multi=false,
Alter backtrace:
#0 TABLE_SHARE::init_from_binary_frm_image (this=0x7fffe0026f80, thd=0x7fffe0000cf8, write=false,
frm_image=0x7fffe00274d8 "\376\001\t\t\003", frm_length=8578)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table.cc:1868
#1 0x0000555555d35cfa in open_table_def (thd=0x7fffe0000cf8, share=0x7fffe0026f80, flags=11)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table.cc:677
#2 0x0000555555e52f43 in tdc_acquire_share (thd=0x7fffe0000cf8, tl=0x7fffe0016098, flags=3,
out_table=0x7ffff09fa958) at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/table_cache.cc:840
#3 0x0000555555b9b885 in open_table (thd=0x7fffe0000cf8, table_list=0x7fffe0016098, ot_ctx=0x7ffff09fad50)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:1831
#4 0x0000555555b9ef6d in open_and_process_table (thd=0x7fffe0000cf8, lex=0x7fffe0004af8,
tables=0x7fffe0016098, counter=0x7ffff09fae40, flags=0, prelocking_strategy=0x7ffff09fae60,
has_prelocking_list=false, ot_ctx=0x7ffff09fad50)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:3619
#5 0x0000555555ba01e0 in open_tables (thd=0x7fffe0000cf8, options=..., start=0x7ffff09fae08,
counter=0x7ffff09fae40, flags=0, prelocking_strategy=0x7ffff09fae60)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.cc:4142
#6 0x0000555555cf02be in open_tables (thd=0x7fffe0000cf8, tables=0x7ffff09fae08, counter=0x7ffff09fae40,
flags=0, prelocking_strategy=0x7ffff09fae60)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_base.h:250
#7 0x0000555555d08e23 in mysql_alter_table (thd=0x7fffe0000cf8, new_db=0x7fffe00053c8,
new_name=0x7fffe0005788, create_info=0x7ffff09fc490, table_list=0x7fffe0016098,
alter_info=0x7ffff09fc3d0, order_num=0, order=0x0, ignore=false)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_table.cc:9087
#8 0x0000555555d924df in Sql_cmd_alter_table::execute (this=0x7fffe00166e0, thd=0x7fffe0000cf8)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_alter.cc:488
#9 0x0000555555c3441b in mysql_execute_command (thd=0x7fffe0000cf8)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_parse.cc:6285
#10 0x0000555555c3974b in mysql_parse (thd=0x7fffe0000cf8, rawbuf=0x7fffe0015fb0 "alter table test.t1 force",
length=25, parser_state=0x7ffff09fd5d0, is_com_multi=false, is_next_command=false)
at /home/anel/workspace/mariadb/mariadb-server-10.3/sql/sql_parse.cc:8091
Literature [1] Altering table
Step 1)
Make CHECK TABLE correctly identify that we are looking at MySQL 5.7 created table with JSON columns
Examine the .frm’s server version and the field types (this is handled by init_from_binary_frm_image()
)
If the .frm’s version is for MySQL 5.7(50700) and up (but not Mariadb 10.0 (10000) and up) and we encounter the field type 245, we know we are in MySQL space and we need to upgrade the table
.frm
file: sql/table.cc : init_from_binary_frm_image()
Step 2) ?
Extend ALTER TABLE
Test 1
select * from test.mysql_json;
t
{"key1": "val1", "key2": "val2"}
alter table test.mysql_json;
select * from test.mysql_json;
t
{"key1": "val1", "key2": "val2"}
alter table test.mysql_json force;
select * from test.mysql_json;
t
$
key1key2val1val2
# Extend `Alter table <table_name> force ` @todo
drop table mysql_json;
drop table t1;
main.mysql_json_2 [ fail ] Found warnings/errors in server log file!
Test ended at 2019-04-24 01:43:56
line
2019-04-24 1:43:56 3 [ERROR] mysqld: Table rebuild required. Please do "ALTER TABLE `mysql.transaction_registry` FORCE" or dump/reload to fix it!
2019-04-24 1:43:56 3 [ERROR] mysqld: Table rebuild required. Please do "ALTER TABLE `mysql.innodb_table_stats` FORCE" or dump/reload to fix it!
2019-04-24 1:43:56 3 [ERROR] mysqld: Table rebuild required. Please do "ALTER TABLE `mysql.innodb_index_stats` FORCE" or dump/reload to fix it!
^ Found warnings in /home/anel/workspace/mariadb/builds/build-jsonbug/mysql-test/var/log/mysqld.1.err
ok
gdb testing
(gdb) p strpos
$1 = (const uchar *) 0x7fffe004e01d "\004\002\b\b"
(gdb) x strpos
0x7fffe004e01d: 0x08080204 // value of strpos : see_bellow
(gdb) p/x *(uint32_t *)0x7fffe004e01d // interpret value of strpos content as a pointer and dereference it
$9 = 0x8080204
(gdb) p (uchar *)0x7fffe004e01d
$14 = (uchar *) 0x7fffe004e01d "\004\002\b\b"
(gdb) x/s strpos
0x7fffe004e01d: "\004\002\b\b
(gdb) x/14bc strpos
0x7fffe004e01d: 4 '\004' 2 '\002' 8 '\b' 8 '\b' 0 '\000' 2 '\002' 0 '\000' 0 '\000'
0x7fffe004e025: 25 '\031' -112 '\220' 20 '\024' 0 '\000' 0 '\000' -11 '\365'
(gdb) p strpos[13]
$13 = 245 '\365'
(gdb) x/5i $pc
=> 0x555555d39de5 <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9733>: movl $0x8b,-0x3ac(%rbp)
0x555555d39def <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9743>: jmp 0x555555d39e1e <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9790>
0x555555d39df1 <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9745>: mov -0x3ac(%rbp),%eax
0x555555d39df7 <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9751>: cmp $0xf5,%al
0x555555d39df9 <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9753>: jne 0x555555d39e1e <TABLE_SHARE::init_from_binary_frm_image(THD*, bool, unsigned char const*, unsigned long)+9790>
Debuging json_mysql_binary::parse_array_or_object
, json_mysql_binary.cc: 1101
(gdb) x/s data
0x7fffe004ee37: "\002"
(gdb) p *(char *)(0x7fffe004ee37)@36
$49 = "\002\000$\000\022\000\004\000\026\000\004\000\f\032\000\f\037\000key1key2\004val1\004val2"
gdb) p *(char *)(0x7fffe004ee37)@40
$55 = "\002\000$\000\022\000\004\000\026\000\004\000\f\032\000\f\037\000key1key2\004val1\004val2\245\245\245\245"
(gdb) x/37bc (0x7fffe004ee37)
0x7fffe004ee37: 2 '\002' 0 '\000' 36 '$' 0 '\000' 18 '\022' 0 '\000' 4 '\004' 0 '\000'
0x7fffe004ee3f: 22 '\026' 0 '\000' 4 '\004' 0 '\000' 12 '\f' 26 '\032' 0 '\000' 12 '\f'
0x7fffe004ee47: 31 '\037' 0 '\000' 107 'k' 101 'e' 121 'y' 49 '1' 107 'k' 101 'e'
0x7fffe004ee4f: 121 'y' 50 '2' 4 '\004' 118 'v' 97 'a' 108 'l' 49 '1' 4 '\004'
0x7fffe004ee57: 118 'v' 97 'a' 108 'l' 50 '2' -91 '\245'
Literature: gdb doc visual gdb
Create just one test-> json_object.sql
and insert values where length of keys and values is not the same for better understanding of binaries.
create table tjson(c1 json);
insert into tjson values ('{"key1":"value_1", "key256":"value_256"}'); # object
#insert into tjson values ('["val3", "val4"]'); # array
#insert into tjson values ('["val5",{"key3":"val6", "key4":"val7"}]'); # mix
From builds-mysql/client
run
mysql> source json_object.sql
todo further ....
Debug other examples mysql_json_2
Special case for NULL
JSONB_TYPE_INT16 0x5
1 {"a": 2}
(gdb) x/13c data
0x7fffe002a33b: 1 '\001' 0 '\000' 12 '\f' 0 '\000' 11 '\v' 0 '\000' 1 '\001'
0 '\000'
0x7fffe002a343: 5 '\005' 2 '\002' 0 '\000' 97 'a' -91 '\245'
Explain:
1
0
: are number of keys(elements) in json object (little endian read_offset_or_size()
) 12
0
: are number of bytes (little endian read_offset_or_size()
) same as length11
0
: is the pointer where the key is stored (same as header_size
) 1
0
: is the length of data above5
: the type of value it holds (#define JSONB_TYPE_INT16 0x5
)2
0
: IN THIS CASE it is a value (optimization)JSONB_TYPE_INT16 0x5
create table neg_val(t json) engine=myisam;
insert into neg_val values ('{"neg":-10}');
(gdb) x/14c data
0x7fffe0053587: 1 '\001' 0 '\000' 14 '\016' 0 '\000'
11 '\v' 0 '\000' 3 '\003' 0 '\000'
0x7fffe005358f: 5 '\005' -10 '\366' -1 '\377' 110 'n' 101 'e'
103 'g'
JSONB_TYPE_INT32 0x7
JSONB_TYPE_UINT64 0xA
create table simple_ints_table_json_object (t json) engine=myisam; # for our patch myisam tables are used !
insert into simple_ints_table_json_object values ('{"neg":-1234567890, "pos":9876543210}'); # will use only json object
gdb mysql
Value::element() breakppoint leads to parse_scalar() and the length of data is 12 (uses all data of values and check for int32 is the len<4 sint3korr(data)-> uses all data? After that it goes back to json_dom.cc:1816
- Json_wrapper_object_iterator::elt()
and we can see p wr.m_value.m_int_value -> $13 = -1234567890
Since mysql is using recursion there is a double procedure of step above !
Lenght = bytes-value_offset ; First data ->data[value_offset]
(gdb) x/12c data
0x7fff7893daf7: 46 '.' -3 '\375' 105 'i' -74 '\266' -22 '\352' 22 '\026' -80 '\260' 76 'L'
0x7fff7893daff: 2 '\002' 0 '\000' 0 '\000' 0 '\000'
(gdb) x/36c data
0x7fffe004ee37: 2 '\002' 0 '\000' 36 '$' 0 '\000' 18 '\022
' 0 '\000' 3 '\003' 0 '\000'
0x7fffe004ee3f: 21 '\025' 0 '\000' 3 '\003' 0 '\000'
7 '\a' 24 '\030' 0 '\000' 10 '\n'
0x7fffe004ee47: 28 '\034' 0 '\000' 110 'n' 101 'e' 103 'g' 112 'p'
111 'o' 115 's'
0x7fffe004ee4f: 46 '.' -3 '\375' 105 'i' -74 '\266' -22 '\352'
22 '\026' -80 '\260' 76 'L'
0x7fffe004ee57: 2 '\002' 0 '\000' 0 '\000' 0 '\000'
Explain:
2
0
: are number of keys(elements) in json object (little endian read_offset_or_size()
) 36
0
: are number of bytes (little endian read_offset_or_size()
) same as length18
0
: is the pointer where the key is stored (same as header_size
) ("pos")3
0
: is the length of first key (above)21
0
: is the pointer where the second key is stored (use read_offset_or_size()) ("neg")3
0
: is the length of second key (above)7
: the type of value it holds (#define JSONB_TYPE_INT32 0x7
)24
0
: where starts? (length is not important it is 4bytes)?
46 '.' -3 '\375' 105 'i' -74 '\266'
But if the large is true than the value is inline (wrt mysql -> parse_scalar()
and Value Value::element(size_t pos)
)
/*
Check if this is an inlined scalar value. If so, return it.
The scalar will be inlined just after the byte that identifies the
type, so it's found on entry_offset + 1.
*/
10
: the type of value it holds (#define JSONB_TYPE_UINT64 0xA
)28
0
: where starts the value? (length is not important 8 bytes ?)
-22 '\352' 22 '\026' -80 '\260' 76 'L' 2 '\002' 0 '\000' 0 '\000' 0 '\000'
JSONB_TYPE_SMALL_ARRAY 0x2
create table simple_array_table_json_object(t json) engine=myisam; # for our patch myisam tables are used !
insert into simple_array_table_json_object values ('{"a":[1,2], "b":["x","y"]}'); # will use only json object
Row data (parse_mysql
)
(gdb) x/45c data
0x7fffe004ee36: 0 '\000' 2 '\002' 0 '\000' 44 ',' 0 '\000' 18 '\022' 0 '\000' 1 '\001'
0x7fffe004ee3e: 0 '\000' 19 '\023' 0 '\000' 1 '\001' 0 '\000' 2 '\002' 20 '\024' 0 '\000'
0x7fffe004ee46: 2 '\002' 30 '\036' 0 '\000' 97 'a' 98 'b' 2 '\002' 0 '\000' 10 '\n'
0x7fffe004ee4e: 0 '\000' 5 '\005' 1 '\001' 0 '\000' 5 '\005' 2 '\002' 0 '\000' 2 '\002'
0x7fffe004ee56: 0 '\000' 14 '\016' 0 '\000' 12 '\f' 10 '\n' 0 '\000' 12 '\f' 12 '\f'
0x7fffe004ee5e: 0 '\000' 1 '\001' 120 'x' 1 '\001' 121 'y'
Explain:
first byte is telling that we are using JSON_SMALL_OBJECT
-> t=0
Others
2
0
: are number of keys(elements) in json object (little endian read_offset_or_size()
) 44
0
: are number of bytes (little endian read_offset_or_size()
) same as length18
0
: is the pointer where the key is stored (same as header_size
) ("pos")1
0
: is the length of first key (above). it is a
19
0
: is the pointer where the second key is stored (use read_offset_or_size()) ("neg")1
0
: is the length of second key (above). it is b
2
: the type of value it holds (#define JSONB_TYPE_SMALL_ARRAY 0x2
)
--- do ovdje -<> nastavi mysql24
0
: where starts? (length is not important it is 4bytes)?
46 '.' -3 '\375' 105 'i' -74 '\266'
But if the large is true than the value is inline (wrt mysql -> parse_scalar()
and Value Value::element(size_t pos)
)
/*
Check if this is an inlined scalar value. If so, return it.
The scalar will be inlined just after the byte that identifies the
type, so it's found on entry_offset + 1.
*/
10
: the type of value it holds (#define JSONB_TYPE_UINT64 0xA
)28
0
: where starts the value? (length is not important 8 bytes ?)
-22 '\352' 22 '\026' -80 '\260' 76 'L' 2 '\002' 0 '\000' 0 '\000' 0 '\000'
Adding suppression of tests:
call mtr.add_suppression("Table './test/mysql_json' is marked as crashed and should be repaired");
Tests of interest:
./mtr --do-test=mysql_json --max-test-fail=0 --force && ./mtr --do-test=anel_ --max-test-fail=0 --force
Milestone 1 for reading the data is finished: https://github.com/MariaDB/server/pull/1211
Alter table <mysql_table_name> FORCE