search

facebook / mysql-5.6

Facebook's branch of the Oracle MySQL database. This includes MyRocks.
http://myrocks.io
Other
2.48k stars 712 forks source link

Add different compression choice #1310

Open sunshine-Chun opened 1 year ago

sunshine-Chun commented 1 year ago

Summary: when using clone to copy data in innodb instance and turn compression on. The speed is about 10 times slower than with compression off. The default compression algorithm used is zlib. Try to use other compression algorithm to tune the performance. From testing, zstd is twice faster than zlib

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

sunshine-Chun commented 1 year ago

Need to fix test.

facebook-github-bot commented 1 year ago

@sunshine-Chun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

sunshine-Chun commented 1 year ago

Looks like we have memory leakage from valgrind test. Need to debug a bit. Will update here once find anything.

  at 0xA5A6795: malloc (vg_replace_malloc.c:381)
   by 0x90917AD: ZSTD_customMalloc (allocations.h:30)
   by 0x90917AD: ZSTD_createDCtx_internal (zstd_decompress.c:296)
   by 0x90917AD: ZSTD_createDCtx (zstd_decompress.c:312)
by 0x7700BEA: zstd_uncompress(mysql_zstd_compress_context*, unsigned char*, unsigned long, unsigned long*) [clone .__uniq.287700427806220246369010965997275247218] (my_compress.cc:232)
   by 0x77009DE: my_uncompress(mysql_compress_context*, unsigned char*, unsigned long, unsigned long*) (my_compress.cc:699)
   by 0x64D4473: net_read_compressed_packet(NET*, unsigned long&) [clone .__uniq.40191296941203724010080517394248553221] (net_serv.cc:2259)
   by 0x64D4364: my_net_read(NET*) (net_serv.cc:2297)
   by 0x85CBC4D: mysql_clone_get_response(THD*, MYSQL*, bool, unsigned int, unsigned char**, unsigned long*, unsigned long*) (clone_protocol_service.cc:545)
facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

sunshine-Chun commented 1 year ago

Memory leakage fixed.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 1 year ago

@sunshine-Chun has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 1 year ago

@sunshine-Chun has updated the pull request. You must reimport the pull request before landing.