Closed benjoe87 closed 9 months ago
@martinsumner how could we proceed with this. We still using leveldb. :/
I wanted to have a test to make sure if we do fix this issue, we're not going to regress the fix.
This updated riak_test upgrade test detects both the problem of compression not working, as well as the previously undetected issue of snappy updates being uncompatable.
However, when I test this on a non-update (e.g. previous and current both develop-3.0
) but with the eleveldb branch switched to benjoe87:fix-snappy-support
- the test still failed. The size of the backend store didn't reduce when comparing snappy with compression disabled:
================ verify_basic_upgrade failure stack trace =====================
{{assert,[{module,verify_basic_upgrade},
{line,115},
{expression,"NativeSz < floor ( ( PlainTextSz * 0.8 ) )"},
{expected,true},
{value,false}]},
[{verify_basic_upgrade,'-confirm/0-fun-12-',2,
[{file,"/Users/martinsumner/riak_test/tests/verify_basic_upgrade.erl"},
{line,115}]},
{verify_basic_upgrade,confirm,0,
[{file,"/Users/martinsumner/riak_test/tests/verify_basic_upgrade.erl"},
{line,115}]},
{riak_test_runner,return_to_exit,3,
[{file,"/Users/martinsumner/riak_test/src/riak_test_runner.erl"},
{line,159}]}]}
===============================================================================
So I can't seem to get this PR to resolve the problem. Is there other evidence this works?
@tburghart is there a fix for this in your version?
Previously when I tested this fix, using the updated verify_basic_upgrade - it FAILED. But this was testing on my OSX laptop.
I tried this fix today on Ubuntu - and it worked.
current
3.0.16, previous
3.0.16 -> verify_basic_upgrade
FAIL (as size of leveldb store not reduced by enabling compression).current
3.0.16 + ELEVELDB PR276, previous
3.0.16 + ELEVELDB PR276 -> verify_basic_upgrade
PASS (compression now enabled)current
3.0.16 + ELEVELDB PR276, previous
3.0.11 -> verify_basic_upgrade
PASS (1.1.9 and 1.04 are in fact backwards/forwards compatible)This would appear to confirm this PR is good on Ubuntu. The original issue is caused by snappy not running and not due to incompatibility between 1.0.4 and 1.1.9.
As OSX is not supported for running production Riak clusters, I think the PR should be accepted so that there can be a 3.0.17 release that will allow multi_backend users (or those who have over-ridden the LZ4 default) to safely upgrade from 3.0.11.
@nsaadouni - is this something you can look at?
The OSX issues can be resolved by setting "std=c++11" in the CFLAGS and COMMON_FLAGS in build_detect_platform:
https://github.com/nhs-riak/leveldb/commit/458bfb4e41d2822b9f2e90b393ec786689249c33
Also in the rebar.config.script so that it is used when compiling eleveldb:
Sorry for being late to the discussion.
The way we've "fixed" it in our version is to completely rewrite eleveldb/c_src/Makefile
to:
leveldb/build_detect_platform
This has given us not only a clean build across platforms, but also a tree that recognizes it's already been built so it doesn't rebuild 8 times when making devrels 🤨
Without exporting the compiling flags the build_detect_platform is not able to detect the snappy support and does not set the SNAPPY C macro.
Fix issue https://github.com/basho/eleveldb/issues/275