basho / bitcask

because you need another a key/value storage engine
1.29k stars 173 forks source link

Merge pull request #253 from basho/1.7 #254

Closed bsparrow435 closed 7 years ago

bsparrow435 commented 7 years ago

Use O_CREAT flag when opening files to match behavior of erlang mode

thumbot commented 7 years ago

Thanks @bsparrow435!

Settings ```yaml --- minimum_reviewers: 1 build_steps: - make clean - make test - make xref - make dialyzer merge: true org_mode: true timeout: 1790 ```
thumbot commented 7 years ago
develop-2.2 a26b6a0 :arrow_right: develop 5fd82c2 :clock1: in_progress
thumbot commented 7 years ago
develop-2.2 f61ab7f :arrow_right: develop 5fd82c2 :clock1: in_progress
thumbot commented 7 years ago
develop-2.2 f61ab7f :arrow_right: develop 7056cf9 :white_check_mark: completed
Looks good! :+1:
:white_check_mark: MERGE

> Started at: 2017-03-01 23:47 > Duration: 1 seconds. > Result: OK > Message: Merge Success: develop-2.2 f61ab7f57797666e68e7337306ed26951fb92eb3 onto target branch: develop 5fd82c2544f7c4572e19da6dd4e289e190ba400d > Exit Code: OK > :page_facing_up:

``` Updating 5fd82c2..f61ab7f Fast-forward (no commit created; -m option ignored) rebar.config | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) ``` --------------------------------------------------
:white_check_mark: MAKE_CLEAN

> Started at: 2017-03-01 23:47 > Duration: 1 seconds. > Result: OK > Message: OK > Exit Code: 0 > :page_facing_up:

``` cd /tmp/thumbs/develop.5fd82c2.develop-2.2.f61ab7f; make clean ./rebar clean ==> develop.5fd82c2.develop-2.2.f61ab7f (clean) ``` --------------------------------------------------
:white_check_mark: MAKE_TEST

> Started at: 2017-03-01 23:47 > Duration: 475 seconds. > Result: OK > Message: OK > Exit Code: 0 > :page_facing_up:

``` cd /tmp/thumbs/develop.5fd82c2.develop-2.2.f61ab7f; make test ... Snipped 680 lines ... ipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41097, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41111, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41125, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41139, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41153, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41167, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41181, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41195, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41209, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41223, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41237, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41251, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error at file /tmp/bc.test.trailingdata/1.bitcask.data offset 41265, skipping 14 bytes =ERROR REPORT==== 1-Mar-2017::23:52:36 === fold_loop: CRC error limit at file "/tmp/bc.test.trailingdata/1.bitcask.data" offset 41279 =INFO REPORT==== 1-Mar-2017::23:52:36 === Deleted 1 stale merge input files from "/tmp/bc.test.trailingdata" [0.221 s] ok bitcask: truncated_merge_test_... =ERROR REPORT==== 1-Mar-2017::23:52:37 === Hintfile '/tmp/bc.test.truncmerge/2.bitcask.hint' contains pointer 0 18 that is greater than total data size 5 =ERROR REPORT==== 1-Mar-2017::23:52:37 === Hintfile '/tmp/bc.test.truncmerge/3.bitcask.hint' invalid =ERROR REPORT==== 1-Mar-2017::23:52:37 === Hintfile '/tmp/bc.test.truncmerge/4.bitcask.hint' invalid =ERROR REPORT==== 1-Mar-2017::23:52:37 === fold_loop: CRC error at file /tmp/bc.test.truncmerge/5.bitcask.data offset 0, skipping 20 bytes [1.215 s] ok bitcask: efile_error_test...[0.041 s] ok bitcask: freeze_close_reopen_test_...[0.733 s] ok bitcask: fold_itercount_test_...[0.203 s] ok bitcask: fold_lockstep_test_...[12.257 s] ok bitcask:3377: no_tombstones_after_reopen_test_...[0.415 s] ok bitcask:3380: zap_hints_no_tombstones_after_reopen_test_...[0.370 s] ok bitcask: update_tstamp_stats_test_...[0.076 s] ok bitcask: scan_err_test_...[0.048 s] ok bitcask: no_crash_on_key_transform_test... =ERROR REPORT==== 1-Mar-2017::23:52:52 === Error converting key <<"k">>: {<<"k">>,naughty_key_transform_failure} =ERROR REPORT==== 1-Mar-2017::23:52:52 === Error converting key <<"k2">>: {<<"k2">>,naughty_key_transform_failure} =ERROR REPORT==== 1-Mar-2017::23:52:52 === Error converting key <<"k3">>: {<<"k3">>,naughty_key_transform_failure} =ERROR REPORT==== 1-Mar-2017::23:52:52 === Invalid key on load <<"k">>: naughty_key_transform_failure =ERROR REPORT==== 1-Mar-2017::23:52:52 === Invalid key on load <<"k2">>: naughty_key_transform_failure =ERROR REPORT==== 1-Mar-2017::23:52:52 === Invalid key on load <<"k3">>: naughty_key_transform_failure[0.083 s] ok bitcask: total_byte_stats_test_...[0.174 s] ok bitcask: merge_batch_test_...[0.739 s] ok bitcask: merge_expired_test_... =INFO REPORT==== 1-Mar-2017::23:52:53 === All keys expired in: "/tmp/bc.merge.expired.files/4.bitcask.data" scheduling file for deletion =INFO REPORT==== 1-Mar-2017::23:52:53 === All keys expired in: "/tmp/bc.merge.expired.files/3.bitcask.data" scheduling file for deletion =INFO REPORT==== 1-Mar-2017::23:52:53 === All keys expired in: "/tmp/bc.merge.expired.files/2.bitcask.data" scheduling file for deletion =INFO REPORT==== 1-Mar-2017::23:52:53 === All keys expired in: "/tmp/bc.merge.expired.files/1.bitcask.data" scheduling file for deletion [0.300 s] ok bitcask: max_merge_size_test_...[0.548 s] ok bitcask: legacy_tombstones_test_...[0.579 s] ok bitcask: update_tombstones_test...[0.133 s] ok [done in 37.302 s] module 'bitcask_merge_delete' bitcask_merge_delete: multiple_merges_during_fold_test_... =ERROR REPORT==== 1-Mar-2017::23:52:54 === Hintfile '/tmp/bc.multiple-merges-fold/111.bitcask.hint' invalid [1.474 s] ok bitcask_merge_delete:209: regression_gh82_test_... =INFO REPORT==== 1-Mar-2017::23:53:31 === Merged {["/tmp/bc.regression_gh82/2.bitcask.data"],[]} in 9.702009 seconds. [40.624 s] ok bitcask_merge_delete:245: change_open_regression_test_... =INFO REPORT==== 1-Mar-2017::23:53:37 === Deleted 4 stale merge input files from "/tmp/bitcask.qc" =INFO REPORT==== 1-Mar-2017::23:53:37 === Deleted 3 stale merge input files from "/tmp/bitcask.qc" [0.491 s] ok bitcask_merge_delete:310: new_20131217_a_test_... =INFO REPORT==== 1-Mar-2017::23:53:38 === Merged {["1488-441217-266075/1.bitcask.data"],[]} in 0.051349 seconds. [2.543 s] ok bitcask_merge_delete:358: new_20131217_c_test_... =INFO REPORT==== 1-Mar-2017::23:53:41 === Deleted 1 stale merge input files from "1488-441219-812092" [1.431 s] ok bitcask_merge_delete:439: new_20131217_e_test_... =INFO REPORT==== 1-Mar-2017::23:53:41 === Deleted 4 stale merge input files from "1488-441221-246079" [0.317 s] ok bitcask_merge_delete: merge_delete_race_pr156_regression_test_... =INFO REPORT==== 1-Mar-2017::23:53:41 === Deleted 1 stale merge input files from "/tmp/bc.merge_delete_race_pr156_regression" [2.120 s] ok [done in 49.021 s] module 'event_logger' module 'bitcask_bump' module 'bitcask_pulse' module 'bitcask_sup' bitcask_qc_expiry: prop_expiry_test_ (module 'bitcask_qc_expiry').......................................................................................................(x10).............(x1)........ Time limit reached: 30.0 seconds. OK, passed 238 tests [30.194 s] ok module 'bitcask_file' module 'bitcask_fileops' module 'handle_errors' bitcask_io: truncate_test_ (module 'bitcask_io')...[0.002 s] ok module 'bitcask_app' bitcask_timeshift: timeshift_test_ (module 'bitcask_timeshift')...[0.115 s] ok module 'token' bitcask_merge_worker: prop_in_window_test_ (module 'bitcask_merge_worker')...[0.004 s] ok module 'generic_qc_fsm' module 'bitcask_qc_fsm' bitcask_qc_fsm:129: qc_test_...................................................Corrupting from 63 to <<"�">> at 125 size 135 ...............................Corrupting from 0 to <<"�">> at 30 size 45 Corrupting from 0 to <<"�">> at 27 size 45 .....................(x10)..Corrupting from 255 to <<0>> at 88 size 94 .Corrupting from 0 to <<"�">> at 83 size 101 Corrupting from 22 to <<"�">> at 60 size 103 Corrupting from 0 to <<"�">> at 176 size 189 .Corrupting from 186 to <<"E">> at 164 size 193 ..Corrupting from 255 to <<0>> at 51 size 56 ..Corrupting from 255 to <<0>> at 52 size 56 ......Corrupting from 255 to <<0>> at 77 size 78 .......Corrupting from 255 to <<0>> at 152 size 155 ....(x1)......... Time limit reached: 45.0 seconds. OK, passed 359 tests 20.28% {opened,{bitcask,merge,1}} 19.21% {opened,{bitcask,get,2}} 19.10% {opened,{bitcask,put,3}} 18.86% {opened,{bitcask,delete,2}} 6.91% {closed,{bitcask_qc_fsm,create_stale_lock,0}} 6.70% {closed,{bitcask,open,2}} 5.62% {init,{bitcask_qc_fsm,set_keys,1}} 1.98% {opened,{bitcask,close,1}} 0.80% {closed,{bitcask_qc_fsm,corrupt_hint,2}} 0.55% {closed,{bitcask_qc_fsm,truncate_hint,2}} [45.323 s] ok [done in 45.326 s] module 'bitcask_nifs' bitcask_nifs: keydir_basic_test_...ok bitcask_nifs: keydir_itr_anon_test_...ok bitcask_nifs: keydir_itr_named_test_...ok bitcask_nifs: keydir_copy_test_...ok bitcask_nifs: keydir_named_test_...ok bitcask_nifs: keydir_named_not_ready_test_...ok bitcask_nifs: keydir_itr_while_itr_error_test_...ok bitcask_nifs: keydir_double_itr_test_...ok bitcask_nifs: keydir_next_notstarted_error_test_...ok bitcask_nifs: keydir_del_while_pending_test_...ok bitcask_nifs: keydir_create_del_while_pending_test_...ok bitcask_nifs: keydir_del_put_while_pending_test_...ok bitcask_nifs: keydir_multi_put_during_itr_test_...ok bitcask_nifs: keydir_itr_out_of_date_test_...ok bitcask_nifs: keydir_itr_many_pending_test_...ok bitcask_nifs: keydir_wait_pending_test_...[1.202 s] ok bitcask_nifs: keydir_get_put_test_....................................................................................................... OK, passed 100 tests [0.008 s] ok [done in 1.261 s] module 'bitcask_schema_tests' bitcask_schema_tests: basic_schema_test_...23:55:01.081 [info] Application lager started on node nonode@nohost [0.184 s] ok bitcask_schema_tests: merge_window_test_...[0.027 s] ok bitcask_schema_tests: override_schema_test_...[0.028 s] ok bitcask_schema_tests: multi_backend_test_...[0.076 s] ok [done in 0.327 s] ======================================================= All 95 tests passed. 23:55:01.295 [error] Supervisor gr_counter_sup had child gr_lager_default_tracer_counters started with gr_counter:start_link(gr_lager_default_tracer_counters) at <0.11143.0> exit with reason killed in context child_terminated 23:55:01.295 [error] Supervisor gr_param_sup had child gr_lager_default_tracer_params started with gr_param:start_link(gr_lager_default_tracer_params) at <0.11142.0> exit with reason killed in context child_terminated 23:55:01.296 [info] Application goldrush exited with reason: stopped ``` --------------------------------------------------
:white_check_mark: MAKE_XREF

> Started at: 2017-03-01 23:55 > Duration: 5 seconds. > Result: OK > Message: OK > Exit Code: 0 > :page_facing_up:

``` cd /tmp/thumbs/develop.5fd82c2.develop-2.2.f61ab7f; make xref ./rebar get-deps ==> meck (get-deps) ==> getopt (get-deps) ==> goldrush (get-deps) ==> lager (get-deps) ==> neotoma (get-deps) ==> cuttlefish (get-deps) ==> develop.5fd82c2.develop-2.2.f61ab7f (get-deps) ./rebar compile ==> meck (compile) ==> getopt (compile) ==> goldrush (compile) ==> lager (compile) ==> neotoma (compile) ==> cuttlefish (compile) Compiled src/cuttlefish_duration_parse.peg Compiled src/conf_parse.peg Compiled src/cuttlefish_duration_parse.erl Compiled src/conf_parse.erl ==> getopt (escriptize) ==> goldrush (escriptize) ==> lager (escriptize) ==> neotoma (escriptize) ==> cuttlefish (escriptize) ==> develop.5fd82c2.develop-2.2.f61ab7f (compile) ./rebar xref skip_deps=true ==> develop.5fd82c2.develop-2.2.f61ab7f (xref) ``` --------------------------------------------------
:white_check_mark: MAKE_DIALYZER

> Started at: 2017-03-01 23:55 > Duration: 26 seconds. > Result: OK > Message: OK > Exit Code: 0 > :page_facing_up:

``` cd /tmp/thumbs/develop.5fd82c2.develop-2.2.f61ab7f; make dialyzer ./rebar get-deps ==> meck (get-deps) ==> getopt (get-deps) ==> goldrush (get-deps) ==> lager (get-deps) ==> neotoma (get-deps) ==> cuttlefish (get-deps) ==> develop.5fd82c2.develop-2.2.f61ab7f (get-deps) ./rebar compile ==> meck (compile) ==> getopt (compile) ==> goldrush (compile) ==> lager (compile) ==> neotoma (compile) ==> cuttlefish (compile) Compiled src/cuttlefish_duration_parse.peg Compiled src/conf_parse.peg Compiled src/cuttlefish_duration_parse.erl Compiled src/conf_parse.erl ==> getopt (escriptize) ==> goldrush (escriptize) ==> lager (escriptize) ==> neotoma (escriptize) ==> cuttlefish (escriptize) ==> develop.5fd82c2.develop-2.2.f61ab7f (compile) Checking whether the PLT /home/ubuntu/.combo_dialyzer_plt is up-to-date... yes done (passed successfully) yes done (passed successfully) Compiling some key modules to native code... done in 0m8.59s Creating PLT .local_dialyzer_plt ... Unknown functions: alarm_handler:set_alarm/1 application:get_application/1 application:get_env/2 application:load/1 application:set_env/3 application:start/1 application:start/2 application:stop/1 beam_lib:chunks/2 calendar:datetime_to_gregorian_seconds/1 calendar:day_of_the_week/1 calendar:gregorian_seconds_to_datetime/1 calendar:last_day_of_the_month/2 calendar:local_time/0 calendar:local_time_to_universal_time_dst/1 calendar:now_to_local_time/1 code:delete/1 code:ensure_loaded/1 code:get_object_code/1 code:is_sticky/1 code:load_binary/3 code:priv_dir/1 code:purge/1 code:soft_purge/1 code:stick_mod/1 code:unstick_mod/1 compile:forms/2 cover:compile_beam/1 cover:compile_beam/2 cover:compile_module/2 cover:export/2 cover:get_term/1 cover:import/1 cover:is_compiled/1 cover:module_info/1 cover:write/2 ct:pal/1 dict:erase/2 dict:fetch/2 dict:fold/3 dict:from_list/1 dict:is_key/2 dict:new/0 dict:store/3 erl_comment_scan:string/1 erl_eval:exprs/2 erl_internal:bif/3 erl_internal:op_type/2 erl_parse:abstract/1 erl_parse:parse_exprs/1 erl_posix_msg:message/1 erl_scan:string/1 erl_syntax:abstract/1 erl_syntax:application/2 erl_syntax:application/3 erl_syntax:arity_qualifier/2 erl_syntax:atom/1 erl_syntax:attribute/2 erl_syntax:case_expr/2 erl_syntax:catch_expr/1 erl_syntax:clause/3 erl_syntax:function/2 erl_syntax:integer/1 erl_syntax:list/1 erl_syntax:match_expr/2 erl_syntax:revert/1 erl_syntax:tuple/1 erl_syntax:underscore/0 erl_syntax:variable/1 erl_syntax_lib:analyze_forms/1 erlang:atom_to_list/1 erlang:binary_to_atom/2 erlang:binary_to_list/1 erlang:binary_to_list/3 erlang:bitstring_to_list/1 erlang:erase/1 erlang:exit/2 erlang:float_to_list/1 erlang:float_to_list/2 erlang:fun_info/2 erlang:fun_to_list/1 erlang:get/1 erlang:get_module_info/1 erlang:get_module_info/2 erlang:get_stacktrace/0 erlang:group_leader/0 erlang:integer_to_list/1 erlang:iolist_size/1 erlang:iolist_to_binary/1 erlang:is_builtin/3 erlang:is_map/1 erlang:is_process_alive/1 erlang:link/1 erlang:list_to_atom/1 erlang:list_to_binary/1 erlang:list_to_existing_atom/1 erlang:list_to_float/1 erlang:list_to_integer/1 erlang:list_to_tuple/1 erlang:localtime/0 erlang:make_fun/3 erlang:map_size/1 erlang:max/2 erlang:min/2 erlang:monitor/2 erlang:pid_to_list/1 erlang:port_to_list/1 erlang:process_flag/2 erlang:process_info/2 erlang:put/2 erlang:raise/3 erlang:ref_to_list/1 erlang:registered/0 erlang:send_after/3 erlang:split_binary/2 erlang:unlink/1 erlang:whereis/1 error_logger:add_report_handler/1 error_logger:delete_report_handler/1 error_logger:error_msg/2 error_logger:warning_msg/1 escript:script_name/0 ets:delete/1 ets:delete_all_objects/1 ets:give_away/3 ets:info/1 ets:info/2 ets:insert/2 ets:insert_new/2 ets:lookup/2 ets:lookup_element/3 ets:match_spec_compile/1 ets:match_spec_run/2 ets:new/2 ets:setopts/2 ets:tab2list/1 ets:update_counter/3 eunit:test/1 file:close/1 file:consult/1 file:datasync/1 file:delete/1 file:format_error/1 file:get_cwd/0 file:open/2 file:read_file/1 file:read_file_info/1 file:rename/2 file:write/2 file:write_file/2 filelib:ensure_dir/1 filelib:is_file/1 filelib:is_regular/1 filelib:wildcard/1 filelib:wildcard/2 filename:absname/1 filename:basename/2 filename:dirname/1 filename:extension/1 filename:join/1 filename:join/2 gen_event:add_sup_handler/3 gen_event:call/3 gen_event:call/4 gen_event:delete_handler/3 gen_event:notify/2 gen_event:sync_notify/2 gen_event:which_handlers/1 gen_server:call/2 gen_server:call/3 gen_server:cast/2 gen_server:reply/2 gen_server:start/3 gen_server:start/4 gen_server:start_link/3 gen_server:start_link/4 hamcrest:assert_that/2 hamcrest:is_matcher/1 inet:parse_address/1 init:get_argument/1 init:stop/1 io:columns/0 io:format/1 io:format/2 io:format/3 io:put_chars/1 io:put_chars/2 io_lib:deep_char_list/1 io_lib:format/2 io_lib:fwrite/2 io_lib:printable_list/1 io_lib:write/2 io_lib_format:fwrite_g/1 lager_default_tracer:info/1 lists:duplicate/2 lists:flatlength/1 lists:keystore/4 lists:prefix/2 lists:seq/2 lists:sublist/2 lists:sum/1 maps:to_list/1 mustache:render/2 orddict:append_list/3 orddict:new/0 orddict:update_counter/3 ordsets:add_element/2 ordsets:from_list/1 ordsets:intersection/2 ordsets:is_element/2 ordsets:new/0 ordsets:union/2 os:getenv/1 os:timestamp/0 os:type/0 proplists:delete/2 proplists:get_all_values/2 proplists:get_bool/2 proplists:get_keys/1 proplists:get_value/2 proplists:get_value/3 proplists:is_defined/2 proplists:lookup/2 re:compile/2 re:replace/4 re:run/3 rebar_config:get_local/3 rebar_mustache:render/2 rebar_rel_utils:get_target_dir/2 rebar_rel_utils:is_rel_dir/0 rebar_rel_utils:load_config/2 sets:add_element/2 sets:from_list/1 sets:is_element/2 sets:new/0 sets:size/1 sets:to_list/1 string:centre/2 string:chr/2 string:cspan/2 string:join/2 string:left/2 string:left/3 string:rchr/2 string:right/2 string:str/2 string:strip/1 string:sub_string/2 string:sub_string/3 string:substr/2 string:substr/3 string:to_lower/1 string:tokens/2 supervisor:delete_child/2 supervisor:start_child/2 supervisor:start_link/3 supervisor:terminate_child/2 timer:now_diff/2 timer:sleep/1 unicode:characters_to_binary/1 unicode:characters_to_binary/3 unicode:characters_to_list/1 unicode:characters_to_list/2 Unknown types: erl_syntax:syntaxTree/0 erlang:timestamp/0 ets:comp_match_spec/0 ets:tab/0 ets:tid/0 file:date_time/0 file:filename/0 file:io_device/0 file:name/0 file:name_all/0 file:posix/0 hamcrest:matchspec/0 proplists:property/0 supervisor:child_spec/0 unicode:charlist/0 done in 0m1.54s done (passed successfully) ==> develop.5fd82c2.develop-2.2.f61ab7f (dialyzer) Checking whether the PLT /home/ubuntu/.combo_dialyzer_plt is up-to-date... yes Checking whether the PLT .local_dialyzer_plt is up-to-date... yes Proceeding with analysis... done in 0m3.49s done (passed successfully) ``` --------------------------------------------------
:white_large_square: 0 of 1 Code reviews from organization basho