Open DifferentialOrange opened 5 months ago
Despite all the retries and checks, sometimes vshard cluster fails to start from config. For example, see https://github.com/tarantool/crud/actions/runs/8541312165/job/23400336786
Log sample:
router | 2024-03-26 17:43:18.344 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.446 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.549 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.551 [446474] main/122/vshard.master_search._static_router/vshard.router init.lua:1185 E> Error during master search: {"code":32,"base_type":"LuajitError","type":"LuajitError","message":"{\"reason\":\"storage is not configured\",\"code\":33,\"type\":\"ShardingError\",\"message\":\"Storage is disabled: storage is not configured\",\"name\":\"STORAGE_IS_DISABLED\"}","trace":[{"file":"./tarantool/src/lua/utils.c","line":679}]} router | 2024-03-26 17:43:18.651 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.754 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.856 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:18.959 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:19.061 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:19.162 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed router | 2024-03-26 17:43:19.164 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed select.backend:"config".engine:"memtx".test_select_safety_too_low_limit ... select.backend:"config".engine:"memtx".test_select_safety_too_low_limit ... (0.000s) error ...ithub/tarantool/crud/test/tarantool3_helpers/cluster.lua:252: crud is not bootstrapped: expected 4 running storages, got the following storage info: {"s1-master":{"status":"running","is_master":true},"s1-replica":{"status":"running","is_master":false},"s2-replica":{"status":"running","is_master":false},"s2-master":{"status":"error","message":"Connection closed","is_master":true}}
luatest_log_capture.zip
Despite all the retries and checks, sometimes vshard cluster fails to start from config. For example, see https://github.com/tarantool/crud/actions/runs/8541312165/job/23400336786
Log sample:
luatest_log_capture.zip