search

initc3 / HoneyBadgerMPC

Robust MPC-based confidentiality layer for blockchains
GNU General Public License v3.0
128 stars 64 forks source link

Fix asynchromix example #424

Closed sbellem closed 4 years ago

sbellem commented 4 years ago

UPDATE: ready for review and merging

Making PR to ease collaboration on fixing problems. "Open" problems will be posted in the comments below or at #425.

sbellem commented 4 years ago

Current problem: not enough one_minus_ones elements. UPDATE: fixed in e3ee1ab469dfddabe0a85e8d7307429bd14e0154

LOGS (click to unfold or fold) ```shell 2020-02-27 05:12:55,980:[asynchromix.py:372]:[INFO]: [3] Running permutation network [193/9595] 2020-02-27 05:12:55,982:[misc.py:16]:[CRITICAL]: Exception: exception=AssertionError("Expected 1 elements of one_minus_ones, but found only 0\nkey is: (0, 4, 1)\ncount is: defaultdict(, {(0, 4, 1): 0})\n")> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 394, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 376, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in batch_switch sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 674, in get_one_minus_ones return self._one_minus_ones.get_value(context) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 101, in get_value to_return, used = self._get_value(context, key, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 433, in _get_value f"Expected " AssertionError: Expected 1 elements of one_minus_ones, but found only 0 key is: (0, 4, 1) count is: defaultdict(, {(0, 4, 1): 0}) ) 2020-02-27 05:12:56,035:[asynchromix.py:281]:[INFO]: [1] totalmasks: 288 inputmasks available: 256 target: 320 Initiating Randousha 32 2020-02-27 05:12:56,090:[asynchromix.py:281]:[INFO]: [2] totalmasks: 288 inputmasks available: 256 target: 320 Initiating Randousha 32 2020-02-27 05:12:56,143:[asynchromix.py:281]:[INFO]: [3] totalmasks: 288 inputmasks available: 256 target: 320 Initiating Randousha 32 2020-02-27 05:12:56,149:[misc.py:16]:[CRITICAL]: Exception: exception=AssertionError("Expected 1 elements of one_minus_ones, but found only 0\nkey is: (1, 4, 1)\ncount is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0})\n")> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 394, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 376, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in batch_switch sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 674, in get_one_minus_ones return self._one_minus_ones.get_value(context) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 101, in get_value to_return, used = self._get_value(context, key, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 433, in _get_value f"Expected " AssertionError: Expected 1 elements of one_minus_ones, but found only 0 key is: (1, 4, 1) count is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0}) ) 2020-02-27 05:12:56,150:[misc.py:16]:[CRITICAL]: Exception: exception=AssertionError("Expected 1 elements of one_minus_ones, but found only 0\nkey is: (2, 4, 1)\ncount is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0, (2, 4, 1): 0})\n")> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 394, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 376, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in batch_switch sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 674, in get_one_minus_ones return self._one_minus_ones.get_value(context) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 101, in get_value to_return, used = self._get_value(context, key, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 433, in _get_value f"Expected " AssertionError: Expected 1 elements of one_minus_ones, but found only 0 key is: (2, 4, 1) count is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0, (2, 4, 1): 0}) ) 2020-02-27 05:12:56,150:[misc.py:16]:[CRITICAL]: Exception: exception=AssertionError("Expected 1 elements of one_minus_ones, but found only 0\nkey is: (3, 4, 1)\ncount is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0, (2, 4, 1): 0, (3, 4, 1): 0})\n")> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 394, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 376, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in batch_switch sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 10, in sbits = [ctx.preproc.get_one_minus_ones(ctx).v for _ in range(n // 2)] File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 674, in get_one_minus_ones return self._one_minus_ones.get_value(context) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 101, in get_value to_return, used = self._get_value(context, key, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/preprocessing.py", line 433, in _get_value f"Expected " AssertionError: Expected 1 elements of one_minus_ones, but found only 0 key is: (3, 4, 1) count is: defaultdict(, {(0, 4, 1): 400, (1, 4, 1): 0, (2, 4, 1): 0, (3, 4, 1): 0}) ) 2020-02-27 05:12:56,273:[asynchromix.py:292]:[INFO]: [0] Randousha finished in 0.3649168014526367 ```
codecov[bot] commented 4 years ago

Codecov Report

Merging #424 into dev will decrease coverage by 0.25646%. The diff coverage is 8.69565%.

@@                 Coverage Diff                 @@
##                 dev        #424         +/-   ##
===================================================
- Coverage   77.27842%   77.02196%   -0.25646%     
===================================================
  Files             50          50                 
  Lines           5585        5601         +16     
  Branches         856         856                 
===================================================
- Hits            4316        4314          -2     
- Misses          1095        1112         +17     
- Partials         174         175          +1
sbellem commented 4 years ago

Occasional problem: batch reconstruction fails.

running make clean seems to prevent the problem ... perhaps there's some leftover files causing the problem ..

LOGS (click to unfold or fold) ```shell 2020-02-27 21:57:32,332:[asynchromix.py:402]:[INFO]: [0] MPC complete ['message:7:19', 'message:7:8', 'message:7:17', 'message:7:3', 'message:7:27', 'message:7:24', 'message:7:0', 'message:7 :23', 'message:7:9', 'message:7:10', 'message:7:12', 'message:7:29', 'message:7:26', 'message:7:31', 'message:7:25', 'message:7:16', 'message:7:20', 'message:7:22', 'message:7:6', 'message:$ :11', 'message:7:7', 'message:7:18', 'message:7:5', 'message:7:13', 'message:7:28', 'message:7:15', 'message:7:2', 'message:7:14', 'message:7:21', 'message:7:4', 'message:7:30', 'message:7:$ '] 2020-02-27 21:57:32,412:[asynchromix.py:402]:[INFO]: [1] MPC complete ['message:7:19', 'message:7:8', 'message:7:17', 'message:7:3', 'message:7:27', 'message:7:24', 'message:7:0', 'message:$ :23', 'message:7:9', 'message:7:10', 'message:7:12', 'message:7:29', 'message:7:26', 'message:7:31', 'message:7:25', 'message:7:16', 'message:7:20', 'message:7:22', 'message:7:6', 'message:$ :11', 'message:7:7', 'message:7:18', 'message:7:5', 'message:7:13', 'message:7:28', 'message:7:15', 'message:7:2', 'message:7:14', 'message:7:21', 'message:7:4', 'message:7:30', 'message:7:$ '] 2020-02-27 21:57:32,486:[asynchromix.py:402]:[INFO]: [2] MPC complete ['message:7:19', 'message:7:8', 'message:7:17', 'message:7:3', 'message:7:27', 'message:7:24', 'message:7:0', 'message:$ :23', 'message:7:9', 'message:7:10', 'message:7:12', 'message:7:29', 'message:7:26', 'message:7:31', 'message:7:25', 'message:7:16', 'message:7:20', 'message:7:22', 'message:7:6', 'message:$ :11', 'message:7:7', 'message:7:18', 'message:7:5', 'message:7:13', 'message:7:28', 'message:7:15', 'message:7:2', 'message:7:14', 'message:7:21', 'message:7:4', 'message:7:30', 'message:7:$ '] 2020-02-27 21:57:32,552:[asynchromix.py:402]:[INFO]: [3] MPC complete ['message:7:19', 'message:7:8', 'message:7:17', 'message:7:3', 'message:7:27', 'message:7:24', 'message:7:0', 'message:$ :23', 'message:7:9', 'message:7:10', 'message:7:12', 'message:7:29', 'message:7:26', 'message:7:31', 'message:7:25', 'message:7:16', 'message:7:20', 'message:7:22', 'message:7:6', 'message:$ :11', 'message:7:7', 'message:7:18', 'message:7:5', 'message:7:13', 'message:7:28', 'message:7:15', 'message:7:2', 'message:7:14', 'message:7:21', 'message:7:4', 'message:7:30', 'message:7:$ '] 2020-02-27 21:57:34,599:[asynchromix.py:70]:[INFO]: [Client] Starting Epoch 8 2020-02-27 21:57:38,210:[asynchromix.py:281]:[INFO]: [3] totalmasks: 576 inputmasks available: 294 target: 320 Initiating Randousha 32 2020-02-27 21:57:38,413:[asynchromix.py:414]:[INFO]: [2] MIX OUTPUT[7] message:7:19,message:7:8,message:7:17,message:7:3,message:7:27,message:7:24,message:7:0,message:7:23,message:7:9,messa$ e:7:10,message:7:12,message:7:29,message:7:26,message:7:31,message:7:25,message:7:16,message:7:20,message:7:22,message:7:6,message:7:11,message:7:7,message:7:18,message:7:5,message:7:13,mes$ age:7:28,message:7:15,message:7:2,message:7:14,message:7:21,message:7:4,message:7:30,message:7:1 2020-02-27 21:57:39,366:[asynchromix.py:281]:[INFO]: [0] totalmasks: 576 inputmasks available: 288 target: 320 Initiating Randousha 32 2020-02-27 21:57:39,584:[asynchromix.py:281]:[INFO]: [1] totalmasks: 576 inputmasks available: 288 target: 320 Initiating Randousha 32 2020-02-27 21:57:39,704:[asynchromix.py:281]:[INFO]: [2] totalmasks: 576 inputmasks available: 288 target: 320 Initiating Randousha 32 2020-02-27 21:57:39,969:[asynchromix.py:292]:[INFO]: [0] Randousha finished in 0.6027922630310059 2020-02-27 21:57:40,280:[asynchromix.py:292]:[INFO]: [1] Randousha finished in 0.6956119537353516 2020-02-27 21:57:40,660:[asynchromix.py:292]:[INFO]: [2] Randousha finished in 0.9556841850280762 2020-02-27 21:57:40,965:[asynchromix.py:292]:[INFO]: [3] Randousha finished in 2.7549543380737305 2020-02-27 21:57:52,341:[asynchromix.py:395]:[INFO]: [3] MPC initiated:8 ... 2020-02-27 21:58:02,827:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,828:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,828:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,828:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,829:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,829:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,829:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,829:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,830:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,830:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,830:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,830:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,830:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,831:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,831:[reed_solomon.py:323]:[CRITICAL]: Optimistic decoding failed 2020-02-27 21:58:02,831:[batch_reconstruction.py:184]:[ERROR]: [BatchReconstruct] P1 reconstruction failed! 2020-02-27 21:58:02,831:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 2) failed! 2020-02-27 21:58:02,831:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 3) failed! 2020-02-27 21:58:02,831:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 2) failed! 2020-02-27 21:58:02,832:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 3) failed! 2020-02-27 21:58:02,832:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 2) failed! 2020-02-27 21:58:02,832:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 3) failed! 2020-02-27 21:58:02,832:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 2) failed! 2020-02-27 21:58:02,832:[mpc.py:183]:[ERROR]: Batch reconstruction for share_array (id: 3) failed! ... 2020-02-27 21:58:02,892:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,892:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,893:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,893:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,896:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 401, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 383, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 15, in batch_switch ms = (await (sbits * (xs - ys)))._shares File "/usr/local/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/utils/misc.py", line 17, in print_exception_callback raise ex File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,896:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 401, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 383, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 15, in batch_switch ms = (await (sbits * (xs - ys)))._shares File "/usr/local/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/utils/misc.py", line 17, in print_exception_callback raise ex File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,897:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 401, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 383, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 15, in batch_switch ms = (await (sbits * (xs - ys)))._shares File "/usr/local/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/utils/misc.py", line 17, in print_exception_callback raise ex File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:02,897:[misc.py:16]:[CRITICAL]: Exception: exception=HoneyBadgerMPCError('Batch reconstruction failed!')> Traceback (most recent call last): File "apps/asynchromix/asynchromix.py", line 401, in _mixing_loop result = await ctx._run() File "/usr/src/HoneyBadgerMPC/honeybadgermpc/mpc.py", line 242, in _run return result.result() File "apps/asynchromix/asynchromix.py", line 383, in prog shuffled = await iterated_butterfly_network(ctx, inps, K) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 48, in iterated_butterfly_network result = await batch_switch(ctx, xs_, ys_, k) File "/usr/src/HoneyBadgerMPC/apps/asynchromix/butterfly_network.py", line 15, in batch_switch ms = (await (sbits * (xs - ys)))._shares File "/usr/local/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/utils/misc.py", line 17, in print_exception_callback raise ex File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/base.py", line 49, in __call__ return await cls._prog(context, *args, **kwargs) File "/usr/src/HoneyBadgerMPC/honeybadgermpc/progs/mixins/share_arithmetic.py", line 42, in _prog f, g = await gather(*[(j - u).open(), (k - v).open()]) honeybadgermpc.exceptions.HoneyBadgerMPCError: Batch reconstruction failed! ) 2020-02-27 21:58:14,524:[asynchromix.py:246]:[INFO]: [0] Bits finished in 12.893511056900024 2020-02-27 21:58:14,819:[asynchromix.py:246]:[INFO]: [1] Bits finished in 12.934111833572388 2020-02-27 21:58:15,109:[asynchromix.py:246]:[INFO]: [2] Bits finished in 12.970342636108398 2020-02-27 21:58:15,457:[asynchromix.py:246]:[INFO]: [3] Bits finished in 13.066404581069946 # hangs ```
sbellem commented 4 years ago

Occasional problem: VM Exception while processing transaction ... UPDATE: handled via tip given in https://github.com/initc3/HoneyBadgerMPC/pull/424#issuecomment-592964150

2020-02-28 21:14:05,042:[misc.py:16]:[CRITICAL]:                                                                                                                                                  
Exception:                                                                                                                                                                                                         
<Task finished coro=<AsynchromixServer._mixing_initiate_loop() done, defined at apps/asynchromix/asynchromix.py:423> exception=ValueError({'message': 'VM Exception while processing transaction: revert', 'code': -
32000, 'data': {'stack': 'c: VM Exception while processing transaction: revert\n    at Function.c.fromResults (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:2:162421)\n    at e.exports (/
usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:53:1405579)', 'name': 'c'}})>                                                                                                                
Traceback (most recent call last):                                                                                                                                                                                  
  File "apps/asynchromix/asynchromix.py", line 438, in _mixing_initiate_loop                                                                                                                      
    {"from": self.w3.eth.accounts[0]}                                                                                                                                                                              
  File "/opt/venv/lib/python3.7/site-packages/web3/contract.py", line 948, in transact                                                                                                                              
    **self.kwargs                                                                                                                                                                                 
  File "/opt/venv/lib/python3.7/site-packages/web3/contract.py", line 1526, in transact_with_contract_function                                                                                                     
    txn_hash = web3.eth.sendTransaction(transact_transaction)                                                                                                                                                       
  File "/opt/venv/lib/python3.7/site-packages/web3/eth.py", line 381, in sendTransaction                                                                                                                            
    get_buffered_gas_estimate(self.web3, transaction),
  File "/opt/venv/lib/python3.7/site-packages/web3/_utils/transactions.py", line 126, in get_buffered_gas_estimate
    gas_estimate = web3.eth.estimateGas(gas_estimate_transaction)
  File "/opt/venv/lib/python3.7/site-packages/web3/eth.py", line 445, in estimateGas
    params,
  File "/opt/venv/lib/python3.7/site-packages/web3/manager.py", line 153, in request_blocking
    raise ValueError(response["error"])
ValueError: {'message': 'VM Exception while processing transaction: revert', 'code': -32000, 'data': {'stack': 'c: VM Exception while processing transaction: revert\n    at Function.c.fromResults (/usr/local/lib/
node_modules/ganache-cli/build/ganache-core.node.cli.js:2:162421)\n    at e.exports (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:53:1405579)', 'name': 'c'}}
)
2020-02-28 21:14:07,055:[asynchromix.py:396]:[INFO]: [0] MPC initiated:7
amiller commented 4 years ago

We should just catch this ValueError. I think what's happening is this: We already know there is a race condition here - all N servers attempt to send "initiate_mix()", but only one of the transactions is expected to have an effect (all of them will be committed on the blockchain, but all but one will encounter "revert" and have no effect. What's changed is that now the web3 is rejecting the transaction client-side, during gas estimation, if it can tell that the transaction will always fail. So, that's fine. We should just catch this and go back around the waiting loop.

amiller commented 4 years ago

Is this complete yet? I don't see further todo's

sbellem commented 4 years ago

Is this complete yet? I don't see further todo's

yes