Implement Heartbeat Messages

claucece commented 6 years ago

Why

As we are currently implementing the revision number 2 of the OTRv4 specification, we need to correctly implement heartbeat messages.

Reference

The current implementation seems not to scale to work with a client.

See "Data Exchange" section, second paragraph, and "Receiving a Data Message" section for reference around heartbeat messages

For further information, check issue #24

Tasks

[x] Check the current behavior of heartbeat messages.
[x] Correctly make time configurable.
[x] Make/change the current implementation to work with a client.
[x] Correctly check the use of the 'IGNORE_UNREADABLE' as it is stated on the OTRv4 spec.
[x] Check the behaviors of the 'IGNORE_UNREADABLE' with the TLVs
[x] Correctly test the heartbeat messages and see which functions really need to be exposed to the tests (check the exposition of the otrl_base64_otr_decode function).
[x] Check how all TLVs are working with this.

Open questions

Might be worth checking the behavior of the 'IGNORE_UNREADABLE' flag with Dave Goulet or DrWhax.

olabini commented 6 years ago

What are the open questions for this issue?

claucece commented 6 years ago

@olabini The behavior of the 'IGNORE_UNREADABLE' is not correctly defined on otrv3 spec, but it is defined over libotr3. Some TLVs on libotr3 have this flag set even though it is not 'specifically' defined as so on the spec. This was a cause of discussion on the past. So, I was proposing to double check this with Dave Goulet or DrWhax.

olabini commented 6 years ago

Ah, I see. We should definitely define it, since it's part of the messages.

claucece commented 6 years ago

Apparently, the heartbeat messages test is now failing randomly and causing memory leaks on the process. This could have been something that was there from the beginning but just started showing.

There the error:

ERROR:test_api.c:1439:test_heartbeat_messages: assertion failed: (alice_client_state->heartbeat->last_msg_sent == time(0))

The mem leak:

==11143== 16 bytes in 1 blocks are possibly lost in loss record 158 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x57B405E: _gcry_xcalloc_secure (global.c:1190)
==11143==    by 0x58778F9: _gcry_mpi_resize (mpiutil.c:187)
==11143==    by 0x5875758: _gcry_mpi_scan (mpicoder.c:134)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x43640F: otrng_dh_init (dh.c:86)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 229 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x587524A: _gcry_mpi_scan (mpicoder.c:601)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x4363C7: otrng_dh_init (dh.c:80)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 230 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x587524A: _gcry_mpi_scan (mpicoder.c:601)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x4363EB: otrng_dh_init (dh.c:83)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 231 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x587524A: _gcry_mpi_scan (mpicoder.c:601)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x43640F: otrng_dh_init (dh.c:86)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 232 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x5877759: _gcry_mpi_alloc (mpiutil.c:84)
==11143==    by 0x436419: otrng_dh_init (dh.c:89)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 234 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x5877CC4: _gcry_mpi_copy (mpiutil.c:358)
==11143==    by 0x4369BA: otrng_dh_mpi_copy (dh.c:256)
==11143==    by 0x438935: otrng_key_manager_set_their_dh (key_management.c:144)
==11143==    by 0x43E864: receive_auth_r (otrng.c:1513)
==11143==    by 0x43F8C1: receive_decoded_message (otrng.c:1913)
==11143==    by 0x43F999: receive_encoded_message (otrng.c:1946)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 235 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x5875363: _gcry_mpi_scan (mpicoder.c:519)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x436646: otrng_dh_keypair_generate_from_shared_secret (dh.c:147)
==11143==    by 0x438AEE: generate_first_ephemeral_keys (key_management.c:183)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143==    by 0x43D4C9: double_ratcheting_init (otrng.c:1083)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 236 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x5877759: _gcry_mpi_alloc (mpiutil.c:84)
==11143==    by 0x436678: otrng_dh_keypair_generate_from_shared_secret (dh.c:154)
==11143==    by 0x438AEE: generate_first_ephemeral_keys (key_management.c:183)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143==    by 0x43D4C9: double_ratcheting_init (otrng.c:1083)
==11143==    by 0x43E92E: receive_auth_r (otrng.c:1530)
==11143==    by 0x43F8C1: receive_decoded_message (otrng.c:1913)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 237 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x5877759: _gcry_mpi_alloc (mpiutil.c:84)
==11143==    by 0x4366AC: otrng_dh_keypair_generate_from_shared_secret (dh.c:157)
==11143==    by 0x438B97: generate_first_ephemeral_keys (key_management.c:197)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143==    by 0x43D4C9: double_ratcheting_init (otrng.c:1083)
==11143==    by 0x43EBB6: receive_auth_i (otrng.c:1580)
==11143==    by 0x43F8E0: receive_decoded_message (otrng.c:1922)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 238 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x58776F9: _gcry_mpi_alloc_secure (mpiutil.c:105)
==11143==    by 0x5875363: _gcry_mpi_scan (mpicoder.c:519)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x436553: otrng_dh_keypair_generate (dh.c:124)
==11143==    by 0x438A1A: otrng_key_manager_generate_ephemeral_keys (key_management.c:163)
==11143==    by 0x439282: rotate_keys (key_management.c:401)
==11143==    by 0x43983F: otrng_key_manager_derive_dh_ratchet_keys (key_management.c:528)
==11143== 
==11143== 32 bytes in 1 blocks are possibly lost in loss record 239 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B30E8: _gcry_malloc (global.c:936)
==11143==    by 0x57B3E9E: _gcry_xmalloc (global.c:1110)
==11143==    by 0x5877759: _gcry_mpi_alloc (mpiutil.c:84)
==11143==    by 0x436588: otrng_dh_keypair_generate (dh.c:132)
==11143==    by 0x438A1A: otrng_key_manager_generate_ephemeral_keys (key_management.c:163)
==11143==    by 0x439282: rotate_keys (key_management.c:401)
==11143==    by 0x43983F: otrng_key_manager_derive_dh_ratchet_keys (key_management.c:528)
==11143==    by 0x43FE9A: send_data_message (otrng.c:2081)
==11143==    by 0x4403F3: otrng_prepare_to_send_data_message (otrng.c:2205)
==11143== 
==11143== 88 bytes in 1 blocks are possibly lost in loss record 274 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x58776BA: _gcry_mpi_alloc_limb_space (mpiutil.c:123)
==11143==    by 0x587770E: _gcry_mpi_alloc_secure (mpiutil.c:106)
==11143==    by 0x5875363: _gcry_mpi_scan (mpicoder.c:519)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x436646: otrng_dh_keypair_generate_from_shared_secret (dh.c:147)
==11143==    by 0x438AEE: generate_first_ephemeral_keys (key_management.c:183)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143== 
==11143== 88 bytes in 1 blocks are possibly lost in loss record 275 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x58776BA: _gcry_mpi_alloc_limb_space (mpiutil.c:123)
==11143==    by 0x587770E: _gcry_mpi_alloc_secure (mpiutil.c:106)
==11143==    by 0x5875363: _gcry_mpi_scan (mpicoder.c:519)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x436553: otrng_dh_keypair_generate (dh.c:124)
==11143==    by 0x438A1A: otrng_key_manager_generate_ephemeral_keys (key_management.c:163)
==11143==    by 0x439282: rotate_keys (key_management.c:401)
==11143== 
==11143== 392 bytes in 1 blocks are possibly lost in loss record 291 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x57B405E: _gcry_xcalloc_secure (global.c:1190)
==11143==    by 0x58778F9: _gcry_mpi_resize (mpiutil.c:187)
==11143==    by 0x5875758: _gcry_mpi_scan (mpicoder.c:134)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x4363C7: otrng_dh_init (dh.c:80)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 392 bytes in 1 blocks are possibly lost in loss record 292 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x57B405E: _gcry_xcalloc_secure (global.c:1190)
==11143==    by 0x58778F9: _gcry_mpi_resize (mpiutil.c:187)
==11143==    by 0x5875758: _gcry_mpi_scan (mpicoder.c:134)
==11143==    by 0x57AF8D8: gcry_mpi_scan (visibility.c:357)
==11143==    by 0x4363EB: otrng_dh_init (dh.c:83)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 392 bytes in 1 blocks are possibly lost in loss record 293 of 336
==11143==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595D5B: otrl_mem_malloc (mem.c:60)
==11143==    by 0x57B276D: do_malloc (global.c:912)
==11143==    by 0x57B2841: _gcry_malloc_secure_core (global.c:946)
==11143==    by 0x57B3F86: _gcry_xmalloc_secure (global.c:1145)
==11143==    by 0x58776BA: _gcry_mpi_alloc_limb_space (mpiutil.c:123)
==11143==    by 0x587770E: _gcry_mpi_alloc_secure (mpiutil.c:106)
==11143==    by 0x5877CC4: _gcry_mpi_copy (mpiutil.c:358)
==11143==    by 0x4369BA: otrng_dh_mpi_copy (dh.c:256)
==11143==    by 0x438935: otrng_key_manager_set_their_dh (key_management.c:144)
==11143==    by 0x43E864: receive_auth_r (otrng.c:1513)
==11143==    by 0x43F8C1: receive_decoded_message (otrng.c:1913)
==11143== 
==11143== 400 bytes in 1 blocks are possibly lost in loss record 294 of 336
==11143==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595E92: otrl_mem_realloc (mem.c:136)
==11143==    by 0x57B3298: _gcry_realloc_core (global.c:998)
==11143==    by 0x57B3F14: _gcry_xrealloc (global.c:1127)
==11143==    by 0x58778B7: _gcry_mpi_resize (mpiutil.c:179)
==11143==    by 0x5871261: _gcry_mpi_sub_ui (mpi-add.c:178)
==11143==    by 0x43643E: otrng_dh_init (dh.c:90)
==11143==    by 0x42F1CA: main (test.c:70)
==11143== 
==11143== 776 bytes in 1 blocks are possibly lost in loss record 313 of 336
==11143==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595E92: otrl_mem_realloc (mem.c:136)
==11143==    by 0x57B3298: _gcry_realloc_core (global.c:998)
==11143==    by 0x57B3F14: _gcry_xrealloc (global.c:1127)
==11143==    by 0x58778B7: _gcry_mpi_resize (mpiutil.c:179)
==11143==    by 0x5873C82: _gcry_mpi_powm (mpi-pow.c:540)
==11143==    by 0x4366A0: otrng_dh_keypair_generate_from_shared_secret (dh.c:155)
==11143==    by 0x438AEE: generate_first_ephemeral_keys (key_management.c:183)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143==    by 0x43D4C9: double_ratcheting_init (otrng.c:1083)
==11143==    by 0x43E92E: receive_auth_r (otrng.c:1530)
==11143==    by 0x43F8C1: receive_decoded_message (otrng.c:1913)
==11143== 
==11143== 776 bytes in 1 blocks are possibly lost in loss record 314 of 336
==11143==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595E92: otrl_mem_realloc (mem.c:136)
==11143==    by 0x57B3298: _gcry_realloc_core (global.c:998)
==11143==    by 0x57B3F14: _gcry_xrealloc (global.c:1127)
==11143==    by 0x58778B7: _gcry_mpi_resize (mpiutil.c:179)
==11143==    by 0x5873C82: _gcry_mpi_powm (mpi-pow.c:540)
==11143==    by 0x4366D4: otrng_dh_keypair_generate_from_shared_secret (dh.c:158)
==11143==    by 0x438B97: generate_first_ephemeral_keys (key_management.c:197)
==11143==    by 0x43909E: otrng_key_manager_ratcheting_init (key_management.c:348)
==11143==    by 0x43D4C9: double_ratcheting_init (otrng.c:1083)
==11143==    by 0x43EBB6: receive_auth_i (otrng.c:1580)
==11143==    by 0x43F8E0: receive_decoded_message (otrng.c:1922)
==11143== 
==11143== 776 bytes in 1 blocks are possibly lost in loss record 315 of 336
==11143==    at 0x4C2B7B2: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11143==    by 0x5595E92: otrl_mem_realloc (mem.c:136)
==11143==    by 0x57B3298: _gcry_realloc_core (global.c:998)
==11143==    by 0x57B3F14: _gcry_xrealloc (global.c:1127)
==11143==    by 0x58778B7: _gcry_mpi_resize (mpiutil.c:179)
==11143==    by 0x5873C82: _gcry_mpi_powm (mpi-pow.c:540)
==11143==    by 0x4365B6: otrng_dh_keypair_generate (dh.c:133)
==11143==    by 0x438A1A: otrng_key_manager_generate_ephemeral_keys (key_management.c:163)
==11143==    by 0x439282: rotate_keys (key_management.c:401)
==11143==    by 0x43983F: otrng_key_manager_derive_dh_ratchet_keys (key_management.c:528)
==11143==    by 0x43FE9A: send_data_message (otrng.c:2081)
==11143==    by 0x4403F3: otrng_prepare_to_send_data_message (otrng.c:2205)

claucece commented 6 years ago

This is just waiting for some comments, so I'm closing this now.

otrv4 / libotr-ng