DC_EVENT_HTTP_GET issues

[ralphtheninja]

There are some edge cases that doesn't work. Need to figure out more information.

works

• foo@librechat.net + valid password

doesn't work

<@hpk> the 1.0.8 console log says: "can't read file" for https://hq5.merlinux.eu/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress=hpk2%40hq5.merlinux.eu
<@hpk> but the url resolves in a browser 
<@hpk> does it work for you (successfully receiving the autoconfig) when you specify test1@hq5.merlinux.eu (password doesn't matter)? 

[ralphtheninja]

@hpk42 If you have some details please add them to this issue. What works, what doesn't work etc.

[hpk42]

v1.0.8 on ubuntu 16.04: when i type as login: "hpk2@hq5.merlinux.eu" (you can use any user and password, this is about domain autoconfig) i get in the console log:

2100 "https://hq5.merlinux.eu/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress=hpk2%40hq5.merlinux.eu" 0 100 0 "Can't read file."

but if you copy+paste that url it resolves.

[ralphtheninja]

@r10s Is it the same thread that causes the DC_EVENT_HTTP_GET events? My first thought was that there's something wrong with the thread synchronization primitives, e.g. the mutex and the cond.

cc @Jikstra

[r10s]

@ralphtheninja can you say if the complete file content is there when dcn_set_http_get_response() is entered? is it there in js-land? if so, is it there after NAPI_ARGV_UTF8_MALLOC()?

just to exclude some paths to search for errors :)

[ralphtheninja]

Added a temporary branch to test this https://github.com/deltachat/deltachat-node/tree/dc-event-http-get-test

This seems to fix that the c side waits properly for the http response but instead I get a segfault in ssl3_cleanup_key_block().

To reproduce:

$ cd deltachat-node/test
$ rm -rf db.sqlite* && node dc-event-http-get.js

$ gdb node core
(gdb) bt
#0  0x00000000012f4ecd in ssl3_cleanup_key_block ()
#1  0x00000000012f5a31 in ssl3_clear ()
#2  0x0000000001320a91 in tls1_clear ()
#3  0x0000000001300b4f in SSL_new ()
#4  0x00007f8fd1d44577 in ssl_data_new_full (fd=17, timeout=10, 
    method=0x7f8fd228eac0 <SSLv23_client_method_data.15386>, callback=0x0, cb_data=0x0)
    at ../libs/libetpan/src/data-types/mailstream_ssl.c:457
#5  0x00007f8fd1d44763 in ssl_data_new (fd=17, timeout=10, callback=0x0, cb_data=0x0)
    at ../libs/libetpan/src/data-types/mailstream_ssl.c:526
#6  0x00007f8fd1d4481e in mailstream_low_ssl_open_full (fd=17, starttls=0, timeout=10, callback=0x0, cb_data=0x0)
    at ../libs/libetpan/src/data-types/mailstream_ssl.c:723
#7  0x00007f8fd1d452d9 in mailstream_low_ssl_open_with_callback_timeout (fd=17, timeout=10, callback=0x0, data=0x0)
    at ../libs/libetpan/src/data-types/mailstream_ssl.c:1243
#8  0x00007f8fd1d450eb in mailstream_ssl_open_with_callback_timeout (fd=17, timeout=10, callback=0x0, data=0x0)
    at ../libs/libetpan/src/data-types/mailstream_ssl.c:1133
#9  0x00007f8fd1d6137f in mailimap_ssl_connect_voip_with_callback (f=0x7f8fb00017d0, 
    server=0x7f8fb0001770 "hq5.merlinux.eu", port=993, voip_enabled=0, callback=0x0, data=0x0)
    at ../libs/libetpan/src/low-level/imap/mailimap_ssl.c:96
#10 0x00007f8fd1d61421 in mailimap_ssl_connect_voip (f=0x7f8fb00017d0, server=0x7f8fb0001770 "hq5.merlinux.eu", 
    port=993, voip_enabled=0) at ../libs/libetpan/src/low-level/imap/mailimap_ssl.c:116
#11 0x00007f8fd1d613e3 in mailimap_ssl_connect (f=0x7f8fb00017d0, server=0x7f8fb0001770 "hq5.merlinux.eu", 
    port=993) at ../libs/libetpan/src/low-level/imap/mailimap_ssl.c:111
#12 0x00007f8fd1d3efd9 in setup_handle_if_needed (imap=0x44c2820) at ../src/dc_imap.c:1123
#13 0x00007f8fd1d3f40a in dc_imap_connect (imap=0x44c2820, lp=0x7f8fb0001060) at ../src/dc_imap.c:1250
#14 0x00007f8fd1d190fa in dc_job_do_DC_JOB_CONFIGURE_IMAP (context=0x44d02d0, job=0x7f8fc37fde70)
    at ../src/dc_configure.c:666
#15 0x00007f8fd1d139c4 in dc_job_perform (context=0x44d02d0, thread=100) at ../src/dc_job.c:634
#16 0x00007f8fd1d13c85 in dc_perform_imap_jobs (context=0x44d02d0) at ../src/dc_job.c:717
#17 0x00007f8fd1cf8b03 in imap_thread_func (arg=0x447c610) at ../src/module.c:168
#18 0x00007f8fd46b56db in start_thread (arg=0x7f8fc37fe700) at pthread_create.c:463
#19 0x00007f8fd43de88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)                                                                                                              

[r10s]

@ralphtheninja good catch - i forgot to set dc_event_http_done back to 0 :/

however, we should not do this before the while loop but after it - where we also reset dcn_context->dc_event_http_response = NULL;.

• this guarantees dc_event_http_response and dc_event_http_done are always in sync
• doing it before can cause problems when js does its job after eventqueue_push() but before we wait for the condition.

i would try the following code:

        pthread_mutex_lock(&dcn_context->dc_event_http_mutex);
          while (!dcn_context->dc_event_http_done) { // while() is to protect against spuriously wakeups
            pthread_cond_wait(&context->dc_event_http_cond, &context->dc_event_http_mutex); // unlock -> wait -> lock
          }
          http_ret = (uintptr_t)dcn_context->dc_event_http_response;
          dcn_context->dc_event_http_response = NULL;
          dcn_context->dc_event_http_done = 0;
        pthread_mutex_unlock(&dcn_context->dc_event_http_mutex);

[ralphtheninja]

Nevermind the SEGFAULT stuff, it's unrelated.

Anyway, managed to get autoconfig to work on hq5.merlinux.eu:

The first url is expected to fail, since invalid subdomain. Second url works.