search

vibe-d / vibe.d

Official vibe.d development
MIT License
1.15k stars 284 forks source link

requestHTTP with headers returns empty result after updating lastest dub version 0.9.4 #2628

Open kerdemdemir opened 3 years ago

kerdemdemir commented 3 years ago

I had this call in my application which is running for a long time. 

            auto e = collectException(requestHTTP(url.dup, (scope req) {
                    req.method = method;
                    req.headers["X-MBX-APIKEY"] = keySecret.key;
                    if (params != null)
                        req.writeFormBody(params);
                }, (scope res) {
                    result = parseJsonString(res.bodyReader.readAllUTF8());
                }));

I decided to update to latest vibe from vibe 0.8.6. But now I am receiving

When I am printing the "result" it is an empty json object.

Everything works fine again if I switch version to 0.8.6 like: 

    "dependencies": {
        "eventcore": "0.9.6",
        "vibe-core": "1.9.2",
            "vibe-d": "0.8.6",
        "zmqd": "*",
    },

My operating system is :

Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal

DMD version is:

DMD64 D Compiler v2.090.1 Copyright (C) 1999-2019 by The D Language Foundation, All Rights Reserved written by Walter Bright

oracle1wr commented 2 years ago

I have the same issue. Rolled back to 0.9.2 - it works as expected.

s-ludwig commented 2 years ago

There have been two major changes to the HTTP client between those versions - fixing the handling of keep-alive connections and sending "application/json; charset=UTF-8" instead of just "application/json". Could any of those lead to this issue?

For diagnosing the error it would be helpful to also have the log messages (running with --vvvv) of the request, as well as the output of just res.bodyWriter.readAllUTF8(), without the parseJsonString.

kerdemdemir commented 2 years ago

I wanted to try removing the parseJsonString unfortunately, the code does not reach to parseJsonString.

I realize this after I remove collectException: 

Running ./dmarketcalls --vvvv
[2021-11-23 11:52:41.097 D2016A37(----) trc] create driver core
[2021-11-23 11:52:41.097 main(----) dbg] Initializing OpenSSL...
[2021-11-23 11:52:41.099 main(----) dbg] ... done.
[2021-11-23 11Lets begin 
Alert15: 
:52:41.100 main(----) dbg] Create HTTP client pool api.binance.com:443 true proxy :0
[2021-11-23 11:52:41.100 main(----) dbg] creating new HTTPClient connection, all 0 are in use
[2021-11-23 11:52:41.101 main(----) dbg]  ... 7F59CD8D5D10
[2021-11-23 11:52:41.101 main(----) dbg] Now got 1 connections
[2021-11-23 11:52:41.102 Eventcore DNS Lookup(----) dbg] Thread exit Eventcore DNS Lookup (index 1) (main=false)
object.Exception@../../.dub/packages/vibe-core-1.21.0/vibe-core/source/vibe/core/net.d(75): Failed to lookup host 'api.binance.com'.
----------------
/snap/dmd/102/bin/../import/phobos/std/exception.d:515 pure @safe void std.exception.bailOut!(Exception).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x558fd1afd5a2]
/snap/dmd/102/bin/../import/phobos/std/exception.d:436 pure @safe bool std.exception.enforce!().enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x558fd1afd51e]
../../.dub/packages/vibe-core-1.21.0/vibe-core/source/vibe/core/net.d:75 @safe vibe.core.net.NetworkAddress vibe.core.net.resolveHost(immutable(char)[], ushort, bool, core.time.Duration) [0x558fd1cc771c]
../../.dub/packages/vibe-core-1.21.0/vibe-core/source/vibe/core/net.d:32 @safe vibe.core.net.NetworkAddress vibe.core.net.resolveHost(immutable(char)[], std.socket.AddressFamily, bool, core.time.Duration) [0x558fd1cc747f]
../../.dub/packages/vibe-d-0.9.4/vibe-d/http/vibe/http/client.d:707 @safe bool vibe.http.client.HTTPClient.doRequest(scope void delegate(vibe.http.client.HTTPClientRequest), ref bool, bool, std.datetime.systime.SysTime) [0x558fd1b9299a]
../../.dub/packages/vibe-d-0.9.4/vibe-d/http/vibe/http/client.d:627 @safe bool vibe.http.client.HTTPClient.doRequestWithRetry(scope void delegate(vibe.http.client.HTTPClientRequest), bool, out bool, out std.datetime.systime.SysTime) [0x558fd1b9242f]
../../.dub/packages/vibe-d-0.9.4/vibe-d/http/vibe/http/client.d:551 @trusted void vibe.http.client.HTTPClient.request(scope void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse)) [0x558fd1b91c65]
../../.dub/packages/vibe-d-0.9.4/vibe-d/http/vibe/http/client.d:108 @safe void vibe.http.client.requestHTTP(vibe.inet.url.URL, void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse), const(vibe.http.client.HTTPClientSettings)) [0x558fd1b909cf]
../../.dub/packages/vibe-d-0.9.4/vibe-d/http/vibe/http/client.d:98 @safe void vibe.http.client.requestHTTP(immutable(char)[], scope void delegate(scope vibe.http.client.HTTPClientRequest), scope void delegate(scope vibe.http.client.HTTPClientResponse), const(vibe.http.client.HTTPClientSettings)) [0x558fd1b908db]
source/Client/ClientHelper/BinanceHelper.d:133 bool Client.ClientHelper.BinanceHelper.BinanceHelper.PrivateCallNoSignature(immutable(char)[], immutable(char)[][immutable(char)[]], ref vibe.data.json.Json, vibe.http.common.HTTPMethod) [0x558fd1a9c417]
source/Client/ClientWrapper/UserSocketWrapperHelper.d:18 Client.ClientWrapper.UserSocketWrapperHelper.UserStreamHelper Client.ClientWrapper.UserSocketWrapperHelper.UserStreamHelper.__ctor(Client.ClientHelper.BinanceHelper.BinanceHelper) [0x558fd1abd08c]
source/Client/ClientWrapper/BinanceWrapper.d:619 Client.ClientWrapper.BinanceWrapper.BinanceWrapper Client.ClientWrapper.BinanceWrapper.BinanceWrapper.__ctor() [0x558fd1ab3c13]
source/Markets/BinanceMarket.d:140 void Markets.BinanceMarket.BinanceMarket.CreateClient() [0x558fd1ac8409]
source/Markets/BinanceMarket.d:47 Markets.BinanceMarket.BinanceMarket Markets.BinanceMarket.BinanceMarket.__ctor() [0x558fd1ac7b91]
source/app.d:49 _Dmain [0x558fd1b5ff1c]
[2021-11-23 11:52:41.123 main(----) dbg] Thread exit main (index 0) (main=true)
[2021-11-23 11:52:41.123 main(----) dia] Main thread exiting
core.exception.InvalidMemoryOperationError@src/core/exception.d(647): Invalid memory operation
----------------
s-ludwig commented 2 years ago

Looks like this may have to do with the version of eventcore (or vibe-core). Did you upgrade those together with vibe-d? Would be interesting to upgrade separately in that case to pinpoint the source.

BTW, I've tried to reproduce the issue, but everything seems to work fine for me (using the same host name).