_DWX_MTX_SEND_MARKETDATA_REQUEST_() only returns data of size <= 4096 bytes long

[geo1590]

The bug would only give me a valid response that is about < 4096 bytes long. For larger response sizes, using Wireshark the response is only about 1-2 bytes long ('00 00'). From this, I knew the bug exists on the MT4 side. I was able to track the bug to the InformPullClient() found inside DWX_ZeroMQ_Server_v2.0.1_RC8.mq4. With this fix (see below for details), the response was tested up to 65500 bytes long. The EURUSD daily goes as far as 2012.08.28 time on DarwinEx demo account which gives me about 2046 prices as of 07/22/2020, so I could not test it for larger response sizes. I could not further troubleshoot the issue since 'ZmqMsg pushReply()' is probably inside a libzmq.dll file. But, this fix does work for me.

My environment is Ubuntu 20.04 (focal) running on a VirtualBox. The MT4 is running inside Wine. The python version is 3.8.2.

Original Code:

File: DWX_ZeroMQ_Server_v2.0.1_RC8.mq4

void InformPullClient(Socket& pSocket, string message) {
   ZmqMsg pushReply(StringFormat("%s", message));

   pSocket.send(pushReply,true); // NON-BLOCKING
}

Fixed Code:

File: DWX_ZeroMQ_Server_v2.0.1_RC8.mq4

void InformPullClient(Socket& pSocket, string message) {
   ZmqMsg pushReply(StringFormat("%s", message));

   pSocket.send(message,true); // NON-BLOCKING
}

My test code:

import time
import pprint

from DWX_ZeroMQ_Connector_v2_0_1_RC8 import DWX_ZeroMQ_Connector

g_zmq = DWX_ZeroMQ_Connector()
time.sleep(3)

t_market_data = g_zmq._DWX_MTX_SEND_MARKETDATA_REQUEST_(
    _symbol='EURUSD',     
    _timeframe=1440,
    _start='2010.01.01',      
    _end='2020.07.20 12:00:00',
    )

time.sleep(5)
t_response = my_get_response()
print('-- t_response:')
pprint.pprint(t_response)

It would be good if someone can fix the ZmqMsg pushReply() function.

[geo1590]

I just want to update since my last post. I found that the StringFormat() will not handle string sizes > 4096 bytes (about) or the combination of using pushReply() and StringFormat() will not work together for these sizes.

So, here is my new fix that works for me. Also, the fix in my original post also works. What I am trying to do here is to isolate where the bug may be.

// This does NOT work.
void InformPullClient(Socket& pSocket, string message) {
   // ZmqMsg pushReply(StringFormat("%s", message));

   string t_msg;
   t_msg = StringFormat("%s", message);
   pSocket.send(t_msg,true); // NON-BLOCKING
}

// This does work.
void InformPullClient(Socket& pSocket, string message) {
   ZmqMsg pushReply(message);

   pSocket.send(pushReply,true); // NON-BLOCKING
}

[elvinex]

Hi, thank you very much for bringing up this issue. Are you sure it has to do with the StringFormat() function? I tested it and did not found an error when using the same start/end dates as in your example. However, I also don't see the necessity for calling StringFormat("%s", message), so we might delete it anyway.

A possible issue when accessing historic data is that MT4 often takes time to update data from non-chart symbols and time frames. Maybe this was causing the observed behaviour in your case. It is a known MT4 issue and only solved by either having the specific chart open, or by retrying multiple times. We will publish an update soon, where the mql4 part will retry multiple times to reduce the frequency of this issue.