search

OpenTrading / OTMql4Zmq

Open Trading Metatrader 4 ZeroMQ Bridge
MIT License
99 stars 46 forks source link

Garbled Message published from OTMql4Zmg #1

Closed OpenTrading closed 9 years ago

OpenTrading commented 9 years ago

The first 16 bytes of a message published from OTMql4Zmg are sometimes (not always) garbled, including nulls. The rest of the message is fine. The workaround it to pad the message with 16 bytes.

I checked with wireshark and it is garbled in the transmission.

I checked in Mt4 and it is going to the dll OK - StringToCharArray is not the culprit.

Pity I can't run metatrader under a debugger...

petermjr commented 9 years ago

After modifying OTZmqCmdEA.mq4 as shown in the gist and OTZmqSubscribe.py, it looks like the garbage text comes out of the debug messages

https://gist.github.com/a3bhea/b865edd21ec7055eb81d

OpenTrading commented 9 years ago

Very Interesting. This kind of thing is very possible, and is why in general with OTMql4Py I have to be rigourous about using sys.exc_clear()

I'll look carefully at the gist information, and even if I can't find the cause, I may check in your code to help with the diagnostics. I really would like to get this working, but the problem is hard to diagnose.

I appreciate your help with tracking this down; you've given me something to look for.

PS: if you "correct" my capitalization, it makes it much harder for me to read the diffs...

OpenTrading commented 9 years ago

I put some more diagnostic messages in where you were working, but the results are not good: see the discussion I've written up at https://github.com/OpenTrading/OTMql4Zmq/wiki/CompiledDllOTMql4Zmq The corruption is coming from a Mt4 print statement to stdout after the ZmqSend.

We'll have to wait until someone who knows C can look at it.

The Python version is running fine: I'll start adding a test harness for it: https://github.com/OpenTrading/OTMql4Zmq/wiki/PythonOTMql4Zmq

Thanks very much for your help on this - it really identified the problem. If you're adding logs from tthis, and if you are able to, upload the logs in a zip or rar file: they contain binary data that may get munged by the gist. And tell ne what OS and version, because the messages are probably being generated in Unicode, which also might not survive the gisting.

OpenTrading commented 9 years ago

I may have found the problem. Even with the debugging code in there, everything seems working for Mt4 sending - havent tried publishing to Mt4 yet.

I simply replaced MQL4/Libraries/OTMql4/libzmq.dll with libzmq-v100-mt-4_0_4.dll from http://miru.hk/archive/ZeroMQ-4.0.4~miru1.0-x86.exe

Do a git pull and see if this helps, and followup here.

OpenTrading commented 9 years ago

I spoke too soon: looks like nothing is working either way anymore.

I'll try recompiling with debugging enabled and see if I can make some progress.

OpenTrading commented 9 years ago

I recompiled the mql4zmq.dll against the libzmq-v100-mt-4_0_4.dll from http://miru.hk/archive/ZeroMQ-4.0.4~miru1.0-x86.exe with MSVC 2010 and put them into src/mql4zmq/pre-compiled.

Make sure you find the old dlls and renamed them to something like .old, then copy mql4zmq.dll into you Metatrader MQL4/Libraries/OTMql4 directory and libzmq.dll into your windows system32 or equivalent directory.

Sending from Mt4 works fine now, but receiving by Mt4 doesn't yet: nothing is received. But wireshark says that the message is sent by bin/OTZmqPublish.py and that there is an ACK back, so we are making progress.

Test it out and post back your results here.

OpenTrading commented 9 years ago

Looks like I've been able to solve the problem by dropping back to v2.2.0.

Give me a few days to write it up and test it out, but both directions appear to be working with it. I couldnt't get the libzmq-v100-mt-4_0_4.dll solution above to receive.

OpenTrading commented 9 years ago

Try the most recently checked in code. It's only been lightly tested, but seems to be working both ways. It's based on ZeroMQ v2.2.0.

I'll be testing it more over then next 2 weeks - post your results here when you try it.

OpenTrading commented 9 years ago

This issue I'm closing because the root of the problem was the MSVC compilation of the DLL. We feel that the recompiled DLL has no issues (other than it's an old ZeroMQ rather than 4.x).

The message has zero length is an unrelated issue, and may be just an overzealous debugging warning from when we were tracking down the DLL issues - I'll look into it.