search

zeromq / pyzmq

PyZMQ: Python bindings for zeromq
http://zguide.zeromq.org/py:all
BSD 3-Clause "New" or "Revised" License
3.6k stars 634 forks source link

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xac in position 22: invalid start byte #1221

Closed masfen closed 4 months ago

masfen commented 5 years ago

Good Day

I am a bit a of newbie to pyzmq and am getting an error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xac in position 22: invalid start byte.

I am using python 3.6.5, on spyder 3.3.1 and version 17.1.2 of pyzmq

this error does not occur all the time

masfen commented 5 years ago

sorry i forgot to include the code and some more info, pyzmq came pre-installed with anaconda3

full error traceback: 

Time= trade.get_Time2(symbol= 'EURUSD', timeframe= 'H1', start_bar=0, end_bar=2)
Traceback (most recent call last):

  File "<ipython-input-3-9df761737036>", line 1, in <module>
    Time= trade.get_Time2(symbol= 'EURUSD', timeframe= 'H1', start_bar=0, end_bar=2)

  File "C:\Users\User\Documents\Programming\PFTNNS\pythonicMT4.py", line 145, in get_Time2
    self.remote_send(self.reqSocket, self.time)

  File "C:\Users\User\Documents\Programming\PFTNNS\pythonicMT4.py", line 27, in remote_send
    msg_send = socket.recv_string()

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 585, in recv_string
    return self._deserialize(msg, lambda buf: buf.decode(encoding))

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 491, in _deserialize
    return load(recvd)

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 585, in <lambda>
    return self._deserialize(msg, lambda buf: buf.decode(encoding))

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte

the code:

import zmq
import numpy as np

class zmq_python():

    def __init__(self):
        # Create ZMQ Context
        self.context = zmq.Context()

        # Create REQ Socket
        self.reqSocket = self.context.socket(zmq.REQ)
        self.reqSocket.connect("tcp://localhost:5555")

        # Create PULL Socket
        self.pullSocket = self.context.socket(zmq.PULL)
        self.pullSocket.connect("tcp://localhost:5556")

        # Create PULL2 Socket
        self.pullSocket2 = self.context.socket(zmq.PULL)
        self.pullSocket2.connect("tcp://localhost:5557")

    def remote_send(self, socket, data):

        try:
            socket.send_string(data)
            msg_send = socket.recv_string()
            print (msg_send)

        except zmq.Again as e:
            print ("Waiting for PUSH from MetaTrader 4..")
            print (str(e))

    def remote_pull(self, socket):

        try:
            msg_pull = ''
            msg_pull = socket.recv(flags=zmq.NOBLOCK)
            print (msg_pull)
            return msg_pull         

        except zmq.Again as e:
            print ("Waiting for PUSH from MetaTrader 4..")
            print (str(e))

    def get_Open(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.open = "OPEN|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.open)
        Open = self.remote_pull(self.pullSocket)
        Open_str= str(Open)
        if 'OPEN' in Open_str:
            Open_lst= Open_str.split(sep='|')[1:-1]
            Open_lst= [float(i) for i in Open_lst]
            Open_lst= Open_lst[::-1]
            Open_arr= np.array(Open_lst)
            return Open_arr

    def get_High(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.high = "HIGH|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.high)
        High= self.remote_pull(self.pullSocket)
        High_str= str(High)
        if 'HIGH' in High_str:
            High_lst= High_str.split(sep='|')[1:-1]
            High_lst= [float(i) for i in High_lst]
            High_lst= High_lst[::-1]
            High_arr= np.array(High_lst)
            return High_arr

    def get_Low(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.low = "LOW|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.low)
        Low= self.remote_pull(self.pullSocket)
        Low_str= str(Low)
        if 'LOW' in Low_str:
            Low_lst= Low_str.split(sep='|')[1:-1]
            Low_lst= [float(i) for i in Low_lst]
            Low_lst= Low_lst[::-1]
            Low_arr= np.array(Low_lst)
            return Low_arr

    def get_Close(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.close = "CLOSE|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.close)
        Close= self.remote_pull(self.pullSocket)
        Close_str= str(Close)
        if 'CLOSE' in Close_str:
            Close_lst= Close_str.split(sep='|')[1:-1]
            Close_lst= [float(i) for i in Close_lst]
            Close_lst= Close_lst[::-1]
            Close_arr= np.array(Close_lst)
            return Close_arr

    def get_Vol(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.vol = "VOL|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.vol)
        volumne= self.remote_pull(self.pullSocket)
        volumne_str= str(volumne)
        if 'VOL' in volumne_str:
            volumne_lst= volumne_str.split(sep='|')[1:-1]
            volumne_lst= [float(i) for i in volumne_lst]
            volumne_lst= volumne_lst[::-1]
            volumne_arr= np.array(volumne_lst)
            return volumne_arr

    def get_Time(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.time = "TIME|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.time)
        times= self.remote_pull(self.pullSocket)
        time_str= str(times)
        if 'TIME' in time_str:
            time_lst= time_str.split(sep = '|')[1:-1]
            time_lst= [j.split()[-1] for j in time_lst]
            time_lst= [str(i) for i in time_lst]
            time_lst= time_lst[::-1]
            time_arr= np.array(time_lst)
            return time_arr

    def get_Time2(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.time = "TIME|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.time)
        times= self.remote_pull(self.pullSocket)
        time_str= str(times)
        if 'TIME' in time_str:
            time_lst= time_str.split(sep = '|')[1:-1]
            time_lst= [str(i) for i in time_lst]
            time_lst= time_lst[::-1]
            time_arr= np.array(time_lst)
            return time_arr

    def get_Ask(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.ask = "ASK|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.time)
        ask= self.remote_pull(self.pullSocket2)
        ask_str= str(ask)
        if 'ASK' in ask_str:
            ask_lst= ask_str.split(sep='|')[1:-1]
            ask_lst= [float(i) for i in ask_lst]
            ask_lst= ask_lst[::-1]
            ask_arr= np.array(ask_lst)
            return ask_arr

    def get_Bid(self, symbol, timeframe, start_bar, end_bar):
        '''
        only start_bar and end_bar as int
        '''
        self.bid = "BID|"+ symbol+"|"+"PERIOD_"+timeframe+"|"+str(start_bar)+"|"+str(end_bar+1)
        self.remote_send(self.reqSocket, self.time)
        bid= self.remote_pull(self.pullSocket2)
        bid_str= str(bid)
        if 'BID' in bid_str:
            bid_lst= bid_str.split(sep='|')[1:-1]
            bid_lst= [float(i) for i in bid_lst]
            bid_lst= bid_lst[::-1]
            bid_arr= np.array(bid_lst)
            return bid_arr

    def get_ACCINFO(self, INFO):

        self.acc = "ACC|"+ INFO
        self.remote_send(self.reqSocket, self.acc)
        acc= self.remote_pull(self.pullSocket)
        acc_str= str(acc)
        acc_str = acc_str.replace("b", '')
        acc_str = acc_str.replace("'", '')
        if 'NAME' in acc_str:
            acc_str = acc_str.replace("NAME|", '')
            acc_nm = str(acc_str)
            return acc_nm
        elif 'BALANCE' in acc_str:
            acc_str = acc_str.replace("BALANCE|", '')
            acc_bal = float(acc_str)
            return acc_bal
        elif 'EQUITY' in acc_str:
            acc_str = acc_str.replace("EQUITY|", '')
            acc_eq = float(acc_str)
            return acc_eq
        elif 'MARGIN' in acc_str:
            acc_str = acc_str.replace("MARGIN|", '')
            acc_mgn = float(acc_str)
            return acc_mgn
        elif 'FREE' in acc_str:
            acc_str = acc_str.replace("FREE|", '')
            acc_fm = float(acc_str)
            return acc_fm
        elif 'PROFIT' in acc_str:
            acc_str = acc_str.replace("PROFIT|", '')
            acc_pt = float(acc_str)
            return acc_pt
        elif 'UNKNOW' in acc_str:
            acc_un = str(acc_str)
            return acc_un   

    def buy_order(self, symbol, stop_loss, take_profit):
        self.buy= "TRADE|OPEN|0|"+ str(symbol)+"|"+str(stop_loss)+"|"+str(take_profit)
        self.remote_send(self.reqSocket, self.buy)
        reply= self.remote_pull(self.pullSocket)
        return reply

    def sell_order(self, symbol, stop_loss, take_profit):
        self.buy= "TRADE|OPEN|1|"+ str(symbol)+"|"+str(stop_loss)+"|"+str(take_profit)
        self.remote_send(self.reqSocket, self.buy)
        reply= self.remote_pull(self.pullSocket)
        return reply

    def close_buy_order(self):
        self.close_buy= "TRADE|CLOSE|0"
        self.remote_send(self.reqSocket, self.close_buy)
        reply= self.remote_pull(self.pullSocket)
        return reply

    def close_sell_order(self):
        self.close_sell= "TRADE|CLOSE|1"
        self.remote_send(self.reqSocket, self.close_sell)
        reply= self.remote_pull(self.pullSocket)
        return reply
minrk commented 5 years ago

It would appear that the response to remote_send is not a utf8-encoded string. It looks like the code that's sending the reply is not included here. You might have better luck debugging if you show the actual reply:

reply = socket.recv_multipart()
print(reply) # or enter a debugger, if you prefer

I suspect that you aren't getting the reply you expect here. What should the reply look like?

masfen commented 5 years ago

this is the code that sends the reply 

//+------------------------------------------------------------------+
//|                                       ZeroMQ_MT4_EA_Template.mq4 |
//|                                    Copyright 2017, Darwinex Labs |
//|                                        https://www.darwinex.com/ |
//+------------------------------------------------------------------+
#property copyright "Copyright 2017, Darwinex Labs."
#property link      "https://www.darwinex.com/"
#property version   "1.00"
#property strict

// Required: MQL-ZMQ from https://github.com/dingmaotu/mql-zmq
#include <Zmq/Zmq.mqh>

extern string PROJECT_NAME = "DWX_ZeroMQ_Example";
extern string ZEROMQ_PROTOCOL = "tcp";
extern string HOSTNAME = "*";
extern int REP_PORT = 5555;
extern int PUSH_PORT = 5556;
extern int PUSH_PORT2 = 5557;
extern int MILLISECOND_TIMER = 100;  // 1 millisecond

extern string t0 = "--- Trading Parameters ---";
extern int MagicNumber = 123456;
extern int MaximumOrders = 1;
extern double MaximumLotSize = 0.01;
extern int Slippage = 3;
int Total, Ticket, Ticket2;
double StopLossLevel, TakeProfitLevel, StopLevel;

// CREATE ZeroMQ Context
Context context(PROJECT_NAME);

// CREATE ZMQ_REP SOCKET
Socket repSocket(context,ZMQ_REP);

// CREATE ZMQ_PUSH SOCKET
Socket pushSocket(context,ZMQ_PUSH);
Socket pushSocket2(context,ZMQ_PUSH);

// VARIABLES FOR LATER
uchar data[];
ZmqMsg request;

//+------------------------------------------------------------------+
//| Expert initialization function                                   |
//+------------------------------------------------------------------+
int OnInit()
  {
//---

   EventSetMillisecondTimer(MILLISECOND_TIMER);     // Set Millisecond Timer to get client socket input

   Print("[REP] Binding MT4 Server to Socket on Port " + REP_PORT + "..");   
   Print("[PUSH1] Binding MT4 Server to Socket on Port " + PUSH_PORT + ".."); 
   Print("[PUSH2] Binding MT4 Server to Socket on Port " + PUSH_PORT2 + "..");

   repSocket.bind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, REP_PORT));
   pushSocket.bind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));
   pushSocket2.bind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT2));

   /*
       Maximum amount of time in milliseconds that the thread will try to send messages 
       after its socket has been closed (the default value of -1 means to linger forever):
   */

   repSocket.setLinger(5000);  // 1000 milliseconds

   /* 
      If we initiate socket.send() without having a corresponding socket draining the queue, 
      we'll eat up memory as the socket just keeps enqueueing messages.

      So how many messages do we want ZeroMQ to buffer in RAM before blocking the socket?
   */

   repSocket.setSendHighWaterMark(20);     // 5 messages only.

//---
   return(INIT_SUCCEEDED);
  }
//+------------------------------------------------------------------+
//| Expert deinitialization function                                 |
//+------------------------------------------------------------------+
void OnDeinit(const int reason)
{
//---
   Print("[REP] Unbinding MT4 Server from Socket on Port " + REP_PORT + "..");
   repSocket.unbind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, REP_PORT));

   Print("[PUSH] Unbinding MT4 Server from Socket on Port " + PUSH_PORT + "..");
   pushSocket.unbind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT));

   Print("[PUSH] Unbinding MT4 Server from Socket on Port " + PUSH_PORT + "..");
   pushSocket2.unbind(StringFormat("%s://%s:%d", ZEROMQ_PROTOCOL, HOSTNAME, PUSH_PORT2));

}
//+------------------------------------------------------------------+
//| Expert timer function                                            |
//+------------------------------------------------------------------+
void OnTimer()
{
//---

   /*
      For this example, we need:
      1) socket.recv(request,true)
      2) MessageHandler() to process the request
      3) socket.send(reply)
   */

   // Get client's response, but don't wait.
   repSocket.recv(request,true);

   // MessageHandler() should go here.   
   ZmqMsg reply = MessageHandler(request);

   // socket.send(reply) should go here.
   repSocket.send(reply);

}
//+------------------------------------------------------------------+

ZmqMsg MessageHandler(ZmqMsg &request) {

   // Output object
   ZmqMsg reply;

   // Message components for later.
   string components[];

   if(request.size() > 0) {

      // Get data from request   
      ArrayResize(data, request.size());
      request.getData(data);
      string dataStr = CharArrayToString(data);

      // Process data
      ParseZmqMessage(dataStr, components);

      // Interpret data
      InterpretZmqMessage(&pushSocket, components);

      // Construct response
      ZmqMsg ret(StringFormat("[SERVER] Processing: %s", dataStr));
      reply = ret;

   }
   else {
      // NO DATA RECEIVED

   }

   return(reply);
}

// Interpret Zmq Message and perform actions
void InterpretZmqMessage(Socket &pSocket, string& compArray[]) {

   Print("ZMQ: Interpreting Message..");

   // Message Structures:

   // 1) Trading
   // TRADE|ACTION|TYPE|SYMBOL|PRICE|SL|TP|COMMENT|TICKET
   // e.g. TRADE|OPEN|1|EURUSD|0|50|50|R-to-MetaTrader4|12345678

   // The 12345678 at the end is the ticket ID, for MODIFY and CLOSE.

   // 2) Data Requests

   // 2.1) RATES|SYMBOL   -> Returns Current Bid/Ask

   // 2.2) DATA|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME

   // NOTE: datetime has format: D'2015.01.01 00:00'

   /*
      compArray[0] = TRADE or RATES
      If RATES -> compArray[1] = Symbol

      If TRADE ->
         compArray[0] = TRADE
         compArray[1] = ACTION (e.g. OPEN, MODIFY, CLOSE)
         compArray[2] = TYPE (e.g. OP_BUY, OP_SELL, etc - only used when ACTION=OPEN)

         // ORDER TYPES: 
         // https://docs.mql4.com/constants/tradingconstants/orderproperties

         // OP_BUY = 0
         // OP_SELL = 1
         // OP_BUYLIMIT = 2
         // OP_SELLLIMIT = 3
         // OP_BUYSTOP = 4
         // OP_SELLSTOP = 5

         compArray[3] = Symbol (e.g. EURUSD, etc.)
         compArray[4] = Open/Close Price (ignored if ACTION = MODIFY)
         compArray[5] = SL
         compArray[6] = TP
         compArray[7] = Trade Comment
   */

   int switch_action = 0;

   if(compArray[0] == "TRADE" && compArray[1] == "OPEN")
      switch_action = 1;
   if(compArray[0] == "RATES")
      switch_action = 2;
   if(compArray[0] == "TRADE" && compArray[1] == "CLOSE")
      switch_action = 3;
   if(compArray[0] == "TIME")
      switch_action = 4;
   if(compArray[0] == "OPEN")
      switch_action = 5;
   if (compArray[0] == "HIGH")
      switch_action = 6;
   if(compArray[0] == "LOW")
      switch_action = 7;
   if(compArray[0] =="CLOSE")
      switch_action = 8;
   if (compArray[0] == "VOL")
      switch_action = 9;
   if (compArray[0] == "ACC")
      switch_action = 10;

   string ret = "";
   int ticket = -1;
   bool ans = FALSE;
   datetime time_array[];
   ArraySetAsSeries(time_array, true);
   double open_array[];
   ArraySetAsSeries(open_array, true);
   double high_array[];
   ArraySetAsSeries(high_array, true);
   double low_array[];
   ArraySetAsSeries(low_array, true);
   double close_array[];
   ArraySetAsSeries(close_array, true);
   long vol_array[];
   ArraySetAsSeries(vol_array, true);

   int time_count = 0;
   int open_count = 0;
   int high_count = 0;
   int low_count = 0;
   int close_count = 0;
   int vol_count = 0;

   switch(switch_action) 
   {
      case 1: 
         //InformPullClient(pSocket, "OPEN TRADE Instruction Received");
         if (compArray[2] == "0") {
            StopLossLevel = Ask - StrToDouble(compArray[4]) * Point;
            TakeProfitLevel = Ask + StrToDouble(compArray[5]) * Point;
            //InformPullClient(pSocket, "Buy Order");
            Ticket= OrderSend(compArray[3], OP_BUY, MaximumLotSize, Ask, Slippage, StopLossLevel,TakeProfitLevel, "Buy(#" + MagicNumber + ")", MagicNumber, 0, DodgerBlue);
            if(Ticket > 0) {
               if (OrderSelect(Ticket, SELECT_BY_TICKET, MODE_TRADES)) {
                    Print("BUY order opened : ", OrderOpenPrice());
                    InformPullClient(pSocket, "OrderSend placed successfully");}
                  } 
               else {
                  InformPullClient(pSocket, "OrderSend failed");
                  Print("Error opening BUY order : ", GetLastError());

              }
            }
        if (compArray[2] == "1") {
            StopLossLevel = Bid + StrToDouble(compArray[4]) * Point;
            TakeProfitLevel = Bid - StrToDouble(compArray[5]) * Point;
            //InformPullClient(pSocket, "Buy Order");
            Ticket= OrderSend(compArray[3], OP_SELL, MaximumLotSize, Bid, Slippage, StopLossLevel,TakeProfitLevel, "Sell(#" + MagicNumber + ")", MagicNumber, 0, DodgerBlue);
            if(Ticket > 0) {
               if (OrderSelect(Ticket, SELECT_BY_TICKET, MODE_TRADES)) {
                    Print("SELL order opened : ", OrderOpenPrice());
                    InformPullClient(pSocket, "OrderSend placed successfully");}
                  } 
               else {
                  InformPullClient(pSocket, "OrderSend failed");
                  Print("Error opening SELL order : ", GetLastError());

              }
            }
         break;
      case 2: 
         ret = "N/A"; 
         if(ArraySize(compArray) > 1) 
            ret = GetBidAsk(compArray[1]); 

         InformPullClient(pSocket, ret); 
         break;
      case 3:
         if (compArray[2] == "0") {
            OrderSelect(Ticket, SELECT_BY_TICKET, MODE_TRADES);
            Ticket2=OrderClose(OrderTicket(), OrderLots(), Bid, Slippage, MediumSeaGreen);

            ret = StringFormat("Trade Closed (Ticket: %d)", ticket);
            InformPullClient(pSocket, ret);
            }
         if (compArray[2] == "1") {
            OrderSelect(Ticket, SELECT_BY_TICKET, MODE_TRADES);
            Ticket2 = OrderClose(OrderTicket(), OrderLots(), Ask, Slippage, DarkOrange);
            }
         break;

      case 4:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: TIME|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         time_count = CopyTime(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        time_array);

         if (time_count > 0) {

            ret = "";

            // Construct string of time|time|time|.. etc and send to PULL client.
            for(int i = 0; i < time_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + TimeToStr(time_array[i]);
               else if(i > 0) {
                  ret = ret + "|" + TimeToStr(time_array[i]);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;

      case 5:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: OPEN|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         open_count = CopyOpen(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        open_array);

         if (open_count > 0) {

            ret = "";

            // Construct string of open|open|open|.. etc and send to PULL client.
            for(int i = 0; i < open_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + DoubleToStr(open_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(open_array[i], 5);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;            

      case 6:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: HIGH|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         high_count = CopyHigh(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        high_array);

         if (high_count > 0) {

            ret = "";

            // Construct string of price|price|price|.. etc and send to PULL client.
            for(int i = 0; i < high_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + DoubleToStr(high_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(high_array[i], 5);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;  

      case 7:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: LOW|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         low_count = CopyLow(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        low_array);

         if (low_count > 0) {

            ret = "";

            // Construct string of price|price|price|.. etc and send to PULL client.
            for(int i = 0; i < low_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + DoubleToStr(low_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(low_array[i], 5);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;  

      case 8:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: DATA|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         close_count = CopyClose(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        close_array);

         if (close_count > 0) {

            ret = "";

            // Construct string of price|price|price|.. etc and send to PULL client.
            for(int i = 0; i < close_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + DoubleToStr(close_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(close_array[i], 5);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;  

      case 9:
         //InformPullClient(pSocket, "HISTORICAL DATA Instruction Received");

         // Format: DATA|SYMBOL|TIMEFRAME|START_DATETIME|END_DATETIME
         vol_count = CopyTickVolume(compArray[1], StrToInteger(compArray[2]), 
                        StrToInteger(compArray[3]), StrToInteger(compArray[4]), 
                        vol_array);

         if (vol_count > 0) {

            ret = "";

            // Construct string of price|price|price|.. etc and send to PULL client.
            for(int i = 0; i < vol_count; i++ ) {

               if(i == 0)
                  ret = compArray[0] + "|" + DoubleToStr(vol_array[i], 5);
               else if(i > 0) {
                  ret = ret + "|" + DoubleToStr(vol_array[i], 5);
               }   
            }

            Print("Sending: " + ret);

            // Send data to PULL client.
            InformPullClient(pSocket, StringFormat("%s", ret));
            // ret = "";
         }

         break;  

      case 10:
         //InformPullClient(pSocket, "ACC DATA Instruction Received");

         ret = "";

         if(compArray[1] == "NAME"){
            string name =  AccountName();
            ret = compArray[1] + "|" + name; 
         }
         else if(compArray[1] == "BALANCE"){
            double balance = AccountBalance();
            ret = compArray[1] + "|" + DoubleToStr(balance, 2);
         }
         else if(compArray[1] == "EQUITY"){
            double equity = AccountEquity();
            ret = compArray[1] + "|" + DoubleToStr(equity, 5);
         }
         else if(compArray[1] == "MARGIN"){
            double margin = AccountMargin();
            ret = compArray[1] + "|" + DoubleToStr(margin, 5);
         }
         else if(compArray[1] == "FREE"){
            double mf = AccountFreeMargin();
            ret = compArray[1] + "|" + DoubleToStr(mf, 2);
         }
         else if (compArray[1]== "PROFIT"){
            double profit =  AccountProfit();
            ret = compArray[1] + "|" + DoubleToStr(profit, 2);
         }
         else{
            ret = "UNKNOW";
         }

         Print("Sending: " + ret);

         // Send data to PULL client.
         InformPullClient(pSocket, StringFormat("%s", ret));
         // ret = "";

         break;           

      default: 
         break;
   }
}

// Parse Zmq Message
void ParseZmqMessage(string& message, string& retArray[]) {

   Print("Parsing: " + message);

   string sep = "|";
   ushort u_sep = StringGetCharacter(sep,0);

   int splits = StringSplit(message, u_sep, retArray);

   for(int i = 0; i < splits; i++) {
      Print(i + ") " + retArray[i]);
   }
}

//+------------------------------------------------------------------+
// Generate string for Bid/Ask by symbol
string GetBidAsk(string symbol) {

   double bid = MarketInfo(symbol, MODE_BID);
   double ask = MarketInfo(symbol, MODE_ASK);

   return(StringFormat("%f|%f", bid, ask));
}

// Inform Client
void InformPullClient(Socket& pushSocket, string message) {

   ZmqMsg pushReply(StringFormat("%s", message));
   // pushSocket.send(pushReply,true,false);

   pushSocket.send(pushReply,true); // NON-BLOCKING

   // pushSocket.send(pushReply,false); // BLOCKING

}

void InformPullClient2(Socket& pushSocket, string message) {

   ZmqMsg pushReply(StringFormat("%s", message));
   // pushSocket.send(pushReply,true,false);

   pushSocket2.send(pushReply,true); // NON-BLOCKING

   // pushSocket2.send(pushReply,false); // BLOCKING

}
masfen commented 5 years ago

here is the part I don't quite understand, below the error occurs in the first execution and then doesn't in the second 

Time= trade.get_Time2(symbol= 'EURUSD', timeframe= 'H1', start_bar=0, end_bar=2)
Traceback (most recent call last):

  File "<ipython-input-15-9df761737036>", line 1, in <module>
    Time= trade.get_Time2(symbol= 'EURUSD', timeframe= 'H1', start_bar=0, end_bar=2)

  File "C:\Users\User\Documents\Programming\PFTNNS\pythonicMT4.py", line 148, in get_Time2
    self.remote_send(self.reqSocket, self.time)

  File "C:\Users\User\Documents\Programming\PFTNNS\pythonicMT4.py", line 28, in remote_send
    msg_send = socket.recv_string()

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 585, in recv_string
    return self._deserialize(msg, lambda buf: buf.decode(encoding))

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 491, in _deserialize
    return load(recvd)

  File "C:\Users\User\Anaconda3\lib\site-packages\zmq\sugar\socket.py", line 585, in <lambda>
    return self._deserialize(msg, lambda buf: buf.decode(encoding))

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc7 in position 5: invalid continuation byte

Time= trade.get_Time2(symbol= 'EURUSD', timeframe= 'H1', start_bar=0, end_bar=2)

[b'TIME|2018.09.05 10:00|2018.09.05 09:00|2018.09.05 08:00']
b'TIME|2018.09.05 10:00|2018.09.05 09:00|2018.09.05 08:00'
lunarplasma commented 5 years ago

I was having this very issue with pyzmq 17.0.0 but fixed it by updating my package to pyzmq 17.1.0. Perhaps there's a clue there?

minrk commented 5 years ago

Start by looking at the message that you are receiving. That's usually where the clue is that either what you are sending or receiving is not what you expect or intend:

reply = socket.recv_multipart()
print(reply) # or enter a debugger, if you prefer
masfen commented 5 years ago

thank you @lunarplasma this seems to have fix the problem

lunarplasma commented 5 years ago

@masfen Can you confirm that updating to pyzmq 17.1.0+ fixed your problem? Your initial post https://github.com/zeromq/pyzmq/issues/1221#issue-354295421 seems to imply you were already above that version.

I was presuming that this bug is actually an underlying issue in libzmq, and pyzmq 17.1.0 bundles the libzmq 4.2.5 which fixed the problem, but I have no evidence.

ashahba commented 5 years ago

I'm seeing a similar issue during installation of pyzmq>=17 which is currently at 18.0.0.

Downloading https://files.pythonhosted.org/packages/64/8d/78975da77627fd863c08e8ea3c7cebce7e51bed2936be5118de6b0050638/pyzmq-18.0.0.tar.gz (1.2MB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-rv3k_yvp/pyzmq/setup.py", line 1241, in <module>
        long_desc = f.read()
      File "/usr/lib64/python3.4/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 29: ordinal not in range(128)

I put a small PR together to at least fix my issue 🙂 I'll submit it shortly.

ashahba commented 5 years ago

The PR to fix README.md parsing is here #1265