search

zgsrc / ibjs

Container to host Interactive Brokers trading system logic written in Javascript.
31 stars 10 forks source link

Sugar created dates do not support timezone #12

Closed tfrojd closed 6 years ago

tfrojd commented 6 years ago

Thanks for a great library, It's my first stab at node so please bear with me if I misunderstood something.

When creating a contract object the dates in .schedule are all Invalid dates for me. It might be because I am trying to run this on the Swedish stock exchange in the CET timezone.

I have tracked this error down to the lines in contract.js that looks like this:

Date.create(date.format("{Month} {dd}, {yyyy}") + " " + time[0] + ":00 " + timeZoneId, { future: true })

In my case the string to be parsed look like this

Date.create("January 12, 2018 09:00:00 MET", { future: true })

That returns an invalid date. A reproducible example below

require("sugar").extend();
console.log("startdate " + Date.create("January 12, 2018 09:00:00 MET", { future: true })); // IB format
console.log("startdate " + Date.create("January 12, 2018 09:00:00 CET", { future: true })); // MET and CET is the same TZ
console.log("startdate " + Date.create("January 12, 2018 09:00:00 UTC", { future: true })); // Returns a CET date. not UTC
console.log("startdate " + Date.create("January 12, 2018 09:00:00", { future: true }));

returns

startdate Invalid Date startdate Invalid Date startdate Fri Jan 12 2018 10:00:00 GMT+0100 (CET) startdate Fri Jan 12 2018 09:00:00 GMT+0100 (CET)

On the Sugar npm page they say that sugar can't handle timezones directly but you can use another library for that.

Sugar does not deal with timezone abbreviations (i.e. "PST", etc). Timezone offsets will be correctly parsed if they are in ISO-8601 format (+09:00, +0900, or Z for UTC), however if an abbreviation exists it will be ignored. Sugar however plays nicely with other libraries that offer full timezone support such as timezone.js.

Any ideas on how to solve this?

zgsrc commented 6 years ago

Hmm... that's a good one. I'll have to check it out. I have really only tested on NYC exchanges and haven't looked at the library in a few months.

I agree sugar doesn't do well with timezones. I think my intention is to add moment.js. My particular build or version of node.js may have let me get away with the timezone hack you found above.

zgsrc commented 6 years ago

I pushed an update to github that may take care of the issue. Want to check it out and tell me if it takes care of the problem you found?

tfrojd commented 6 years ago

Absolutely! Probably won’t have time until thursday But I’ll come back to you as soon as I can. sön 14 jan. 2018 kl. 18:35 skrev Jonathan Sterling Hollinger < notifications@github.com>:

I pushed an update to github that may take care of the issue. Want to check it out and tell me if it takes care of the problem you found?

On Sat, Jan 13, 2018 at 10:41 AM Jonathan Hollinger < mail@jonathanhollinger.com> wrote:

Hmm... that's a good one. I'll have to check it out. I have really only tested on NYC exchanges and haven't looked at the library in a few months.

I agree sugar doesn't do well with timezones. I think my intention is to add moment.js. My particular build or version of node.js may have let me get away with the timezone hack you found above.

On Fri, Jan 12, 2018 at 7:01 AM tfrojd notifications@github.com wrote:

Thanks for a great library, It's my first stab at node so please bear with me if I misunderstood something.

When creating a contract object the dates in .schedule are all Invalid dates for me. It might be because I am trying to run this on the Swedish stock exchange in the CET timezone.

I have tracked this error down to the lines in contract.js that looks like this:

Date.create(date.format("{Month} {dd}, {yyyy}") + " " + time[0] + ":00 "

  • timeZoneId, { future: true })

In my case the string to be parsed look like this

Date.create("January 12, 2018 09:00:00 MET", { future: true })

That returns an invalid date. A reproducible example below

require("sugar").extend();console.log("startdate " + Date.create("January 12, 2018 09:00:00 MET", { future: true })); // IB formatconsole.log("startdate " + Date.create("January 12, 2018 09:00:00 CET", { future: true })); // MET and CET is the same TZconsole.log("startdate " + Date.create("January 12, 2018 09:00:00 UTC", { future: true })); // Returns a CET date. not UTCconsole.log("startdate " + Date.create("January 12, 2018 09:00:00", { future: true }));

returns

startdate Invalid Date startdate Invalid Date startdate Fri Jan 12 2018 10:00:00 GMT+0100 (CET) startdate Fri Jan 12 2018 09:00:00 GMT+0100 (CET)

On the Sugar npm page https://www.npmjs.com/package/sugar they say that sugar can't handle timezones directly but you can use another library for that.

Sugar does not deal with timezone abbreviations (i.e. "PST", etc). Timezone offsets will be correctly parsed if they are in ISO-8601 format (+09:00, +0900, or Z for UTC), however if an abbreviation exists it will be ignored. Sugar however plays nicely with other libraries that offer full timezone support such as timezone.js.

Any ideas on how to solve this?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/triploc/ib-sdk/issues/12, or mute the thread < https://github.com/notifications/unsubscribe-auth/AA9hiXAzy0svHNvhPiUhcbfKPS7UfuDyks5tJ0mNgaJpZM4RcMu7

.

-- Jonathan Hollinger mail@jonathanhollinger.com

-- Jonathan Hollinger mail@jonathanhollinger.com

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/triploc/ib-sdk/issues/12#issuecomment-357527929, or mute the thread https://github.com/notifications/unsubscribe-auth/AA1gU7i1AK0Xbpreu7Hs1Tf4gYn6Dcdnks5tKjr2gaJpZM4RcMu7 .

tfrojd commented 6 years ago

Hi, thanks for the reply. I have looked a bit more on it. Sadly I can not get it to work.

First It refused to run the script at all complaining that the variable ”date ”in contract.js is defined twice. I suspect this is just a mistake and that the first definition on row 59 should be deleted.

Secondly the 

Date.create(date + " " + (time || "00:00:00")).format("{Weekday}, {dd} {Mon} {yyyy} {HH}:{mm}:{ss}") + " " + (timezone || "")   
    )

Part in the  datetime() function does not seem to evaluate correctly. For me the value of date that gets passed into the function is ”2018-01-18T23:00:00.000Z” when I console.log() it.

The value of the time variable passed in is ”09:00”.

The evaluation for ”date + " " + (time || "00:00:00”)” part is ”Fri Jan 19 2018 00:00:00 GMT+0100 (CET) 09:00” It therefore seems to me that on your system date evaluates only to the date part while on mine it adds the timestamp and timezone info rendering the expression meaningless. Maybe it is because we have different node setups.

Third, fixing this, I have not been able to get DateTime.fromHTTP() to accept CET or MET as a timezone although I might use it wrong.

My experience looks like this


"use strict";
require("sugar").extend();
const { DateTime } = require('luxon');

// trying to simulate response when running the script 20170117 after market close in Stockholm
// market opens 09.00 next day
let date = Date.create("20180117", { future: true });
const time = "09:00"; // time from IB 

//How luxor would convert it to HTTP header format
let luxordate = DateTime.fromJSDate(date);
console.log(date); // no idea why the date is 16th here
console.log(luxordate.toHTTP() ); // same here

// simulating what is happening when ib-sdk is run
let formatted_date = date.format('%a, %d {Mon} {yyyy}') + " " + time + ":00" + " " + "MET"
console.log("\n" +formatted_date) // date got correct for some reason
console.log( DateTime.fromHTTP(formatted_date) ) //fails to parse likely because "MET" is not recognized

// Trying a with "GMT" as timezone
formatted_date = date.format('%a, %d {Mon} {yyyy}') + " " + time + ":00" + " " + "GMT"
console.log("\n" +formatted_date)
console.log( DateTime.fromHTTP(formatted_date) ) // ignores the timezone provided and instead returns my local timezone, time is one hour off from what I want (09.00)

And output looks like this

2018-01-16T23:00:00.000Z Tue, 16 Jan 2018 23:00:00 GMT

Wed, 17 Jan 2018 09:00:00 MET DateTime { Invalid, reason: unparsable }

Wed, 17 Jan 2018 09:00:00 GMT DateTime { ts: 2018-01-17T10:00:00.000+01:00, zone: Europe/Stockholm, locale: und }

Even if you don't plan to spend time on this I would be happy to hear any suggestions for what could be done. From my limited experience with your library most things seems to work but there is a problem when getting the order book as an error is triggered if the market is closed. Another solution for me might be to just handle that error.

Thanks in advance!

zgsrc commented 6 years ago

Think I fixed it.

tfrojd commented 6 years ago

Awesome! The security.js example now runs as expected for my contract at the Swedish stock exchage with the exception of the line

(await AAPL.charts.stream()).log();

I probably won't use this anyway and it might very well be that this part of the API is not valid for this contract so I won't look more into it. But if you are interested I post the error output below.

Thanks again, looking forward to start using your package.

UNCAUGHT ERROR MESSAGE
Error: Cannot send data when disconnected.
    at Controller.emitError (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:101:22)
    at Controller._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:73:10)
    at Controller.<anonymous> (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:35:21)
    at CommandBuffer._process (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:65:16)
    at CommandBuffer.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:101:12)
    at Controller.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:117:27)
    at IB._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:20:20)
    at IB.cancelRealTimeBars (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:145:8)
    at dispatch.instance.req (/Users/thomas/Documents/ib-bib/ib-sdk/service/service.js:143:39)
    at Request.cancel (/Users/thomas/Documents/ib-bib/ib-sdk/service/request.js:85:17)
UNCAUGHT ERROR MESSAGE
Error: Cannot send data when disconnected.
    at Controller.emitError (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:101:22)
    at Controller._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:73:10)
    at Controller.<anonymous> (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:35:21)
    at CommandBuffer._process (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:65:16)
    at CommandBuffer.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:101:12)
    at Controller.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:117:27)
    at IB._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:20:20)
    at IB.reqHeadTimestamp (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:284:8)
    at dispatch.instance.req (/Users/thomas/Documents/ib-bib/ib-sdk/service/service.js:142:28)
    at Request.send (/Users/thomas/Documents/ib-bib/ib-sdk/service/request.js:64:21)
UNCAUGHT ERROR MESSAGE
Error: Cannot send data when disconnected.
    at Controller.emitError (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:101:22)
    at Controller._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:73:10)
    at Controller.<anonymous> (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:35:21)
    at CommandBuffer._process (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:65:16)
    at CommandBuffer.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:101:12)
    at Controller.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:117:27)
    at IB._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:20:20)
    at IB.reqRealTimeBars (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:398:8)
    at dispatch.instance.req (/Users/thomas/Documents/ib-bib/ib-sdk/service/service.js:142:28)
    at Request.send (/Users/thomas/Documents/ib-bib/ib-sdk/service/request.js:64:21)
UNCAUGHT ERROR MESSAGE
Error: Cannot send data when disconnected.
    at Controller.emitError (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:101:22)
    at Controller._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:73:10)
    at Controller.<anonymous> (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:35:21)
    at CommandBuffer._process (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:65:16)
    at CommandBuffer.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/command-buffer/dist/command-buffer.js:101:12)
    at Controller.schedule (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/controller.js:117:27)
    at IB._send (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:20:20)
    at IB.cancelRealTimeBars (/Users/thomas/Documents/ib-bib/ib-sdk/node_modules/ib/lib/index.js:145:8)
    at dispatch.instance.req (/Users/thomas/Documents/ib-bib/ib-sdk/service/service.js:143:39)
    at Request.cancel (/Users/thomas/Documents/ib-bib/ib-sdk/service/request.js:85:17)
{ Error: Request reqRealTimeBars({"symbol":"ABB","secType":"STK","expiry":"","strike":0,"right":"","exchange":"SFB","currency":"SEK","localSymbol":"ABB","tradingClass":"ABB","conId":4620605,"multiplier":"","primaryExch":"SFB"}, 5, "TRADES", false) timed out.
    at Timeout.timeout.setTimeout (/Users/thomas/Documents/ib-bib/ib-sdk/service/request.js:27:44)
    at ontimeout (timers.js:466:11)
    at tryOnTimeout (timers.js:304:5)
    at Timer.listOnTimeout (timers.js:264:5) timeout: 10000 }