factory InstrumentsCandlesFactory Invalid value specified for 'to'. Time is in the future

[svenissimo]

Hi Mr Hootnot.

Not sure if this is an issue with the factory or Oanda have changed their behaviour in practice v20.

Seems that you can no longer get the last daily candle. Once upon a time the last candle would be returned for current day and then oanda would have an attribute in the json that showed the candle was complete or not.

I think they have removed this behaviour and you can no longer get the open candle.

So assuming I call the below on the 21st June 2019 the following happens.

IINFO:oandapyV20.oandapyV20:performing request https://api-fxpractice.oanda.com/v3/instruments/EUR_USD/candles
DEBUG:urllib3.connectionpool:https://api-fxpractice.oanda.com:443 "GET /v3/instruments/EUR_USD/candles?granularity=D&includeFirst=True&from=2019-06-20T00%3A00%3A00Z&to=2019-06-21T08%3A47%3A51Z HTTP/1.1" 400 74
ERROR:oandapyV20.oandapyV20:request https://api-fxpractice.oanda.com/v3/instruments/EUR_USD/candles failed [400,{"errorMessage":"Invalid value specified for 'to'. Time is in the future"}]

    params = {
        'granularity': 'D',
        'from': '2017-01-01T00:00:00Z',
        'count': 100
    }
    for r in InstrumentsCandlesFactory(instrument='EUR_USD', params=params):
        res = api.request(r)

I can make it go away / workaround if I set the 'to' to today -1 to avoid the future call.

    params = {
        'granularity': 'D',
        'from': '2017-01-01T00:00:00Z',
        'to': '2019-06-20T00:00:00Z',
        'count': 100
    }

[hootnot]

Hi,

I can't reproduce that, my output:

      "complete": true,
      "volume": 23056,
      "time": "2019-06-19T21:00:00.000000000Z",
      "bid": {
        "o": "1.12250",
        "h": "1.13165",
        "l": "1.12248",
        "c": "1.12907"
      },
      "mid": {
        "o": "1.12264",
        "h": "1.13174",
        "l": "1.12264",
        "c": "1.12916"
      },
      "ask": {
        "o": "1.12278",
        "h": "1.13183",
        "l": "1.12278",
        "c": "1.12924"
      }
    },
    {
      "complete": false,
      "volume": 8590,
      "time": "2019-06-20T21:00:00.000000000Z",
      "bid": {
        "o": "1.12901",
        "h": "1.13140",
        "l": "1.12825",
        "c": "1.13039"
      },
      "mid": {
        "o": "1.12931",
        "h": "1.13159",
        "l": "1.12831",
        "c": "1.13044"
      },
      "ask": {
        "o": "1.12961",
        "h": "1.13178",
        "l": "1.12837",
        "c": "1.13049"
      }
    }
  ]
}

So, the last one is the incomplete record of today.

What version do you use ?

[svenissimo]

Hi I think its my timezone or clock skew

        "index": "pypi",
        "version": "==0.6.3"

The bars coming back are 21:00 UTC. if I check the error its attempting a 'to' of 10:56 on the 21st. (its 11:56 local time here)

just checked and if I set the 'to' in the params to the 21st but start of day 2019-06-21T00:00:00Z it brings back an open candle for today.

seems UTC/ BST or my local clock is off because if I set it back 60seconds everything works (without specifying 'to') :/

[svenissimo]

I've just refreshed my 'time' (auto set anyway) and now it work fine so I'm thinking clock skew. It was around 2 secs off.

with that in mind I know I can work around by specifying T00:00:00 for daily candles.

not sure your library can cater for such issues but in the case of longer timeframes like D its easy enough to cater for stupid clocks. ta

[hootnot]

the to-date is calculated as a UTC time. But the time should not be 'in the future'. You say your clock was off 2 secs. That is not a lot. But running the script it will be enough to be 'in the future'. Using ntp will save you here.

I have code running to update a database with different granularities. Never had an issue.

Good luck with further developments!

I consider this one as closed.

[svenissimo]

yes of course close it.. and I'm supposed to have synchronised clocks but this was on my development mac and...well macs :)

thanks for taking the time and much appreciatd.

[Umthwa]

I'm having the same problem. Today, I've routinely been receiving the same error message i.e. "Invalid value specified for 'to'. Time is in the future". It's not the first time I'm receiving this message, it has happened before, but it's been several weeks of using the same code successfully without it happening. That was, until today.

It's rather frustrating because, at face value, the code itself is correct and ought not produce any issues (and, for the most part, it functions perfectly) but it just decides to misbehave. Inexplicably.

end_date = datetime.datetime.utcnow(); params = {"granularity": granularity, "to": DateTime(end_date).value, "count": N} r = instruments.InstrumentsCandles(instrument= instrument, params= params); rv = client.request(r)

[Umthwa]

For instance, this same code is back to working now, without that recurrent error message, even though I made absolutely no changes to it. What could be the cause? it would be nice to know so that, in future, I can avoid it and run my code continuously with confidence that there will be no interruptions.

[hootnot]

Check the time you send. It is definitely in the future

[Umthwa]

As you can see, I specify the time instantaneously as datetime.datetime.utcnow(). Just like the "to" specification in params, it is UTC. Inspection doesn't seem to suggest that it's in the future, unless such a discrepancy is on the order of microseconds or nanoseconds.

The code is working now though. And I haven't changed anything. Not sure where the error might lie; human, software or hardware?

[hootnot]

check you clock, is ts synced NTP? If OANDA raises the error that your time is in the future, then you can assume it is I guess. The library does not do anything with that parameter. So, it is between you and OANDA. If it is working now and several hours back it didn't, maybe you only have a small time difference. Make sure it is NTP synchronized