Failed validation: #/definitions/ChartResultObject only at the morning when trading period just started(?)

[himgodfreyho]

Validation Error

Minimal Reproduction

const result = await yahooFinance._chart('^HSI', {
      period1: dayjs().add(-60, 'day').format('YYYY-MM-DD'),
      period2: dayjs().format('YYYY-MM-DD'),
      interval: '1d'
    })

Symbol(s) that it happened for

any HK stock, e.g., ^HSI, 0868.HK, 0968.HK

Error Message

The following result did not validate with schema: #/definitions/ChartResultObject
[
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/quote/0/high/38",
    "schemaPath": "#/definitions/ChartIndicatorQuote/properties/high/items/yahooFinanceType",
    "data": null
  },
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/quote/0/low/38",
    "schemaPath": "#/definitions/ChartIndicatorQuote/properties/low/items/yahooFinanceType",
    "data": null
  },
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/quote/0/open/38",
    "schemaPath": "#/definitions/ChartIndicatorQuote/properties/open/items/yahooFinanceType",
    "data": null
  },
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/quote/0/close/38",
    "schemaPath": "#/definitions/ChartIndicatorQuote/properties/close/items/yahooFinanceType",
    "data": null
  },
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/quote/0/volume/38",
    "schemaPath": "#/definitions/ChartIndicatorQuote/properties/volume/items/yahooFinanceType",
    "data": null
  },
  {
    "keyword": "yahooFinanceType",
    "message": "Expecting number'ish but got null",
    "params": {
      "schema": "number",
      "data": null
    },
    "instancePath": "/indicators/adjclose/0/adjclose/38",
    "schemaPath": "#/definitions/ChartIndicatorAdjclose/properties/adjclose/items/yahooFinanceType",
    "data": null
  }
]

Environment

Browser or Node: node Node version (if applicable): v14.19.0 Npm version: 6.14.16 Browser verion (if applicable): NA Library version (e.g. 1.10.1): 2.3.0

Additional Context

I suspect this only happens when the 'today' chart data is somehow not ready when the trading period just started. This error only happens when I use the function when the trading period just start, e.g., 30min after trading starts. It was ok when I tried it in the afternoon. Also, if I change the period2: dayjs().format('YYYY-MM-DD') to yesterday (e.g., period2: dayjs().add(-1, 'day').format('YYYY-MM-DD'), there will be no errors, while the latest data in the quotes array is 2022-03-04 ('today' is 2022-03-08 though).

Also, if I used yahooFinance.historical with the same option (i.e., period2 = today) and at the similar time I used _chart, there will be no error, but the result from yahooFinance.historical only returns the data till 2022-03-04 ('today' is 2022-03-08)

[gadicc]

Hey @himgodfreyho, thanks for reporting this. Yes, there are a lot of quirks with Yahoo Finance and it's really helpful when users report their findings to help everyone else. Especially when they figure out the exact circumstances to trigger them. So thanks.

Since I can't reproduce this easily (especially in my timezone), it would be really helpful if you could:

1. edit src/modules/chart.spec.ts and around line 20, append to const symbols = testSymbols.filter() a .concat(["^HSI", "0868.HK");
2. run yarn test (it will fail)
3. submit a PR

Assuming you're comfortable with such operations (it's fine if not). That way we can develop and check against the actual data you were getting.

Need to also think about the best way to handle this. I think actually in historical we added code to just skip null rows, which was most useful to the developer. Please let us know if you have any thoughts about it. I don't have the full data result in front of me so I'm not sure if there's any other useful data there.

[samhess]

For me it is sufficiant to set period2 to today insted of leaving it unset, it does not have to be yesterday.

period2: new Date().toISOString().slice(0,10),

[beau-mind]

I think actually in historical we added code to just skip null rows, which was most useful to the developer.

Thank you for this great library. I recently started using it.

I moved to using historical since my code was failing with row of null values for 30 Nov for HDFC.NS. This was failing on 1 Dec. This seems to be an intermittent issue with Yahoo since there are no null values now (2 Dec).

Not sure why the behavior of historical and _chart should differ regarding skipping null rows.