Yahoo stopped to allow .historical() request : unauthorized error

[stouch]

Bug Report

Describe the bug

.historical() requests now fails with "User not logged in" error. https://www.reddit.com/r/sheets/comments/1farvxr/broken_yahoo_finance_url/

[gadicc]

Thanks, we were tracking this at https://github.com/gadicc/node-yahoo-finance2/issues/349#issuecomment-2335142620 but going to move here to make this easier to find (as a recent issue). My original response:

Mmm, on the Yahoo page for this, e.g. https://finance.yahoo.com/quote/TSLA/history/, I'm pretty sure there used to be a "download" link that would let us retrieve this data as CSV. I don't see that link anymore. Maybe it still appears if you're logged in? I didn't try as I don't have a Yahoo account.

In any event, all the data on the page itself (vs from a "download" link), seems to come from the chart API now. So, I guess our options are:

1. See if this is temporary and will fix itself again
2. See if it's possible if you're logged in, adapt the library to support Yahoo credentials
3. Probably best option, move over to the chart API (although, it would be nice if we adapted historical() to do this for the user with no code changes - I think we should do that but I can't commit to a time yet, maybe someone else will have a chance to do a PR before I get it).

https://github.com/gadicc/node-yahoo-finance2/blob/devel/docs/modules/chart.md

[technologyatyottoldotcom]

{"finance":{"result":null,"error":{"code":"forbidden","description":"User is not subscribed to Premium or has invalid cookies"}}}

[freudl]

@gadicc

I'm pretty sure there used to be a "download" link that would let us retrieve this data as CSV. I don't see that link anymore. Maybe it still appears if you're logged in? I didn't try as I don't have a Yahoo account.

Seems to be gone for authenticated users as well.

[dtslvr]

Probably best option, move over to the chart API

I guess I was able to fix the problem in the short term at Ghostfolio by switching from historical() to chart() and transforming the result like this: https://github.com/ghostfolio/ghostfolio/pull/3737/files#diff-7be88e7782ecbd969f1651da9ab3b2fc9ec36a2963784030dc2cbcec79cae690R313

[Verdant31]

Probably best option, move over to the chart API

I guess I was able to fix the problem in the short term at Ghostfolio by switching from historical() to chart() and transforming the result like this: https://github.com/ghostfolio/ghostfolio/pull/3737/files#diff-7be88e7782ecbd969f1651da9ab3b2fc9ec36a2963784030dc2cbcec79cae690R313

Worked for me, its a good temporary solution for those that are looking for this

[stouch]

Thanks ! According to https://github.com/ghostfolio/ghostfolio/pull/3737/files#diff-7be88e7782ecbd969f1651da9ab3b2fc9ec36a2963784030dc2cbcec79cae690R313, these are simple steps to temporary fix this issue :

1. Replace your .historical() calls by .chart() calls
2. Just use this function to convert the chart call result to historical result:

import { ChartResultArray } from 'yahoo-finance2/dist/esm/src/modules/chart'; import { HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

/ // Original answer: const convertToHistoricalResult = ( result: ChartResultArray ): HistoricalHistoryResult => { return result.quotes; } /

// ⚠️ As chart and historical types have some different behaviour, you may have to convert a little bit differently with :

const convertToHistoricalResult = ( result: ChartResultArray ): HistoricalHistoryResult => { return result.quotes .map((quote) => ({ ...quote, open: quote.open || null, high: quote.high || null, low: quote.low || null, close: quote.close || null, volume: quote.volume || null, })) .filter( (dailyQuote) => dailyQuote.low !== null || dailyQuote.high !== null ); };

3. For type resolution, try to use `"moduleResolution": "node",` in your `tsconfig.json` to resolve yahoo-finance2/dist/... types.

--- 

In case you used .historical to get dividends, use this function instead of `convertToHistoricalResult` : 

```typescript
import { HistoricalDividendsResult, HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

// ...

const convertToDividendResult = (
    result: ChartResultArray
  ): HistoricalDividendsResult => {
    return result.events.dividends.map(({ amount: dividends, date }) => {
      return { date, dividends };
    });
}

[gadicc]

Hey all, just a big thanks from me to everyone above, for sharing all your findings, and most especially, your solutions and their verification, which is a great help to the community at a time when I'm tied down with other commitments.

I'll just note that switching to yf2's chart() API really is the correct path forward here. All I intend to do when I have a chance is implement the same transformers like you have above into historical() to ease things for other busy people, but will still include a deprecation notice and warning to move over to chart().

Thanks again everyone :pray:

[mattpetters]

Thanks ! According to https://github.com/ghostfolio/ghostfolio/pull/3737/files#diff-7be88e7782ecbd969f1651da9ab3b2fc9ec36a2963784030dc2cbcec79cae690R313, these are simple steps to temporary fix this issue :

1. Replace your .historical() calls by .chart() calls
2. Just use this function to convert the chart call result to historical result:

import { ChartResultArray } from 'yahoo-finance2/dist/esm/src/modules/chart';
import { HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

/*
// Original answer:
const convertToHistoricalResult = (
    result: ChartResultArray
  ): HistoricalHistoryResult => {
    return result.quotes;
}
*/

// ⚠️ As chart and historical types have some different behaviour, you may have to convert a little bit differently with : 

const convertToHistoricalResult = (
  result: ChartResultArray
): HistoricalHistoryResult => {
  return result.quotes
    .map((quote) => ({
      ...quote,
      open: quote.open || null,
      high: quote.high || null,
      low: quote.low || null,
      close: quote.close || null,
      volume: quote.volume || null,
    }))
    .filter(
      (dailyQuote) => dailyQuote.low !== null || dailyQuote.high !== null
    );
};

3. For type resolution, try to use "moduleResolution": "node", in your tsconfig.json to resolve yahoo-finance2/dist/... types.

In case you used .historical to get dividends, use this function instead of convertToHistoricalResult :

import { HistoricalDividendsResult, HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

// ...

const convertToDividendResult = (
    result: ChartResultArray
  ): HistoricalDividendsResult => {
    return result.events.dividends.map(({ amount: dividends, date }) => {
      return { date, dividends };
    });
}

Thanks for laying this out so plainly, refactoring a project now and this is a godsend

[DreamSkyMXDan]

Thanks for the temp solution. Does anyone know if historical() will be fixed? When will it probably be fixed? thanks

[GiorgioBaz]

Thanks ! According to https://github.com/ghostfolio/ghostfolio/pull/3737/files#diff-7be88e7782ecbd969f1651da9ab3b2fc9ec36a2963784030dc2cbcec79cae690R313, these are simple steps to temporary fix this issue :

1. Replace your .historical() calls by .chart() calls
2. Just use this function to convert the chart call result to historical result:

import { ChartResultArray } from 'yahoo-finance2/dist/esm/src/modules/chart';
import { HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

/*
// Original answer:
const convertToHistoricalResult = (
    result: ChartResultArray
  ): HistoricalHistoryResult => {
    return result.quotes;
}
*/

// ⚠️ As chart and historical types have some different behaviour, you may have to convert a little bit differently with : 

const convertToHistoricalResult = (
  result: ChartResultArray
): HistoricalHistoryResult => {
  return result.quotes
    .map((quote) => ({
      ...quote,
      open: quote.open || null,
      high: quote.high || null,
      low: quote.low || null,
      close: quote.close || null,
      volume: quote.volume || null,
    }))
    .filter(
      (dailyQuote) => dailyQuote.low !== null || dailyQuote.high !== null
    );
};

3. For type resolution, try to use "moduleResolution": "node", in your tsconfig.json to resolve yahoo-finance2/dist/... types.

In case you used .historical to get dividends, use this function instead of convertToHistoricalResult :

import { HistoricalDividendsResult, HistoricalHistoryResult } from 'yahoo-finance2/dist/esm/src/modules/historical';

// ...

const convertToDividendResult = (
    result: ChartResultArray
  ): HistoricalDividendsResult => {
    return result.events.dividends.map(({ amount: dividends, date }) => {
      return { date, dividends };
    });
}

This is amazing thank you so much <3

[Emre4x0]

hello, does anyone know when the error will be fixed on .historical()? i will update my code to .chart() accordingly. if .historical() is to be fixed soon, i don't want to change my code. 🫠

[sachin8606]

Historical is working fine in python script, for node js you can use chart() function

[gadicc]

Thanks, everyone. Finally had a chance to work on this. As discussed, it's simply a convenience to map the requests to chart() for you... you'd be better using chart() directly. The first time you call historical(), you'll get a deprecation notice like this:

[Deprecated] historical() relies on an API that Yahoo have removed.  We'll map this request to chart()
for convenience, but, please consider using chart() directly instead; for more info see
https://github.com/gadicc/node-yahoo-finance2/issues/795.  This will only be shown once, but you
can suppress this message in future with `yahooFinance.supressNotices(['ripHistorical'])`.

There'll be an automated message here once the build is published. As this build contains a number of other big internal changes, early testing and feedback (even without using historical()) would be greatly appreciated. All tests are passing so we assume everything should work like usual, but, since we completely changed the way validation works, something might have slipped through. (For any such issues other than historical(), please open a new issue).

[gadicc]

:tada: This issue has been resolved in version 2.12.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[gadicc]

Resolved but leaving this issue open for feedback and discoverability.

[gadicc]

Oops, now I'm seeing some of those null errors that @stouch mentioned :sweat_smile: That fix will come next :sweat_smile::sweat_smile:

e.g. yahoo-finance historical 0P0000XTS7 '{"period1":"2024-09-12"}'

[gadicc]

Above issue fixed in 2.12.1. I re-used the old row null logic from before (and as described in docs/historical.md#null-rows), which skips when entries when everything is null, but throws otherwise and asks the user to report. In the last few years that's never happened but who knows, could come in handy in the future :sweat_smile:

P.S. @stouch, in your fix you have { volume: quote.volume || null }. Although it's rare, volume can be 0, in which case this will make it null, which might be unintended, depending on your use case. (Quick fix, use ?? instead of ||. I've never seen 0 for the other fields but better safe than sorry :stuck_out_tongue:).

[mtorregrosa]

I've just updated to version 2.12.1, but I cannot compile because the following types are no longer exported:

import { HistoricalDividendsResult, HistoricalHistoryResult, HistoricalRowDividend, HistoricalRowHistory } from 'yahoo-finance2/dist/esm/src/modules/historical'; import { QuoteResponseArray } from 'yahoo-finance2/dist/esm/src/modules/quote'; import { CalendarEvents, DefaultKeyStatistics, Price, QuoteSummaryResult, SummaryDetail, SummaryProfile } from 'yahoo-finance2/dist/esm/src/modules/quoteSummary-iface';

Any ideas on how to solve it.

Thanks

[gadicc]

@mtorregrosa, right you are, this was a mistake on our side when we changed over to the new type system. Hopefully fixed in 2.12.2, can you confirm?

And everyone, one other critical fix in this release is the broken coercion (e.g. you wouldn't get Date instances and correct "transformed" types where expected). This is fixed now too, sorry about that! :bow:

[mtorregrosa]

For me, seems all fine in version 2.12.2, except that the following types have dissapeared:

import { HistoricalRowDividend, HistoricalRowHistory } from 'yahoo-finance2/dist/esm/src/modules/historical'

[mtorregrosa]

Furthermore, types seem to have changed from previous versions. A lot of numeric fields that previously where of type number, now are of type number | { raw: number }. For example: regularMarketPrice, beta, dividendYield, bookValue, etc.

[gadicc]

Thanks, @mtorregrosa, this is really helpful. Our apologies again for the hassle and our thanks for taking the time to report and for your patience. I'm moving the type issues to https://github.com/gadicc/node-yahoo-finance2/issues/797 so we can track them all in one place :pray:

[gadicc]

Another small issue with the conversion that I found, for adjClose, because:

1. In historical() it's adjClose
2. In chart() it's adjclose

We don't currently handle this correctly, I need to fix it.

[gadicc]

Above fixed / released in 2.12.14 :tada: