OpenBB-finance / OpenBB

Investment Research for Everyone, Everywhere.
https://openbb.co
Other
34.01k stars 3.11k forks source link

[Bug]stocks/ba/queries - google_view.py: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. #1862

Closed deeleeramone closed 2 years ago

deeleeramone commented 2 years ago

stocks/ba/queries - SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame.

Does not break the terminal, but prints a warning message.

(πŸ¦‹) /stocks/ba/ $ queries
/Users/danglewood/GitHub/OpenBBTerminal/openbb_terminal/common/behavioural_analysis/google_view.py:223: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_related_queries["value"] = df_related_queries["value"].apply(

 Top GME's related queries 
┏━━━━━━━━━━━━━━━━━┳━━━━━━━┓
┃ query           ┃ value ┃
┑━━━━━━━━━━━━━━━━━╇━━━━━━━┩
β”‚ stock gme       β”‚ 100%  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ amc             β”‚ 29%   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ amc stock       β”‚ 24%   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ gme price       β”‚ 23%   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ gme stock price β”‚ 14%   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ gme share       β”‚ 7%    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ gme share price β”‚ 6%    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ reddit          β”‚ 6%    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ gme reddit      β”‚ 6%    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ bb              β”‚ 5%    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”˜
jmaslek commented 2 years ago

myself and @deeleeramone no longer see this

deeleeramone commented 2 years ago

Reopening because I am now seeing this again.

Screen Shot 2022-08-19 at 8 28 18 AM
boyestrous commented 2 years ago

I'm going to try to take this one. I'm also seeing it in stocks/options

boyestrous commented 2 years ago

Learned some things yesterday:

It occurred to me that this expiration is a "standard" expiration instead of a weekly one. So i checked other expirations and was unable to reproduce the warning for:

Needs more investigation. My current theory is that it may be related to the expiration being a "standard" within a month that also contains weekly ones. Not sure why that would be the case, but I'm going to have to keep testing.

deeleeramone commented 2 years ago

Interesting. Any thoughts here, @jmaslek?

jmaslek commented 2 years ago

Does the command still work?

If it's just the warning that's usually just a pandas thing like the warning says

deeleeramone commented 2 years ago

It does work, but it would be nice to get rid of these messages.

image

deeleeramone commented 2 years ago

User report from this morning, where it doesn't work:

image

boyestrous commented 2 years ago

I did some more research last night and found a couple of things:

for stocks/options

the warning occurs only when some of the options in the chain have implied volatility <= 0. Those rows are filtered out in yfinance_model, line 72: df_list = [x[x["impliedVolatility"] > 0] for x in df_list]

On line 80, when greeks are set, we're now modifying a slice of the original DataFrame. This only happens if there isn't enough activity for a certain option and the price remains constant, which is why this warning only appears intermittently.

There's no reason to keep the original df, so adding a .copy() statement will permanently handle the warning df_list = [x[x["impliedVolatility"] > 0].copy() for x in df_list]

for stocks/ba

I don't know why the warning is intermittent and it stopped appearing mid-testing last night (seems to be back this morning).

But, I actually found an unrelated issue that does break the functionality.

The --export flag no longer works for stocks/ba:

I can fix this pretty easily, but it's related to the same code that is likely causing the SettingWithCopyWarning. Should I make it a separate issue?

deeleeramone commented 2 years ago

Screen Shot 2022-08-25 at 12 29 46 PM

queries --export csv does break it in my installer package version.

Screen Shot 2022-08-25 at 12 30 55 PM

Do we need a new issue, or, just keep this one evolving because it is the same command it was raised for, @DidierRLopes?

boyestrous commented 2 years ago

My two cents:

I can fix the export problem and both SettingWithCopyWarnings immediately with just a few tweaks. I am just working on updating the pytests because they don't appear to be properly checking whether the --export flag works correctly.

The other error (related to Google returning 429) is going to be a bit more involved. We'll have to look at the core _get_data() function and figure out how to make it return something that is acceptable to all of the other functions that call it. Right now, it raises an exception, but then it still returns the rest of the JSON response. Shouldn't be too tricky, but it's definitely a separate set of considerations, in my opinion.