search

ranaroussi / yfinance

Download market data from Yahoo! Finance's API
https://aroussi.com/post/python-yahoo-finance
Apache License 2.0
13.27k stars 2.34k forks source link

Volume = 0 #1935

Open jvallsgm opened 5 months ago

jvallsgm commented 5 months ago

Describe bug

Hello, in the 1 or 5 minute history, for example of 'BTC-USD' often the volume is 0. There seems to be no negative volume at all. Could it be that it is failing volume and whenever it has to be negative a 0 is stored? I have tried both with 'repair' to True or False. Thanks.

Simple code that reproduces your problem

import yfinance as yf stock = yf.Ticker('BTC-USD') historical_prices = stock.history(start='2024-05-09', end='2024-05-10', interval='5m', repair=True)

Debug log

DEBUG Entering history() DEBUG Entering history() DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-09 00:00:00+00:00', 'period2': '2024-05-10 00:00:00+00:00', 'interval': '5m', 'includePrePost': False, 'events': 'div,splits,capitalGains'} DEBUG BTC-USD: yfinance received OHLC data: 2024-05-09 00:00:00 -> 2024-05-09 23:55:00 DEBUG BTC-USD: OHLC after cleaning: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00 DEBUG BTC-USD: OHLC after combining events: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00 DEBUG BTC-USD: checking OHLC for repairs ... DEBUG Entering _fix_unit_mixups() DEBUG Entering _fix_unit_switch() DEBUG Entering _fix_prices_sudden_change() DEBUG price-repair-split: appears_suspended=False, idx_latest_active=0 (2024-05-09) INFO price-repair-split: No 100x errors detected DEBUG Exiting _fix_prices_sudden_change() DEBUG Exiting _fix_unit_switch() DEBUG Entering _fix_unit_random_mixups() INFO price-repair-100x: No sporadic 100x errors DEBUG Exiting _fix_unit_random_mixups() DEBUG Exiting _fix_unit_mixups() DEBUG Entering _fix_bad_stock_split() DEBUG Exiting _fix_bad_stock_split() DEBUG Entering _fix_zeroes() DEBUG Entering _reconstruct_intervals_batch() DEBUG min_dt=2024-03-18 00:00:00+00:00 interval=5m sub_interval=2m DEBUG grp_max_size = 30 days, 0:00:00 DEBUG Repair groups: DEBUG - 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:45:00+00:00 DEBUG df_block: Open High Low \ 2024-05-09 00:00:00+00:00 61162.906250 61204.425781 61147.816406
2024-05-09 00:05:00+00:00 61224.207031 61224.207031 61186.906250
2024-05-09 00:10:00+00:00 61194.218750 61308.046875 61182.968750
2024-05-09 00:15:00+00:00 61247.027344 61266.984375 61221.957031
2024-05-09 00:20:00+00:00 61208.257812 61288.429688 61208.257812
... ... ... ...
2024-05-09 23:30:00+00:00 62953.804688 62975.207031 62940.707031
2024-05-09 23:35:00+00:00 62979.972656 62979.972656 62883.500000
2024-05-09 23:40:00+00:00 62895.757812 62899.800781 62886.890625
2024-05-09 23:45:00+00:00 62927.777344 62965.730469 62927.777344
2024-05-09 23:50:00+00:00 63046.648438 63147.433594 63046.648438 

                                           Close     Adj Close      Volume  Dividends  \
         2024-05-09 00:00:00+00:00  61204.425781  61204.425781        -1.0        0.0   
         2024-05-09 00:05:00+00:00  61186.906250  61186.906250        -1.0        0.0   
         2024-05-09 00:10:00+00:00  61260.464844  61260.464844   2453504.0        0.0   
         2024-05-09 00:15:00+00:00  61221.957031  61221.957031   2572288.0        0.0   
         2024-05-09 00:20:00+00:00  61288.429688  61288.429688        -1.0        0.0   
         ...                                 ...           ...         ...        ...   
         2024-05-09 23:30:00+00:00  62975.207031  62975.207031        -1.0        0.0   
         2024-05-09 23:35:00+00:00  62883.500000  62883.500000        -1.0        0.0   
         2024-05-09 23:40:00+00:00  62892.656250  62892.656250        -1.0        0.0   
         2024-05-09 23:45:00+00:00  62965.730469  62965.730469        -1.0        0.0   
         2024-05-09 23:50:00+00:00  63147.433594  63147.433594  23889920.0        0.0   

                                    Stock Splits  Repaired?  
         2024-05-09 00:00:00+00:00           0.0      False  
         2024-05-09 00:05:00+00:00           0.0      False  
         2024-05-09 00:10:00+00:00           0.0      False  
         2024-05-09 00:15:00+00:00           0.0      False  
         2024-05-09 00:20:00+00:00           0.0      False  
         ...                                 ...        ...  
         2024-05-09 23:30:00+00:00           0.0      False  
         2024-05-09 23:35:00+00:00           0.0      False  
         2024-05-09 23:40:00+00:00           0.0      False  
         2024-05-09 23:45:00+00:00           0.0      False  
         2024-05-09 23:50:00+00:00           0.0      False  

         [287 rows x 9 columns]

DEBUG Fetching 2m prepost=False 2024-05-08->2024-05-11 DEBUG Entering history() DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-08 00:00:00+00:00', 'period2': '2024-05-11 00:00:00+00:00', 'interval': '2m', 'includePrePost': False, 'events': 'div,splits,capitalGains'} DEBUG BTC-USD: yfinance received OHLC data: 2024-05-08 00:00:00 -> 2024-05-10 23:58:00 DEBUG BTC-USD: OHLC after cleaning: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00 DEBUG BTC-USD: OHLC after combining events: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00 DEBUG BTC-USD: checking OHLC for repairs ... DEBUG Entering _fix_unit_mixups() DEBUG Entering _fix_unit_switch() DEBUG Entering _fix_prices_sudden_change() DEBUG price-repair-split: appears_suspended=True, idx_latest_active=4 (2024-05-10) INFO price-repair-split: No 100x errors detected DEBUG Exiting _fix_prices_sudden_change() DEBUG Exiting _fix_unit_switch() DEBUG Entering _fix_unit_random_mixups() INFO price-repair-100x: No sporadic 100x errors DEBUG Exiting _fix_unit_random_mixups() DEBUG Exiting _fix_unit_mixups() DEBUG Entering _fix_bad_stock_split() DEBUG Exiting _fix_bad_stock_split() DEBUG Entering _fix_zeroes() DEBUG Entering _reconstruct_intervals_batch() DEBUG min_dt=2024-04-17 00:00:00+00:00 interval=2m sub_interval=1m DEBUG grp_max_size = 5 days, 0:00:00 DEBUG Repair groups: DEBUG - 2024-05-08 00:02:00+00:00 -> 2024-05-10 23:58:00+00:00 DEBUG df_block: Open High Low \ 2024-05-08 00:00:00+00:00 62322.082031 62343.535156 62322.082031
2024-05-08 00:02:00+00:00 62367.093750 62412.796875 62367.093750
2024-05-08 00:04:00+00:00 62380.914062 62380.914062 62380.472656
2024-05-08 00:06:00+00:00 62374.738281 62374.738281 62374.738281
2024-05-08 00:08:00+00:00 62438.960938 62457.699219 62438.960938
... ... ... ...
2024-05-10 23:50:00+00:00 60899.343750 60899.343750 60898.917969
2024-05-10 23:52:00+00:00 60903.011719 60903.011719 60898.910156
2024-05-10 23:54:00+00:00 60881.300781 60881.300781 60881.300781
2024-05-10 23:56:00+00:00 60785.386719 60806.562500 60785.386719
2024-05-10 23:58:00+00:00 -1.000000 -1.000000 -1.000000 

                                              Close     Adj Close      Volume  Dividends  \
            2024-05-08 00:00:00+00:00  62343.535156  62343.535156   5607424.0        0.0   
            2024-05-08 00:02:00+00:00  62412.796875  62412.796875        -1.0        0.0   
            2024-05-08 00:04:00+00:00  62380.472656  62380.472656  15757312.0        0.0   
            2024-05-08 00:06:00+00:00  62374.738281  62374.738281         0.0        0.0   
            2024-05-08 00:08:00+00:00  62457.699219  62457.699219  16173056.0        0.0   
            ...                                 ...           ...         ...        ...   
            2024-05-10 23:50:00+00:00  60898.917969  60898.917969        -1.0        0.0   
            2024-05-10 23:52:00+00:00  60898.910156  60898.910156        -1.0        0.0   
            2024-05-10 23:54:00+00:00  60881.300781  60881.300781         0.0        0.0   
            2024-05-10 23:56:00+00:00  60806.562500  60806.562500    673792.0        0.0   
            2024-05-10 23:58:00+00:00     -1.000000     -1.000000        -1.0        0.0   

                                       Stock Splits  Repaired?  
            2024-05-08 00:00:00+00:00           0.0      False  
            2024-05-08 00:02:00+00:00           0.0      False  
            2024-05-08 00:04:00+00:00           0.0      False  
            2024-05-08 00:06:00+00:00           0.0      False  
            2024-05-08 00:08:00+00:00           0.0      False  
            ...                                 ...        ...  
            2024-05-10 23:50:00+00:00           0.0      False  
            2024-05-10 23:52:00+00:00           0.0      False  
            2024-05-10 23:54:00+00:00           0.0      False  
            2024-05-10 23:56:00+00:00           0.0      False  
            2024-05-10 23:58:00+00:00           0.0      False  

            [2160 rows x 9 columns]

DEBUG Fetching 1m prepost=False 2024-05-07->2024-05-13 DEBUG Entering history() DEBUG BTC-USD: Yahoo GET parameters: {'period1': '2024-05-07 00:00:00+00:00', 'period2': '2024-05-13 00:00:00+00:00', 'interval': '1m', 'includePrePost': False, 'events': 'div,splits,capitalGains'} DEBUG BTC-USD: yfinance received OHLC data: 2024-05-07 00:00:00 -> 2024-05-12 23:58:00 DEBUG BTC-USD: OHLC after cleaning: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00 DEBUG BTC-USD: OHLC after combining events: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00 DEBUG BTC-USD: checking OHLC for repairs ... DEBUG Entering _fix_unit_mixups() DEBUG Entering _fix_unit_switch() DEBUG Entering _fix_prices_sudden_change() DEBUG price-repair-split: appears_suspended=False, idx_latest_active=1 (2024-05-12) INFO price-repair-split: No 100x errors detected DEBUG Exiting _fix_prices_sudden_change() DEBUG Exiting _fix_unit_switch() DEBUG Entering _fix_unit_random_mixups() INFO price-repair-100x: No sporadic 100x errors DEBUG Exiting _fix_unit_random_mixups() DEBUG Exiting _fix_unit_mixups() DEBUG Entering _fix_bad_stock_split() DEBUG Exiting _fix_bad_stock_split() DEBUG Entering _fix_zeroes() DEBUG Entering _reconstruct_intervals_batch() DEBUG Exiting _reconstruct_intervals_batch() DEBUG Exiting _fix_zeroes() DEBUG Entering _fix_missing_div_adjust() DEBUG Exiting _fix_missing_div_adjust() DEBUG BTC-USD: yfinance returning OHLC: 2024-05-07 00:00:00+00:00 -> 2024-05-12 23:58:00+00:00 DEBUG Exiting history() DEBUG df_new: Open Close Adj Close \ 2024-05-08 00:00:00+00:00 62322.082031 62343.535156 62343.535156
2024-05-08 00:02:00+00:00 62367.093750 62412.796875 62412.796875
2024-05-08 00:04:00+00:00 62380.914062 62380.472656 62380.472656
2024-05-08 00:06:00+00:00 62374.738281 62374.738281 62374.738281
2024-05-08 00:08:00+00:00 62438.960938 62457.699219 62457.699219
... ... ... ...
2024-05-10 23:50:00+00:00 60899.343750 60898.917969 60898.917969
2024-05-10 23:52:00+00:00 60903.011719 60898.910156 60898.910156
2024-05-10 23:54:00+00:00 60881.300781 60881.300781 60881.300781
2024-05-10 23:56:00+00:00 60785.386719 60806.562500 60806.562500
2024-05-10 23:58:00+00:00 NaN NaN NaN 

                                                Low          High  Dividends    Volume  
            2024-05-08 00:00:00+00:00  62322.082031  62343.535156        0.0   6049792  
            2024-05-08 00:02:00+00:00  62367.093750  62412.796875        0.0         0  
            2024-05-08 00:04:00+00:00  62380.472656  62380.914062        0.0  15757312  
            2024-05-08 00:06:00+00:00  62374.738281  62374.738281        0.0         0  
            2024-05-08 00:08:00+00:00  62438.960938  62457.699219        0.0  16173056  
            ...                                 ...           ...        ...       ...  
            2024-05-10 23:50:00+00:00  60898.917969  60899.343750        0.0         0  
            2024-05-10 23:52:00+00:00  60898.910156  60903.011719        0.0         0  
            2024-05-10 23:54:00+00:00  60881.300781  60881.300781        0.0         0  
            2024-05-10 23:56:00+00:00  60785.386719  60806.562500        0.0    673792  
            2024-05-10 23:58:00+00:00           NaN           NaN        0.0         0  

            [2160 rows x 7 columns]

DEBUG Price calibration ratio (raw) = 1.000000 DEBUG Exiting _reconstruct_intervals_batch() INFO price-repair-missing: BTC-USD: fixed 616/616 value=0 errors in 2m price data DEBUG Exiting _fix_zeroes() DEBUG Entering _fix_missing_div_adjust() DEBUG Exiting _fix_missing_div_adjust() DEBUG BTC-USD: yfinance returning OHLC: 2024-05-08 00:00:00+00:00 -> 2024-05-10 23:58:00+00:00 DEBUG Exiting history() DEBUG df_new: Open Close Adj Close \ 2024-05-09 00:00:00+00:00 61162.906250 61186.906250 61186.906250
2024-05-09 00:10:00+00:00 61194.218750 61221.957031 61221.957031
2024-05-09 00:20:00+00:00 61208.257812 61354.125000 61354.125000
2024-05-09 00:30:00+00:00 61343.410156 61353.480469 61353.480469
2024-05-09 00:40:00+00:00 61357.382812 61403.101562 61403.101562
... ... ... ...
2024-05-09 23:10:00+00:00 63128.957031 62976.238281 62976.238281
2024-05-09 23:20:00+00:00 62987.125000 62955.250000 62955.250000
2024-05-09 23:30:00+00:00 62953.804688 62883.500000 62883.500000
2024-05-09 23:40:00+00:00 62895.757812 62965.730469 62965.730469
2024-05-09 23:50:00+00:00 63046.648438 63098.777344 63098.777344 

                                             Low          High  Dividends    Volume  
         2024-05-09 00:00:00+00:00  61147.816406  61224.207031        0.0   4972544  
         2024-05-09 00:10:00+00:00  61182.968750  61308.046875        0.0  17688576  
         2024-05-09 00:20:00+00:00  61208.257812  61354.125000        0.0   4491264  
         2024-05-09 00:30:00+00:00  61304.511719  61361.765625        0.0   4648960  
         2024-05-09 00:40:00+00:00  61357.382812  61435.660156        0.0         0  
         ...                                 ...           ...        ...       ...  
         2024-05-09 23:10:00+00:00  62976.238281  63128.957031        0.0   8067072  
         2024-05-09 23:20:00+00:00  62902.449219  62987.125000        0.0  33947648  
         2024-05-09 23:30:00+00:00  62883.500000  62979.972656        0.0    833536  
         2024-05-09 23:40:00+00:00  62886.890625  62965.730469        0.0    301056  
         2024-05-09 23:50:00+00:00  63046.648438  63147.433594        0.0  38248448  

         [144 rows x 7 columns]

DEBUG Price calibration ratio (raw) = 0.999958 DEBUG Yahoo didn't return finer-grain data for these intervals: [Timestamp('2024-05-09 00:05:00+0000', tz='UTC'), Timestamp('2024-05-09 00:25:00+0000', tz='UTC'), Timestamp('2024-05-09 00:35:00+0000', tz='UTC'), Timestamp('2024-05-09 00:45:00+0000', tz='UTC'), Timestamp('2024-05-09 00:55:00+0000', tz='UTC'), Timestamp('2024-05-09 01:05:00+0000', tz='UTC'), Timestamp('2024-05-09 01:15:00+0000', tz='UTC'), Timestamp('2024-05-09 01:25:00+0000', tz='UTC'), Timestamp('2024-05-09 01:35:00+0000', tz='UTC'), Timestamp('2024-05-09 01:45:00+0000', tz='UTC'), Timestamp('2024-05-09 02:05:00+0000', tz='UTC'), Timestamp('2024-05-09 02:15:00+0000', tz='UTC'), Timestamp('2024-05-09 02:25:00+0000', tz='UTC'), Timestamp('2024-05-09 02:55:00+0000', tz='UTC'), Timestamp('2024-05-09 03:15:00+0000', tz='UTC'), Timestamp('2024-05-09 03:45:00+0000', tz='UTC'), Timestamp('2024-05-09 04:05:00+0000', tz='UTC'), Timestamp('2024-05-09 04:25:00+0000', tz='UTC'), Timestamp('2024-05-09 04:35:00+0000', tz='UTC'), Timestamp('2024-05-09 04:45:00+0000', tz='UTC'), Timestamp('2024-05-09 04:55:00+0000', tz='UTC'), Timestamp('2024-05-09 05:05:00+0000', tz='UTC'), Timestamp('2024-05-09 05:15:00+0000', tz='UTC'), Timestamp('2024-05-09 05:55:00+0000', tz='UTC'), Timestamp('2024-05-09 06:05:00+0000', tz='UTC'), Timestamp('2024-05-09 06:25:00+0000', tz='UTC'), Timestamp('2024-05-09 06:35:00+0000', tz='UTC'), Timestamp('2024-05-09 06:45:00+0000', tz='UTC'), Timestamp('2024-05-09 06:55:00+0000', tz='UTC'), Timestamp('2024-05-09 07:05:00+0000', tz='UTC'), Timestamp('2024-05-09 07:15:00+0000', tz='UTC'), Timestamp('2024-05-09 07:25:00+0000', tz='UTC'), Timestamp('2024-05-09 07:35:00+0000', tz='UTC'), Timestamp('2024-05-09 07:55:00+0000', tz='UTC'), Timestamp('2024-05-09 08:05:00+0000', tz='UTC'), Timestamp('2024-05-09 08:15:00+0000', tz='UTC'), Timestamp('2024-05-09 08:35:00+0000', tz='UTC'), Timestamp('2024-05-09 08:45:00+0000', tz='UTC'), Timestamp('2024-05-09 08:55:00+0000', tz='UTC'), Timestamp('2024-05-09 09:15:00+0000', tz='UTC'), Timestamp('2024-05-09 09:25:00+0000', tz='UTC'), Timestamp('2024-05-09 09:35:00+0000', tz='UTC'), Timestamp('2024-05-09 09:55:00+0000', tz='UTC'), Timestamp('2024-05-09 10:15:00+0000', tz='UTC'), Timestamp('2024-05-09 10:25:00+0000', tz='UTC'), Timestamp('2024-05-09 10:45:00+0000', tz='UTC'), Timestamp('2024-05-09 12:25:00+0000', tz='UTC'), Timestamp('2024-05-09 13:05:00+0000', tz='UTC'), Timestamp('2024-05-09 13:15:00+0000', tz='UTC'), Timestamp('2024-05-09 13:25:00+0000', tz='UTC'), Timestamp('2024-05-09 13:35:00+0000', tz='UTC'), Timestamp('2024-05-09 13:55:00+0000', tz='UTC'), Timestamp('2024-05-09 14:25:00+0000', tz='UTC'), Timestamp('2024-05-09 14:45:00+0000', tz='UTC'), Timestamp('2024-05-09 14:55:00+0000', tz='UTC'), Timestamp('2024-05-09 15:15:00+0000', tz='UTC'), Timestamp('2024-05-09 17:45:00+0000', tz='UTC'), Timestamp('2024-05-09 18:15:00+0000', tz='UTC'), Timestamp('2024-05-09 18:25:00+0000', tz='UTC'), Timestamp('2024-05-09 18:35:00+0000', tz='UTC'), Timestamp('2024-05-09 18:45:00+0000', tz='UTC'), Timestamp('2024-05-09 19:05:00+0000', tz='UTC'), Timestamp('2024-05-09 19:15:00+0000', tz='UTC'), Timestamp('2024-05-09 19:25:00+0000', tz='UTC'), Timestamp('2024-05-09 19:45:00+0000', tz='UTC'), Timestamp('2024-05-09 20:15:00+0000', tz='UTC'), Timestamp('2024-05-09 20:25:00+0000', tz='UTC'), Timestamp('2024-05-09 20:35:00+0000', tz='UTC'), Timestamp('2024-05-09 20:45:00+0000', tz='UTC'), Timestamp('2024-05-09 20:55:00+0000', tz='UTC'), Timestamp('2024-05-09 21:25:00+0000', tz='UTC'), Timestamp('2024-05-09 21:35:00+0000', tz='UTC'), Timestamp('2024-05-09 22:15:00+0000', tz='UTC'), Timestamp('2024-05-09 22:25:00+0000', tz='UTC'), Timestamp('2024-05-09 23:05:00+0000', tz='UTC'), Timestamp('2024-05-09 23:15:00+0000', tz='UTC'), Timestamp('2024-05-09 23:25:00+0000', tz='UTC'), Timestamp('2024-05-09 23:35:00+0000', tz='UTC'), Timestamp('2024-05-09 23:45:00+0000', tz='UTC')] DEBUG Exiting _reconstruct_intervals_batch() INFO price-repair-missing: BTC-USD: fixed 86/165 value=0 errors in 5m price data DEBUG Exiting _fix_zeroes() DEBUG Entering _fix_missing_div_adjust() DEBUG Exiting _fix_missing_div_adjust() DEBUG BTC-USD: yfinance returning OHLC: 2024-05-09 00:00:00+00:00 -> 2024-05-09 23:55:00+00:00 DEBUG Exiting history() DEBUG Exiting history()

Bad data proof

imagen

yfinance version

0.2.38

Python version

3.12.3

Operating system

Windows 11

jvallsgm commented 5 months ago

I did not explain myself well when I referred to negative values. Ignore this part. The problem is the many zeros that appear, and they are incorrect. Thanks