SimFin / simfin

Simple financial data for Python
https://simfin.com/
Other
298 stars 39 forks source link

Bug Report #26

Closed XJTLUmedia closed 6 months ago

XJTLUmedia commented 6 months ago

Bug Report

If you experience errors or bugs, please make a serious effort to solve the problem yourself before asking here. The problem is quite likely in your own code and a simple Google search for the error-message may help you solve it. If it is a problem directly related to simfin, then please search the closed GitHub issues, because it may already have been answered there.

Please make sure you have the latest simfin package installed by running:

pip install --upgrade simfin

And make sure you have downloaded fresh data-files from the SimFin server by setting refresh_days=0 (see example below).

If you still cannot solve the problem and need our help, then please provide the following.

Description

Please write a brief description of the bug / error you have experienced.

System Details

Code Example

Please write a minimal source-code example that reproduces the problem. You can indent the code-block to get proper code-formatting, for example:

import simfin as sf

# Print SimFin version.
print(sf.__version__)

# Configure simfin.
sf.set_data_dir('~/simfin_data/')
sf.load_api_key(path='~/simfin_api_key.txt', default_key='free')
#  Calculate Growth signals
df_growth_signals = sf.growth_signals(df_prices=df_prices,
                      df_income_ttm=df_income_ttm,
                      df_income_qrt=df_income_qrt,
                      df_balance_ttm=df_balance_ttm,
                      df_balance_qrt=df_balance_qrt,
                      df_cashflow_ttm=df_cashflow_ttm,
                      df_cashflow_qrt=df_cashflow_qrt,
                      fill_method='ffill')

print(df_growth_signals.info())
#  Calculate value signals
df_value_signals = sf.val_signals(df_prices=df_prices,
                                df_income_ttm=df_income_ttm,
                                df_balance_ttm=df_balance_ttm,
                                df_cashflow_ttm=df_cashflow_ttm,
                                fill_method='ffill')
print(df_value_signals.info())
#  Calculate financial signals
df_financial_signals = sf.fin_signals(df_prices=df_prices,
                                df_income_ttm=df_income_ttm,
                                df_balance_ttm=df_balance_ttm,
                                df_cashflow_ttm=df_cashflow_ttm,
                                fill_method='ffill')
print(df_financial_signals.info())

Result / Error

Please write the full output and error message you received. You can also indent this text-block by pressing the tab-key to get proper formatting of the error-message.

InvalidIndexError                         Traceback (most recent call last)
Cell In[12], [line 13](vscode-notebook-cell:?execution_count=12&line=13)
      [1](vscode-notebook-cell:?execution_count=12&line=1) #  Calculate Growth signals
      [2](vscode-notebook-cell:?execution_count=12&line=2) # df_growth_signals = sf.growth_signals(df_prices=df_prices,
      [3](vscode-notebook-cell:?execution_count=12&line=3) #                       df_income_ttm=df_income_ttm,
   (...)
     [11](vscode-notebook-cell:?execution_count=12&line=11) # print(df_growth_signals.info())
     [12](vscode-notebook-cell:?execution_count=12&line=12) #  Calculate value signals
---> [13](vscode-notebook-cell:?execution_count=12&line=13) df_value_signals = sf.val_signals(df_prices=df_prices,
     [14](vscode-notebook-cell:?execution_count=12&line=14)                                 df_income_ttm=df_income_ttm,
     [15](vscode-notebook-cell:?execution_count=12&line=15)                                 df_balance_ttm=df_balance_ttm,
     [16](vscode-notebook-cell:?execution_count=12&line=16)                                 df_cashflow_ttm=df_cashflow_ttm,
     [17](vscode-notebook-cell:?execution_count=12&line=17)                                 fill_method='ffill')
     [18](vscode-notebook-cell:?execution_count=12&line=18) print(df_value_signals.info())
     [19](vscode-notebook-cell:?execution_count=12&line=19) #  Calculate financial signals

File [~/.local/lib/python3.12/site-packages/simfin/cache.py:201](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:201), in cache.<locals>.wrapper(cache_name, cache_refresh, cache_format, **kwargs)
    [146](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:146) """
    [147](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:147) This uses more advanced Python features to wrap `func` using a
    [148](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:148) function-decorator, which are not explained so well in the
   (...)
    [196](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:196)     or the results from computing the wrapped function.
    [197](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:197) """
    [199](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:199) if cache_refresh is None:
    [200](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:200)     # Never use the cache, always just compute the function.
--> [201](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:201)     df_result = func(**kwargs)
    [202](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:202) else:
    [203](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:203)     # We want to use a cache-file. Determine if it should be refreshed.
    [204](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:204) 
    [205](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:205)     # Ensure the cache-format is a valid string.
    [206](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:206)     assert cache_format.startswith('pickle') or \
    [207](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/cache.py:207)            cache_format in ['parquet', 'feather']

File [~/.local/lib/python3.12/site-packages/simfin/signals.py:793](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:793), in val_signals(df_prices, df_income_ttm, df_balance_ttm, df_cashflow_ttm, fill_method, offset, func, date_index, shares_index, group_index, banks, insurance)
    [790](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:790) df_cf = df_cashflow_ttm[columns]
    [792](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:792) # Combine all the data. This creates a new copy that we can add columns to.
--> [793](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:793) df = pd.concat([df_inc, df_bal, df_cf], axis=1)
    [795](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:795) # Calculate derived financial data such as Free Cash Flow (FCF),
    [796](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:796) # and add it as new columns to the DataFrame.
    [797](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:797) # This is only TTM data with 4 data-points per year, so it is
    [798](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:798) # faster than calculating it for the daily data-points below.
    [799](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/simfin/signals.py:799) df[FCF] = free_cash_flow(df_cashflow_ttm)

File [~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:393](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:393), in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
    [378](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:378)     copy = False
    [380](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:380) op = _Concatenator(
    [381](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:381)     objs,
    [382](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:382)     axis=axis,
   (...)
    [390](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:390)     sort=sort,
    [391](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:391) )
--> [393](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:393) return op.get_result()

File [~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:676](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:676), in _Concatenator.get_result(self)
    [674](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:674)         obj_labels = obj.axes[1 - ax]
    [675](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:675)         if not new_labels.equals(obj_labels):
--> [676](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:676)             indexers[ax] = obj_labels.get_indexer(new_labels)
    [678](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:678)     mgrs_indexers.append((obj._mgr, indexers))
    [680](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:680) new_data = concatenate_managers(
    [681](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:681)     mgrs_indexers, self.new_axes, concat_axis=self.bm_axis, copy=self.copy
    [682](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/reshape/concat.py:682) )

File [~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3875](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3875), in Index.get_indexer(self, target, method, limit, tolerance)
   [3872](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3872) self._check_indexing_method(method, limit, tolerance)
   [3874](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3874) if not self._index_as_unique:
-> [3875](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3875)     raise InvalidIndexError(self._requires_unique_msg)
   [3877](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3877) if len(target) == 0:
   [3878](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/futuresight/python-projects/~/.local/lib/python3.12/site-packages/pandas/core/indexes/base.py:3878)     return np.array([], dtype=np.intp)

InvalidIndexError: Reindexing only valid with uniquely valued Index objects

it is the same for all three functions. Please take a look

thf24 commented 6 months ago

Thanks, looks like some data problem. Investigating right now and should be fixed in the next bulk iteration by tomorrow morning.

thf24 commented 6 months ago

We fixed the problems now, bulk files should reflect this by tomorrow.

thf24 commented 6 months ago

seems like there is another issue, we'll recreate the bulk files asap once we fixed it, I'll update you here

thf24 commented 6 months ago

we fixed the problem now, should work with the new bulk files by tomorrow.

thf24 commented 6 months ago

should work again now, introduced some checks also so that this problem should not occur again (some data problem for a handful of companies).