ZhengyaoJiang / PGPortfolio

PGPortfolio: Policy Gradient Portfolio, the source code of "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem"(https://arxiv.org/pdf/1706.10059.pdf).
GNU General Public License v3.0
1.73k stars 748 forks source link

KeyError: 'BTS_BTC' #145

Open dalle244 opened 2 years ago

dalle244 commented 2 years ago

Hi,

I've tried several configurations of my anaconda environment, at first, I managed to make the python main.py --mode=download_data part work, but then I ran into the issues with the update of pandas mentioned in other issues. Trying to fix that I cannot come back to my initial progress even though I've made a new environment and forked the repo once again.

The error I get is:

Traceback (most recent call last): File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\main.py", line 132, in main() File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\main.py", line 71, in main DataMatrices(start=start, File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\datamatrices.py", line 44, in init self.history_manager = gdm.HistoryManager(coin_number=coin_filter, end=self.end, File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\globaldatamatrix.py", line 24, in init__ self._coin_list = CoinList(end, volume_average_days, volume_forward) File "C:\Users\Alexander.S.Dahlberg\source\repos\PGPortfolio\pgportfolio\marketdata\coinlist.py", line 35, in init__ prices.append(1.0 / float(ticker[k]['last'])) KeyError: 'BTS_BTC'

I've no clue how to solve this. Have any others experienced the issue?

Thanks

tiassap commented 1 year ago

Hello, I am also facing this error. Seems like the error comes out when using class CoinList(), there is a mismatch between dict key from self._polo.marketVolume() and self._polo.marketTicker()

  File "/home/tias/Data_science/1_project/zhengyaojiang_pgportfolio_dev/pgportfolio/marketdata/coinlist.py", line 36, in __init__
    prices.append(1.0 / float(ticker[k]['last']))
KeyError: 'BTS_BTC'

https://github.com/ZhengyaoJiang/PGPortfolio/blob/48cc5a4af5edefd298e7801b95b0d4696f5175dd/pgportfolio/marketdata/coinlist.py#L35

dekrypting-karma commented 1 year ago

@tiassap did you solve the error?

tiassap commented 1 year ago

@dekrypting-karma

I could avoid KeyError: 'BTS_BTC' by replacing the for-loop inside CoinList.__init__() (inside coinlist.py) with this

for k in ticker.keys():
            if k.startswith("BTC_") or k.endswith("_BTC"):
                pairs.append(k)
                for c in k.split("_"):
                    if c != 'BTC':
                        if k.endswith('_BTC'):
                            coins.append('reversed_' + c)
                            prices.append(1.0 / float(ticker[k]['last']))
                        else:
                            coins.append(c)
                            prices.append(float(ticker[k]['last']))
                    else:
                        volumes.append(self.__get_total_volume(pair=k, global_end=end,
                                                               days=volume_average_days,
                                                               forward=volume_forward))

But there are still many errors will come out related to sql command line parsing and also error because in HistoryManager.get_global_panel() pandas Panel is used (deprecated since pandas version 0.20.0)

dekrypting-karma commented 1 year ago

@tiassap this helped in removing KeyError: 'BTS_BTC'. But I am also facing multiple errors in HistoryManager.get_global_panel().

Were you able to successfully run this project? If yes can you please share the code? Any sort of help is appreciated.

chenjjcccc commented 1 year ago

@dekrypting-karma

I could avoid KeyError: 'BTS_BTC' by replacing the for-loop inside CoinList.__init__() (inside coinlist.py) with this

for k in ticker.keys():
            if k.startswith("BTC_") or k.endswith("_BTC"):
                pairs.append(k)
                for c in k.split("_"):
                    if c != 'BTC':
                        if k.endswith('_BTC'):
                            coins.append('reversed_' + c)
                            prices.append(1.0 / float(ticker[k]['last']))
                        else:
                            coins.append(c)
                            prices.append(float(ticker[k]['last']))
                    else:
                        volumes.append(self.__get_total_volume(pair=k, global_end=end,
                                                               days=volume_average_days,
                                                               forward=volume_forward))

But there are still many errors will come out related to sql command line parsing and also error because in HistoryManager.get_global_panel() pandas Panel is used (deprecated since pandas version 0.20.0)

did you solve the error?

lucacarniato commented 1 year ago

you could use xarray.DataArray

https://docs.xarray.dev/en/stable/generated/xarray.DataArray.html

tiassap commented 1 year ago

@dekrypting-karma , @lucacarniato

Sorry I was unable to response for a long time. I created Docker container to make environment so we can use old pandas version and old tensorflow module here: https://github.com/tiassap/PGPortfolio

Seems like there is still problem with Binance API.. Please try it and help me make the code works again.