ValueError: If using all scalar values, you must pass an index using binance processor

[Daiiszuki]

I'm using the meta Dataprocessor

import meta
from meta.data_processor import DataProcessor

When I run the following :

#Set constants
#
LIST_OF_SYMBOLS = ['ADAUSDT' ,'ATOMUSDT' ,'BNBUSDT', 'BTCUSDT' ,'DOTUSDT' ,'ETCUSDT', 'ETHUSDT','LINKUSDT', 'LTCUSDT' ,'SOLUSDT' ,'XMRUSDT' ,'XRPUSDT']
#Set time interval
TIME_INTERVAL = '1D'

#Training start
START_TRAIN =  '2018-01-1'

#Training end
END_TRAIN = '2020-12-1'

#Trading start
START_TRADE = '2020-12-1'

#Trading end
END_TRADE = '2022-06-01'

#List of technical indicators
TECHNICAL_INDICATORS = ['rsi',
                        'cci',
                        'macd',
                        'macd_signal',
                        'macd_hist',
                        'dx'
                             ]  

if_vix = False

processorObj = DataProcessor(data_source = 'binance', start_date= START_TRAIN, end_date =END_TRAIN, time_interval=TIME_INTERVAL) 
processorObj.download_data(LIST_OF_SYMBOLS)
processorObj.clean_data()
processorObj.add_technical_indicator(TECHNICAL_INDICATORS)
frame = processorObj.dataframe

frame.head() 

I get :

ValueError                                Traceback (most recent call last)
[<ipython-input-26-256be0b7d088>](https://localhost:8080/#) in <module>
     30 
     31 processorObj = DataProcessor(data_source = 'binance', start_date= START_TRAIN, end_date =END_TRAIN, time_interval=TIME_INTERVAL)
---> 32 processorObj.download_data(LIST_OF_SYMBOLS)
     33 processorObj.clean_data()
     34 processorObj.add_technical_indicator(TECHNICAL_INDICATORS)

7 frames
[/FinRL-Meta/meta/data_processor.py](https://localhost:8080/#) in download_data(self, ticker_list)
     84 
     85     def download_data(self, ticker_list):
---> 86         self.processor.download_data(ticker_list=ticker_list)
     87         self.dataframe = self.processor.dataframe
     88 

[/FinRL-Meta/meta/data_processors/binance.py](https://localhost:8080/#) in download_data(self, ticker_list)
     53             final_df = pd.DataFrame()
     54             for i in ticker_list:
---> 55                 hist_data = self.dataframe_with_limit(symbol=i)
     56                 df = hist_data.iloc[:-1].dropna()
     57                 df["tic"] = i

[/FinRL-Meta/meta/data_processors/binance.py](https://localhost:8080/#) in dataframe_with_limit(self, symbol)
    183         while True:
    184 
--> 185             new_df = self.get_binance_bars(last_datetime, symbol)
    186             if new_df is None:
    187                 break

[/FinRL-Meta/meta/data_processors/binance.py](https://localhost:8080/#) in get_binance_bars(self, last_datetime, symbol)
    121           #r = requests.get(self.url, params=req_params)
    122           #print(r.text)
--> 123 
    124 
    125         df = pd.DataFrame(requests.get(self.url, params=req_params).json())        if df.empty:

[/usr/local/lib/python3.7/dist-packages/pandas/core/frame.py](https://localhost:8080/#) in __init__(self, data, index, columns, dtype, copy)
    612         elif isinstance(data, dict):
    613             # GH#38939 de facto copy defaults to False only in non-dict cases
--> 614             mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
    615         elif isinstance(data, ma.MaskedArray):
    616             import numpy.ma.mrecords as mrecords

[/usr/local/lib/python3.7/dist-packages/pandas/core/internals/construction.py](https://localhost:8080/#) in dict_to_mgr(data, index, columns, dtype, typ, copy)
    463 
    464     return arrays_to_mgr(
--> 465         arrays, data_names, index, columns, dtype=dtype, typ=typ, consolidate=copy
    466     )
    467 

[/usr/local/lib/python3.7/dist-packages/pandas/core/internals/construction.py](https://localhost:8080/#) in arrays_to_mgr(arrays, arr_names, index, columns, dtype, verify_integrity, typ, consolidate)
    117         # figure out the index, if necessary
    118         if index is None:
--> 119             index = _extract_index(arrays)
    120         else:
    121             index = ensure_index(index)

[/usr/local/lib/python3.7/dist-packages/pandas/core/internals/construction.py](https://localhost:8080/#) in _extract_index(data)
    623 
    624         if not indexes and not raw_lengths:
--> 625             raise ValueError("If using all scalar values, you must pass an index")
    626 
    627         if have_series:

ValueError: If using all scalar values, you must pass an index

Even when I try defining the list as ['ADAUSDT'] ,I still get the error

@zhumingpassional @BruceYanghy @XiaoYangLiu-FinRL

Please assist

[Daiiszuki]

Strange because the exact same code ran without errors previously.

[Daiiszuki]

Using the cctx downloader I get

ValueError: time data '2018-01-1' does not match format '%Y%m%d %H:%M:%S'

Then tried changing args

START_TRAIN =  '2018-01-01 00:00:00'

#Training end
END_TRAIN = '2020-12-01 00:00:00'

#Trading start
START_TRADE = '2020-12-01 00:00:00'

#Trading end
END_TRADE = '2022-06-01 00:00:00'

Output: ValueError: time data '2018-01-1 0:0:0' does not match format '%Y%m%d %H:%M:%S'

[zhumingpassional]

pls use the format of date: '2018-01-01' and use the format of time interval '1d' instead of '1D'. then it works.

[Daiiszuki]

Thank you. rookie mistake

By the way, is there a finRL discord or telegram?

2.On the list of usable indicators, or rather how to use them. e,g those that require ohlc

[ZiyiXia]

We do have a discord channel! Please use the following link: https://discord.gg/CcDgRupu By the way, there are more people in our slack channel and welcome to join us: https://join.slack.com/t/ai4financeworkspace/shared_invite/zt-1h2vj05oy-oVNrrbOabXmfR6Ke_FCxGg

[Daiiszuki]

@ZiyiXia any suggestions for this;

ModuleNotFoundError                       Traceback (most recent call last)
[<ipython-input-11-91a7710ee446>](https://localhost:8080/#) in <module>
     41                              ]  
     42 
---> 43 processorObj = DataProcessor(data_source = 'binance', start_date= START_TRAIN, end_date =END_TRADE, time_interval=TIME_INTERVAL   )
     44 processorObj.download_data(LIST_OF_SYMBOLS)
     45 processorObj.clean_data()

1 frames
[/FinRL-Meta/meta/data_processor.py](https://localhost:8080/#) in __init__(self, data_source, start_date, end_date, time_interval, **kwargs)
     38 
     39         elif self.data_source == "binance":
---> 40             from meta.data_processors.binance import Binance
     41 
     42             processor_dict = {self.data_source: Binance}

[/FinRL-Meta/meta/data_processors/binance.py](https://localhost:8080/#) in <module>
     10 import pandas as pd
     11 import requests
---> 12 from _base import check_date
     13 
     14 from meta.config import BINANCE_BASE_URL

ModuleNotFoundError: No module named '_base'

It was alright just yesterday

[Daiiszuki]

Running on colab. What are some potential causes?fixes?

[zhumingpassional]

@Daiiszuki pls test it again. we updated the code

[Daiiszuki]

---

AttributeError Traceback (most recent call last) in 5 import meta 6 #from finrl.agents.stablebaselines3.models import DRLAgent as cryptoTrader ----> 7 from agents.stablebaselines3_models import DRLAgent as cryptoTrader 8 from meta.data_processor import DataProcessor 9 #from re import LOCALE

6 frames /usr/local/lib/python3.7/dist-packages/gym/envs/registration.py in load_env_plugins(entry_point) 248 def load_env_plugins(entry_point="gym.envs"): 249 # Load third-party environments --> 250 for plugin in metadata.entry_points().get(entry_point, []): 251 # Python 3.8 doesn't support plugin.module, plugin.attr 252 # So we'll have to try and parse this ourselves

AttributeError: 'EntryPoints' object has no attribute 'get'

[Daiiszuki]

Before I commented outfrom finrl.meta.preprocessor.preprocessors import FeatureEngineer, data_split

---

AttributeError Traceback (most recent call last) in 1 from finrl.meta.data_processors.processor_yahoofinance import YahooFinanceProcessor 2 ----> 3 from finrl.meta.preprocessor.preprocessors import FeatureEngineer, data_split 4 5

6 frames /usr/local/lib/python3.7/dist-packages/gym/envs/registration.py in load_env_plugins(entry_point) 248 def load_env_plugins(entry_point="gym.envs"): 249 # Load third-party environments --> 250 for plugin in metadata.entry_points().get(entry_point, []): 251 # Python 3.8 doesn't support plugin.module, plugin.attr 252 # So we'll have to try and parse this ourselves

[Daiiszuki]

Imports:

from finrl.meta.data_processors.processor_yahoofinance import YahooFinanceProcessor

from finrl.meta.preprocessor.preprocessors import FeatureEngineer, data_split

from finrl.agents.stablebaselines3.models import DRLAgent as cryptoTrader from meta.data_processor import DataProcessor from re import LOCALE import gym from gym.utils import seeding

from gym import spaces import pandas as panda import matplotlib.pyplot as plt import numpy as np

from datetime import datetime

from stable_baselines3.common.vec_env import DummyVecEnv

from finrl.meta.preprocessor.preprocessors import data_split ,FeatureEngineer

from finrl.agents.stablebaselines3.models import DRLAgent as cryptoTrader

[Daiiszuki]

Strangely import gymreturns the same error any suggestions?

[Daiiszuki]

import gymruns with no issue in a fresh notebook

upon running

!pip install git+https://github.com/AI4Finance-LLC/FinRL-Library.git

The error appears.

[Daiiszuki]

Any recommendations?

I based an entire project fin-rl. Not much time to pivot