search

autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 268 forks source link

TypeError: ("cannot convert the series to <class 'float'>", 'occurred at index loss') #299

Closed RahulKishor closed 5 years ago

RahulKishor commented 5 years ago

Thanks so much for coming here to raise an issue. Please take a moment to 'check' the below boxes:

If you still have an error, please submit complete trace and a code with:

You can provide the code in pastebin / gist or any other format you like.

RahulKishor commented 5 years ago

I am getting the following error while running scan() 

TypeError                                 Traceback (most recent call last)
<ipython-input-48-a383f58d1676> in <module>()
     16 x=X
     17 y=y
---> 18 h = ta.Scan(x,y,params=p,model=create_model)
     19 #create_model()

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\scan\Scan.py in __init__(self, x, y, params, model, dataset_name, experiment_no, x_val, y_val, val_split, shuffle, round_limit, grid_downsample, random_method, seed, search_method, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, reduce_loss, last_epoch_value, clear_tf_session, disable_progress_bar, print_params, debug)
    168         # input parameters section ends
    169 
--> 170         self._null = self.runtime()
    171 
    172     def runtime(self):

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\scan\Scan.py in runtime(self)
    173 
    174         self = scan_prepare(self)
--> 175         self = scan_run(self)

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\scan\scan_run.py in scan_run(self)
     24     self.peak_epochs_df = peak_epochs_todf(self)
     25 
---> 26     self = scan_finish(self)

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\scan\scan_finish.py in scan_finish(self)
     70 
     71     # convert to numeric
---> 72         print(self.data)
     73     self.data = string_cols_to_numeric(self.data)
     74 

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\utils\string_cols_to_numeric.py in string_cols_to_numeric(data, destructive)
     22     for col in data.columns:
     23 
---> 24         if data[col].apply(isnumber).sum() == len(data):
     25             try:
     26                 data[col] = data[col].astype(int)

C:\Users\I061652\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\pandas\core\frame.py in apply(self, func, axis, broadcast, raw, reduce, args, **kwds)
   4260                         f, axis,
   4261                         reduce=reduce,
-> 4262                         ignore_failures=ignore_failures)
   4263             else:
   4264                 return self._apply_broadcast(f, axis)

C:\Users\I061652\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\pandas\core\frame.py in _apply_standard(self, func, axis, ignore_failures, reduce)
   4356             try:
   4357                 for i, v in enumerate(series_gen):
-> 4358                     results[i] = func(v)
   4359                     keys.append(v.name)
   4360             except Exception as e:

C:\Users\I061652\AppData\Roaming\Python\Python36\site-packages\talos\utils\string_cols_to_numeric.py in isnumber(value)
      6 
      7     try:
----> 8         float(value)
      9         return True
     10     except ValueError:

C:\Users\I061652\AppData\Local\conda\conda\envs\tensorflow\lib\site-packages\pandas\core\series.py in wrapper(self)
     95             return converter(self.iloc[0])
     96         raise TypeError("cannot convert the series to "
---> 97                         "{0}".format(str(converter)))
     98 
     99     return wrapper

TypeError: ("cannot convert the series to <class 'float'>", 'occurred at index loss')

My code is :


def create_model(x_train,x_test,y_train,y_test,params):
#def create_model():
    model = Sequential()
    model.add(LSTM(units=30, return_sequences= True, input_shape=(X.shape[1],5)))
    model.add(LSTM(units=30, return_sequences=True))
    model.add(LSTM(units=30))
    model.add(Dense(units=1))
    model.summary()
    model.compile(optimizer=params['optimizer'](), loss=params['loss'],metrics=['acc'])
    #model.compile(optimizer='Adam', loss='binary_crossentropy',metrics=['acc'])
    #out = model.fit(X,y,batch_size=100,epochs=4)
    out = model.fit(x,y,batch_size=params['batch_size'],epochs=params['epochs'])
    #out = model.fit(x_train,y_train,batch_size=params['batch_size'],epochs=params['epochs'],validation_data=[x_test,y_test])
    #out.history['loss']= pd.to_numeric(out.history['loss'])
    print('------------out----------------')
    print(out.history['loss'])
    print(out.history['acc'])
    print('------------model--------------')
    print(model)

    return out, model

p = {'batch_size': [500],
     'epochs': [2,4],
     'optimizer': [Adam],
     'loss': ['binary_crossentropy']}

#parallel_gpu_jobs(0.5)
x=X
y=y
h = ta.Scan(x,y,params=p,model=create_model)

The issue seems to be related to the loss index of history. It's a standard history and should work without any problem. Anything I am missing?

mikkokotila commented 5 years ago

Could you try the early release of the next LTS version:

pip install -U git+https://github.com/autonomio/talos@params-api-test

I don't think you will get this error anymore.

troybozarth commented 5 years ago

I'm getting the same error. I installed the version you linked and it looks like the scan object works now but the Scan init is different, mainly it doesn't look to have grid_downsample along with experiment_no and dataset_name.

Cheers

troybozarth commented 5 years ago

Also is reduction taken out of this version? The reduction_metric call doesnt seem to go anywhere and gets hit with ['val_acc'] not in index.

mikkokotila commented 5 years ago

In the v.0.6 Scan accepts: 

def __init__(self, x, y, params, model,
             experiment_name=None,
             x_val=None,
             y_val=None,
             val_split=.3,
             random_method='uniform_mersenne',
             performance_target=None,
             fraction_limit=None,
             round_limit=None,
             time_limit=None,
             boolean_limit=None,
             reduction_method=None,
             reduction_interval=50,
             reduction_window=20,
             reduction_threshold=0.2,
             reduction_metric='val_acc',
             minimize_loss=False,
             seed=None,
             clear_session=True,
             disable_progress_bar=False,
             print_params=False,
             debug=False):

Feel free to open a new ticket if you have any issues.

For this issue, if you are getting an error still, can you post the new trace.

ChrisMacaluso commented 5 years ago

Could you try the early release of the next LTS version:

pip install -U git+https://github.com/autonomio/talos@params-api-test

I don't think you will get this error anymore.

so this version does get rid of the TypeError: ("cannot convert the series to <class 'float'>", 'occurred at index loss') error?

mikkokotila commented 5 years ago

@55thSwiss That's now the current version, so you should install it anyway. But with:

pip install -U git+https://github.com/autonomio/talos

This error should not appear anymore for sometime.

ChrisMacaluso commented 5 years ago

@55thSwiss That's now the current version, so you should install it anyway. But with:

pip install -U git+https://github.com/autonomio/talos

This error should not appear anymore for sometime.

The error is gone, thanks for the information.

JayasriC commented 5 years ago

import pandas as pd import geopandas

input_bell='XXX' df = pd.read_csv(input_bell,encoding = 'unicode_escape',usecols = ['Latitude','Longitude'])

gdf = geopandas.GeoDataFrame(df, geometry=geopandas.points_from_xy(df.Longitude, df.Latitude))

import pandas as pd import geopandas as gpd from shapely.geometry import Point

print(df.head())

df['geometry'] = df.apply(lambda x: Point(float(df.Longitude), float(x.Latitude)), axis=1)

df_geo = gpd.GeoDataFrame(df, crs = neighborhoods.crs, geometry = df.geometry) print(type(df_geo))

output:

runfile('/Users/jayasri/geopanads.py', wdir='/Users/jayasri') Latitude Longitude 0 53.518491 -113.520213 1 45.435753 -75.644008 2 48.13555 -78.125831 3 47.69876 -70.36251 4 49.2881 -123.12615 Traceback (most recent call last):

File "", line 1, in runfile('/Users/jayasri/geopanads.py', wdir='/Users/jayasri')

File "//anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 827, in runfile execfile(filename, namespace)

File "//anaconda3/lib/python3.7/site-packages/spyder_kernels/customize/spydercustomize.py", line 110, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "/Users/jayasri/geopanads.py", line 19, in df['geometry'] = df.apply(lambda x: Point(float(df.Longitude), float(x.Latitude)), axis=1)

File "//anaconda3/lib/python3.7/site-packages/pandas/core/frame.py", line 6487, in apply return op.get_result()

File "//anaconda3/lib/python3.7/site-packages/pandas/core/apply.py", line 151, in get_result return self.apply_standard()

File "//anaconda3/lib/python3.7/site-packages/pandas/core/apply.py", line 257, in apply_standard self.apply_series_generator()

File "//anaconda3/lib/python3.7/site-packages/pandas/core/apply.py", line 286, in apply_series_generator results[i] = self.f(v)

File "/Users/jayasri/geopanads.py", line 19, in df['geometry'] = df.apply(lambda x: Point(float(df.Longitude), float(x.Latitude)), axis=1)

File "//anaconda3/lib/python3.7/site-packages/pandas/core/series.py", line 93, in wrapper "{0}".format(str(converter)))

TypeError: ("cannot convert the series to <class 'float'>", 'occurred at index 0')]

JayasriC commented 5 years ago

It throws this error, am trying to convert the dataframe to a Geodataframe, can anyone tell me what's the issue.

ChrisMacaluso commented 5 years ago

@JayasriC This thread is about the error related to Talos, not Pandas or Geopandas