KeyError: "['val_acc'] not in index"

Thanks so much for coming here to raise an issue. Please take a moment to 'check' the below boxes:

[x] I'm up-to-date with the latest release:

pip install -U talos
[x] I've confirmed that my Keras model works outside of Talos.

If you still have an error, please submit complete trace and a code with:

output of shape for x and y e.g. (212,12)
Talos params dictionary
The Keras model wired for Talos
Description of extra variables in the model

You can provide the code in pastebin / gist or any other format you like.

Hi Mikko,

I was originally receiving the error noted here: https://github.com/autonomio/talos/issues/299?_pjax=%23js-repo-pjax-container

As per your instructions I installed talos 0.6.0 which solved that error, however if I try and run Scan with reduction I know receive a new error.

Here is my code: X.shape Out[13]: (84, 6)

y.shape Out[14]: (84,)

p = {'lr': (0.1, 1, 10), 'first_neuron': [12,24,48,96], 'Dropout': (0,0.4,10), 'loss': ['mse', 'mae'], 'second_neuron': [3,6,12], 'Activation': ['relu', 'sigmoid'], 'optimizer': [rmsprop, adam, sgd], 'batch_size': [8,16,32], 'epochs': [50,100,200], 'kn_init': ['normal', 'uniform'] }

def create_model(X_train, y_train, X_val, y_val,params):

model = Sequential()
model.add(Dense(params['first_neuron'], input_dim=6, kernel_initializer = params['kn_init']))
model.add(Activation('relu'))
model.add(Dropout(params['Dropout']))
model.add(Dense(params['second_neuron'], kernel_initializer = params['kn_init']))
model.add(Activation(params['Activation']))
model.add(Dropout(params['Dropout']))
model.add(Dense(1, activation='linear', kernel_initializer = params['kn_init']))

model.compile(loss=params['loss'], optimizer=params['optimizer'](lr=lr_normalizer(params['lr'],params['optimizer'])))

out = model.fit(X_train, y_train,
          batch_size=params['batch_size'],
          epochs=params['epochs'],
          verbose=0,
          validation_data=(X_val, y_val))

return(out, model)

h = ta.Scan(X,y, params=p, model = create_model, experiment_name='NN', random_method='quantum', fraction_limit=0.001, reduction_method='correlation', reduction_interval=20, reduction_window=10, reduction_threshold=0.4, reduction_metric='val_acc' )

Error Trace:

7%|▋ | 18/259 [00:23<05:47, 1.44s/it]Traceback (most recent call last):

File "", line 10, in reduction_metric='val_acc'

File "C:\Anaconda\Lib\site-packages\talos\scan\Scan.py", line 172, in init self.runtime()

File "C:\Anaconda\Lib\site-packages\talos\scan\Scan.py", line 177, in runtime self = scan_run(self)

File "C:\Anaconda\Lib\site-packages\talos\scan\scan_run.py", line 26, in scan_run self = scan_round(self)

File "C:\Anaconda\Lib\site-packages\talos\scan\scan_round.py", line 28, in scan_round self = reduce_run(self)

File "C:\Anaconda\Lib\site-packages\talos\reducers\reduce_run.py", line 48, in reduce_run label_and_value = correlation(self)

File "C:\Anaconda\Lib\site-packages\talos\reducers\correlation.py", line 19, in correlation data = data[[self.reduction_metric] + self._param_dict_keys]

File "C:\Anaconda\Lib\site-packages\pandas\core\frame.py", line 2934, in getitem raise_missing=True)

File "C:\Anaconda\Lib\site-packages\pandas\core\indexing.py", line 1354, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, **kwargs)[1]

File "C:\Anaconda\Lib\site-packages\pandas\core\indexing.py", line 1161, in _get_listlike_indexer raise_missing=raise_missing)

File "C:\Anaconda\Lib\site-packages\pandas\core\indexing.py", line 1252, in _validate_read_indexer raise KeyError("{} not in index".format(not_found))

KeyError: "['val_acc'] not in index"

I know the accuracy parameters aren't meant for regression problems but I've tried several of the metrics as listed in names.py and the same error comes up for all of them.

Thanks

autonomio / talos

KeyError: "['val_acc'] not in index" #310