Closed chstarr closed 5 years ago
I think you have to set talos.Scan( ... reset_tf_graph=False ... )
.
Note that this is reset_graph=False
from v.0.6 onwards.
Let me know if that works.
Unforunately didn't work...here is what I saw:
t = ta.Scan(np.expand_dims(trainHists2,axis=2), trainLabels, params=p, model=mods.cnnMP, reset_tf_graph=False)
Traceback (most recent call last):
File "
TypeError: init() got an unexpected keyword argument 'reset_tf_graph'
Try clear_tf_session
instead. Sorry for the confusion.
Closing this as resolved. Feel free to open a new issue if anything.
I am using talos to do hyperparameter optimization on a keras CNN that I am using to analyze 1D data. I am running into the error described in the title, and can't figure out how to proceed. Below, I have pasted the code that I think might be relevant to the issue. Please let me know if you can spot the issue. Thank you!
Here is the model definition: def cnnMP(x_train, y_train, x_val, y_val, params):
initialize the model
Here is where I set up the hyperparameter scan (just trying to look at batch size for now): p = { 'nBins': [512], 'nClasses': [2], 'batchSize': [32, 64, 128], 'epochs': [50], 'activation': ['relu'], 'filters1': [50], 'kernelSize1': [5], 'poolSize1': [2], 'filters2': [20], 'kernelSize2': [5], 'poolSize2': [2], 'dimFC1': [500], 'loss': ["categorical_crossentropy"], 'opt': [SGD(lr=0.01)], 'verbose': [0] }
t = ta.Scan(np.expand_dims(trainHist2,axis=2), trainLabels, params=p, model=mods.cnnMP)
I needed to use the np.expand(...) function to get my data to pass through keras. By the way, I did confirm that this model works well with keras on its own. The one difference is that I have to remove the [ ] brackets from the parameter list when passing into model.fit() because this function requires actual parameters, not iterable lists.
Finally, here is the error I get when running ta.Scan(...):
0%| | 0/3 [00:00<?, ?it/s]
33%|███▎ | 1/3 [00:03<00:07, 3.50s/it]Traceback (most recent call last):
File "", line 1, in
runfile('C:/Users/cstarr/Documents/GitHub/MP_nnDetector/cnnMP_optimHP.py', wdir='C:/Users/cstarr/Documents/GitHub/MP_nnDetector')
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile execfile(filename, namespace)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/cstarr/Documents/GitHub/MP_nnDetector/cnnMP_optimHP.py", line 122, in
t = ta.Scan(np.expand_dims(trainHist2,axis=2), trainLabels, params=p, model=mods.cnnMP)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\talos\scan\Scan.py", line 170, in init self._null = self.runtime()
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\talos\scan\Scan.py", line 175, in runtime self = scan_run(self)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\talos\scan\scan_run.py", line 18, in scan_run self = scan_round(self)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\talos\scan\scan_round.py", line 32, in scan_round _hr_out, self.keras_model = ingest_model(self)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\talos\model\ingest_model.py", line 10, in ingest_model self.round_params)
File "C:\Users\cstarr\Documents\GitHub\MP_nnDetector\modelDefs.py", line 38, in cnnMP validation_data=[x_val,y_val])
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\engine\training.py", line 1010, in fit self._make_train_function()
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\engine\training.py", line 509, in _make_train_function loss=self.total_loss)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper return func(*args, **kwargs)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\keras\optimizers.py", line 196, in get_updates v = self.momentum m - lr g # velocity
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\ops\variables.py", line 935, in _run_op return tensor_oper(a.value(), *args, **kwargs)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 810, in binary_op_wrapper with ops.name_scope(None, op_name, [x, y]) as name:
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 6083, in enter g = _get_graph_from_inputs(self._values)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 5713, in _get_graph_from_inputs _assert_same_graph(original_graph_element, graph_element)
File "C:\Users\cstarr\AppData\Local\Continuum\anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 5649, in _assert_same_graph original_item))
ValueError: Tensor("training/SGD/Variable:0", shape=(5, 1, 50), dtype=float32_ref) must be from the same graph as Tensor("SGD_1/momentum/read:0", shape=(), dtype=float32).