训练自己的数据集

[louxy126]

Hi@TianzhongSong 博主你好，我现在想用你的网络训练自己的数据集（51类），我已经将数据集制作成需要的格式，并生成了train_list.txt和test_list.txt,我将models.py中的nb_classes改为了51；将train_c3d.py中的num_classes也改为了51，我在WIN10下训练，没有找到train_c3d.py下定义的root_path，我将root_path改为视频类别所在的目录，但是在运行train_c3d.py时，model.fit_generator处报错： Epoch 1/6 2018-10-12 21:58:30.598905: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 2018-10-12 21:58:30.600815: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance. 2018-10-12 21:58:33.877366: E tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Traceback (most recent call last): File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 195, in <module> main() File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 173, in main verbose=1) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1415, in fit_generator initial_epoch=initial_epoch) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training_generator.py", line 213, in fit_generator class_weight=class_weight) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1215, in train_on_batch outputs = self.train_function(ins) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2666, in __call__ return self._call(inputs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2635, in _call session) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2587, in _make_callable callable_fn = session._make_callable_from_options(callable_opts) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1414, in _make_callable_from_options return BaseSession._Callable(self, callable_options) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1368, in __init__ session._session, options_ptr, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Exception ignored in: <bound method BaseSession._Callable.__del__ of <tensorflow.python.client.session.BaseSession._Callable object at 0x0000027A1EE51D30>> Traceback (most recent call last): File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1398, in __del__ self._session._session, self._handle, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 2723102589000 我在网上没找到解决方案，所以想请教我这种做法有哪里出错了？十分感谢

[TianzhongSong]

@louxy126 收到你的邮件了。你的Tensorflow和Keras的版本是多少？

[TianzhongSong]

@louxy126 如果tensorflow版本是最新的，建议降为1.4、1.5再试试。Keras也是，推荐使用2.0.8、或2.1.0

[louxy126]

谢谢回复！ 按照你的提示，我更改了Keras和Tensorflow版本，Keras=2.0.8，tensorflow=1.3.0（如果安装tf1.3的gpu版本还是会报错，因为我的CUDA=9.2.148和CUDNN=9.2版本与TF1.3-gpu不适应，所以我用cpu训练了）；现在可以训练了，十分感谢！

[ucasiggcas]

tf1.15可以吗？

[Ruyue-Wang]

Hi@TianzhongSong 博主你好，我现在想用你的网络训练自己的数据集（51类），我已经将数据集制作成需要的格式，并生成了train_list.txt和test_list.txt,我将models.py中的nb_classes改为了51；将train_c3d.py中的num_classes也改为了51，我在WIN10下训练，没有找到train_c3d.py下定义的root_path，我将root_path改为视频类别所在的目录，但是在运行train_c3d.py时，model.fit_generator处报错： Epoch 1/6 2018-10-12 21:58:30.598905: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 2018-10-12 21:58:30.600815: I tensorflow/core/common_runtime/process_util.cc:69] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance. 2018-10-12 21:58:33.877366: E tensorflow/core/common_runtime/executor.cc:697] Executor failed to create kernel. Invalid argument: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Traceback (most recent call last): File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 195, in <module> main() File "L:/match/2018jiangsupostgraduatecv/C3D-keras-master/train_c3d.py", line 173, in main verbose=1) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper return func(*args, **kwargs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1415, in fit_generator initial_epoch=initial_epoch) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training_generator.py", line 213, in fit_generator class_weight=class_weight) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\engine\training.py", line 1215, in train_on_batch outputs = self.train_function(ins) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2666, in __call__ return self._call(inputs) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2635, in _call session) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\keras\backend\tensorflow_backend.py", line 2587, in _make_callable callable_fn = session._make_callable_from_options(callable_opts) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1414, in _make_callable_from_options return BaseSession._Callable(self, callable_options) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1368, in __init__ session._session, options_ptr, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Value for attr 'data_format' of "NDHWC" is not in the list of allowed values: "NHWC", "NCHW" ; NodeDef: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1); Op<name=_MklInputConversion; signature=input_0:T, input_1:T, mkl_input_0:uint8, mkl_input_1:uint8 -> output_0:T, output_1:T, mkl_output_0:uint8, mkl_output_1:uint8; attr=T:type,allowed=[DT_HALF, DT_FLOAT, DT_DOUBLE, DT_UINT8, DT_INT8, DT_UINT16, DT_INT16, DT_INT32, DT_INT64, DT_COMPLEX64, DT_COMPLEX128]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]> [[Node: MklInputConversion/_237 = _MklInputConversion[T=DT_FLOAT, _kernel="MklOp", data_format="NDHWC", _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv3d_1/convolution, conv3d_1/Reshape, DMT/_10, conv3d_1/Reshape:1)]] Exception ignored in: <bound method BaseSession._Callable.__del__ of <tensorflow.python.client.session.BaseSession._Callable object at 0x0000027A1EE51D30>> Traceback (most recent call last): File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\client\session.py", line 1398, in __del__ self._session._session, self._handle, status) File "K:\Anaconda\envs\classifyvedio\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 519, in __exit__ c_api.TF_GetCode(self.status.status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 2723102589000 我在网上没找到解决方案，所以想请教我这种做法有哪里出错了？十分感谢

hi你好，可不可以方便说一下怎么制作自己的数据集？