search

imsoo / fight_detection

Real time Fight Detection Based on 2D Pose Estimation and RNN Action Recognition
MIT License
197 stars 43 forks source link

Python action. Py prompts for error #5

Open gzchenjiajun opened 4 years ago

gzchenjiajun commented 4 years ago

tensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'CudnnRNN' used by node cu_dnnlstm/CudnnRNN (defined at action.py:28) with these attrs: [seed=0, dropout=0, input_mode="linear_input", T=DT_FLOAT, direction="unidirectional", rnn_mode="lstm", seed2=0, is_training=true] Registered devices: [CPU] Registered kernels:

[[cu_dnnlstm/CudnnRNN]]
gzchenjiajun commented 4 years ago

Device: MacBook Pro 2016

imsoo commented 4 years ago

@gzchenjiajun,

CudnnLSTM is a GPU-only operation. (it depends on CuDNN) So we need to transform CuDNNLSTM to LSTM. (CuDNNLSTM/LSTM weights are interchangeable)

Delete action.py Line 26 and insert below code. (Creating a standard model with the same architecture and load weights.)

https://github.com/imsoo/fight_detection/blob/4eb493056c744a88b2255054130c9b299fb7ad0b/server/action.py#L26

from tensorflow.keras import layers
lambda_loss_amount = 0.0015
model = tf.keras.Sequential([
   # relu activation
   layers.Dense(n_hidden, activation='relu',
       kernel_initializer='random_normal',
       bias_initializer='random_normal',
       batch_input_shape=(batch_size, n_steps, n_input)
   ),

   # cuDNN
   # layers.CuDNNLSTM(n_hidden, return_sequences=True,  unit_forget_bias=1.0),
   # layers.CuDNNLSTM(n_hidden,  unit_forget_bias=1.0),

   layers.LSTM(n_hidden, return_sequences=True,  unit_forget_bias=1.0),
   layers.LSTM(n_hidden,  unit_forget_bias=1.0),

   layers.Dense(n_classes, kernel_initializer='random_normal',
       bias_initializer='random_normal',
       kernel_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
       bias_regularizer=tf.keras.regularizers.l2(lambda_loss_amount),
       activation='softmax'
   )
])
model.load_weights("weights/action.h5")
gzchenjiajun commented 4 years ago

This repo is an LSTM, I want to use it as real-time streaming video detection (RTSP), is it feasible? Any Suggestions for modification?

@imsoo

imsoo commented 4 years ago

@gzchenjiajun

In this project LSTM is expensive but Pose Estimation (CNN) is more expensive. It spend most of time (About 95 percent)

So if your machine can process pose estimation in real time, I think it may feasible. You can check that pose estimation works fine in this repository.

In my case, I tested it in the following environment. (Note : Only one client.) Client <------ Web Cam Stream ----> GCP Server (Nvidia K80)

I think that following case will need more resource.

gzchenjiajun commented 4 years ago

If I want to realize real-time image detection of RTSP stream, do you have any Suggestions?

gzchenjiajun commented 4 years ago

If I want to realize real-time image detection of RTSP stream, do you have any Suggestions? @imsoo

imsoo commented 4 years ago

@gzchenjiajun

Sorry for my late reply.

I've never used RTSP. I regret to say that I am unable to help you.

But I read about RTSP and then I'll let you know if good idea comes of it.

gzchenjiajun commented 4 years ago

Ok, thank you very much! It appears that this repo does not currently support RTSP live streaming. So if I want to predict the sequence of images in a folder, is that ok?

@imsoo

imsoo commented 4 years ago

@gzchenjiajun

Unfortunately This repo doesn't support sequence of images.
Only support video (MP4, AVI ...) or web cam stream. You need to convert an image sequence into a video before you start.

gzchenjiajun commented 4 years ago

Ok, I will try it with the video, and I will ask you if I have any questions. Thank you @imsoo

gzchenjiajun commented 4 years ago

I would like to ask if there is any recommended that can realize real-time fight detection? @imsoo

imsoo commented 4 years ago

@gzchenjiajun

Sorry I'm not sure what's meant by "warehouse".

I think that "warehouse" means repository or dataset. Is that right? if so, Below dataset looks good.

And in case of repository, I don't know actually which one is the best. I think it all depends on your purpose.

gzchenjiajun commented 3 years ago

@gzchenjiajun

Unfortunately This repo doesn't support sequence of images. Only support video (MP4, AVI ...) or web cam stream. You need to convert an image sequence into a video before you start.

What is the format of msg data? #17 I want to convert it into msg format based on the stream camera. How should I do it?