Open wtnan2003 opened 4 years ago
lequytra
commented
4 years ago Hi @wtnan2003,
We added some updates to the codebase recently. Did you re-build with python setup.py build develop after pulling the new updates? We also added a small fix today so I suggest pulling that as well for the demo.
wtnan2003
commented
4 years ago @lequytra thx!
There are some problems after I update the code :
when I run code, got KeyError:'Non-existent config key:TENSORBOARD.MODEL_VIS.TOPK'
why ava.names are deleted? where should I found LABEL_FILE to fill LABEL_FILE_PATH in config file，
if I fill LABEL_FILE_PATH with "/home/nd/SlowFast/demo/AVA/ava.names"
Fail to load file from /home/nd/SlowFast/demo/AVA/ava.names with error Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last): File "tools/run_net.py", line 39, in
main() File "tools/run_net.py", line 35, in main demo(cfg) File "/home/nd/SlowFast/tools/demo_net.py", line 97, in demo for frames in tqdm.tqdm(run_demo(cfg, frame_provider)): File "/home/nd/anaconda3/envs/SlowFast/lib/python3.6/site-packages/tqdm/std.py", line 1130, in iter for obj in iterable: File "/home/nd/SlowFast/tools/demo_net.py", line 49, in run_demo cfg.TENSORBOARD.MODEL_VIS.COLORMAP, File "/home/nd/SlowFast/slowfast/visualization/video_visualizer.py", line 358, in init self.classnames, , _ = get_class_names(class_names_path, None, None) TypeError: 'NoneType' object is not iterable
I can run demo with the old version
wtnan2003
commented
4 years ago @lequytra
I delete TENSORBOARD and change ava.names to ava.json like :
Demo can be run, but the result of detection seems to be incorrect:
new version:
old version:
The surveillance video is two people fight.
Ps: the new version of demo runs very slowly by using the Webcam (6s/it) and the size of the window is also unnormal(very small)
lequytra
commented
4 years ago Hi @wtnan2003,
Thank you for playing with the PySlowFast codebase.
I do want to apologize for failing to inform you about the change in class name file format. Demo is a new and developing feature so everything is not entirely pushed and well-documented. We have plan to release this feature officially in a few weeks so we will publish all related files and documentations then. Off the bat, our class name files start with index 0 instead of 1. That might explain the wildly wrong results you see in the new demo. One other thing I want to add is that we have TENSORBOARD.MODEL_VIS.TOPK_PREDS param that can be changed to increase/decrease the number of predictions visualized. We also have new updates coming up for choosing predictions with threshold as well. So watch out for that :)
Regarding speed, the new demo version waits to collect all necessary frames (in this case 32x2 = 64 frames), make predictions and visualization on those frames and then display them. This causes some necessary delay since, say with a fps of 30 for the webcam, we need to wait > 2s just to collect enough frames to make the predictions and visualization. This was not a problem in the previous demo release since we visualize results to whatever frames we just capture (this means that results for frames 1...64 is visualized to 65...128, causing some mismatch problem in the visualization). If your webcam's fps rate is very slow, I would suggest increasing DEMO.BUFFER_SIZE parameter (which I believe is set to 0 by default). This basically use the last few x frames from the previous clip (the previous 64 frames) to make a prediction, so you can save some wait time capturing enough new frames. With that being said, we will have new updates in a few days to help with speed/delay.
I'll look into the small window size problem. Did you experience this with the old demo version?
Milkve
commented
4 years ago Hi @lequytra
I updated the slowfast code on my own computer, and I found I lost 2 files in ./demo subdir, I just run this command
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml
and I got this problem:
Traceback (most recent call last): File "tools/run_net.py", line 39, in <module> main() File "tools/run_net.py", line 19, in main cfg = load_config(args) File "/home/rbmilkve/SlowFast/slowfast/utils/parser.py", line 78, in load_config cfg.merge_from_file(args.cfg_file) File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/fvcore/common/config.py", line 108, in merge_from_file loaded_cfg = type(self)(loaded_cfg) File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/yacs/config.py", line 86, in __init__ init_dict = self._create_config_tree_from_dict(init_dict, key_list) File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/yacs/config.py", line 126, in _create_config_tree_from_dict dic[k] = cls(v, key_list=key_list + [k]) File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/yacs/config.py", line 86, in __init__ init_dict = self._create_config_tree_from_dict(init_dict, key_list) File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/yacs/config.py", line 132, in _create_config_tree_from_dict ".".join(key_list + [str(k)]), type(v), _VALID_TYPES File "/home/rbmilkve/anaconda3/envs/pyslowfast/lib/python3.7/site-packages/yacs/config.py", line 525, in _assert_with_logging assert cond, msg AssertionError: Key DEMO.LABEL_FILE_PATH with value <class 'NoneType'> is not a valid type; valid types: {<class 'str'>, <class 'int'>, <class 'bool'>, <class 'tuple'>, <class 'list'>, <class 'float'>}
this error is caused by label file lost seemingly, I remembered there were two label files come with two config files in ./demo/AVA and ./demo/Kinetics respectively. Now how could I run demo on my computer without training and testing?
wtnan2003
commented
4 years ago @lequytra thanks so much for your detailed reply!
I make the index start with 0, the demo seems to work fine, thx!
I notice that:
demo finish with “3it [00:10,3.40s/it]” (my test video is 6s) "3.40s/it" means 64 frames processed in 3.4s or 3.4s per frame?
Anyway, I am looking forward to your new updates.
I would be appreciated if updates coming in a few days!
wtnan2003
commented
4 years ago @Milkve change class name file format to this work for me
DEMO.LABEL_FILE_PATH = /path/to/ava.json
lequytra
commented
4 years ago @wtnan2003 That'll be 64 frames in 3.4s. Thank you for the feedback and sharing the json file. :)
Priyabrata-M
commented
4 years ago @lequytra I delete TENSORBOARD and change ava.names to ava.json like :
Demo can be run, but the result of detection seems to be incorrect:
new version:
old version:
The surveillance video is two people fight.
Ps: the new version of demo runs very slowly by using the Webcam (6s/it) and the size of the window is also unnormal(very small)
@wtnan2003 you said you have deleted TENSORBOARD, what does that mean, Have you selectively removed all the statements & dependencies on tensorboard?
wtnan2003
commented
4 years ago @Priyabrata-M I just annotate three lines of TENSORBOARD Here is my SLOWFAST_32x2_R101_50_50.yaml file：
Priyabrata-M
commented
4 years ago Thanks @wtnan2003 for the quick response! It worked for me:)
wwdok
commented
3 years ago @lequytra I delete TENSORBOARD and change ava.names to ava.json like :
Demo can be run, but the result of detection seems to be incorrect:
new version:
old version:
The surveillance video is two people fight.
Ps: the new version of demo runs very slowly by using the Webcam (6s/it) and the size of the window is also unnormal(very small)
Hi, @wtnan2003 , could you please share me your dataset ( if thewhole is too big, fall down, sit , stand, get up, martial art are enough), thanks !
sujaybabruwad
commented
3 years ago Hi @wtnan2003,
Thank you for playing with the PySlowFast codebase.
I do want to apologize for failing to inform you about the change in class name file format. Demo is a new and developing feature so everything is not entirely pushed and well-documented. We have plan to release this feature officially in a few weeks so we will publish all related files and documentations then. Off the bat, our class name files start with index 0 instead of 1. That might explain the wildly wrong results you see in the new demo. One other thing I want to add is that we have
TENSORBOARD.MODEL_VIS.TOPK_PREDSparam that can be changed to increase/decrease the number of predictions visualized. We also have new updates coming up for choosing predictions with threshold as well. So watch out for that :)Regarding speed, the new demo version waits to collect all necessary frames (in this case 32x2 = 64 frames), make predictions and visualization on those frames and then display them. This causes some necessary delay since, say with a fps of 30 for the webcam, we need to wait > 2s just to collect enough frames to make the predictions and visualization. This was not a problem in the previous demo release since we visualize results to whatever frames we just capture (this means that results for frames 1...64 is visualized to 65...128, causing some mismatch problem in the visualization). If your webcam's
fpsrate is very slow, I would suggest increasingDEMO.BUFFER_SIZEparameter (which I believe is set to 0 by default). This basically use the last fewxframes from the previous clip (the previous 64 frames) to make a prediction, so you can save some wait time capturing enough new frames. With that being said, we will have new updates in a few days to help with speed/delay.I'll look into the small window size problem. Did you experience this with the old demo version?
Hi @lequytra
When I try to infer on a video and output inference to a window and not a file. I don't see the video display but just a very small window as shown below.
@haooooooqi @lequytra If I use a webcam instead of a video input. Does the inference visualization be real-time ?
Thanks
INGnowait
commented
3 years ago Hi @wtnan2003, Thank you for playing with the PySlowFast codebase. I do want to apologize for failing to inform you about the change in class name file format. Demo is a new and developing feature so everything is not entirely pushed and well-documented. We have plan to release this feature officially in a few weeks so we will publish all related files and documentations then. Off the bat, our class name files start with index 0 instead of 1. That might explain the wildly wrong results you see in the new demo. One other thing I want to add is that we have
TENSORBOARD.MODEL_VIS.TOPK_PREDSparam that can be changed to increase/decrease the number of predictions visualized. We also have new updates coming up for choosing predictions with threshold as well. So watch out for that :) Regarding speed, the new demo version waits to collect all necessary frames (in this case 32x2 = 64 frames), make predictions and visualization on those frames and then display them. This causes some necessary delay since, say with a fps of 30 for the webcam, we need to wait > 2s just to collect enough frames to make the predictions and visualization. This was not a problem in the previous demo release since we visualize results to whatever frames we just capture (this means that results for frames 1...64 is visualized to 65...128, causing some mismatch problem in the visualization). If your webcam'sfpsrate is very slow, I would suggest increasingDEMO.BUFFER_SIZEparameter (which I believe is set to 0 by default). This basically use the last fewxframes from the previous clip (the previous 64 frames) to make a prediction, so you can save some wait time capturing enough new frames. With that being said, we will have new updates in a few days to help with speed/delay. I'll look into the small window size problem. Did you experience this with the old demo version?Hi @lequytra
When I try to infer on a video and output inference to a window and not a file. I don't see the video display but just a very small window as shown below.
@haooooooqi @lequytra If I use a webcam instead of a video input. Does the inference visualization be real-time ?
Thanks
I have the same problem with you. Have you solved it? Thanks
INGnowait
commented
3 years ago @lequytra thanks so much for your detailed reply!
I make the index start with 0, the demo seems to work fine, thx!
I notice that:
demo finish with “3it [00:10,3.40s/it]” (my test video is 6s) "3.40s/it" means 64 frames processed in 3.4s or 3.4s per frame?
Anyway, I am looking forward to your new updates.
I would be appreciated if updates coming in a few days!
Hi @wtnan2003 ，can you tell me how you solve the small window size? Thanks
kaqyoin7
commented
1 year ago I also get an error when I execute :
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml
The error message is :
KeyError：“Non-existent config key：DEMO.INPUT_VIDEO”
I want to know why this is and how to solve this problem
Any help would be appreciated !!!
Any help would be appreciated !!!
Young-eng
commented
6 months ago I also get an error when I execute :
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yamlThe error message is :KeyError：“Non-existent config key：DEMO.INPUT_VIDEO”I want to know why this is and how to solve this problem Any help would be appreciated !!! Any help would be appreciated !!!
I think you need to add parameters in xxx.yaml file like this: DATA: INPUT_VIDEO: path-to-your-video
or python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yaml DEMO.INPUT_VIDEO path-to-your-video
I want to input video for demo, so I change DEMO.DATA_SOURCE to my video path, run:
python tools/run_net.py --cfg demo/AVA/SLOWFAST_32x2_R101_50_50.yamlbut I got:
ValueError: Type mismatch (<class 'int'> vs. <class 'str'>) with values (0 vs. ./fight2.mp4) for config key: DEMO.DATA_SOURCE
Traceback
File "tools/run_net.py", line 37, inSo I change add INPUT_VIDEO in DEMO part: INPUT_VIDEO:"./fight2.mp4"
then I got
KeyError：“Non-existent config key：DEMO.INPUT_VIDEO”
Any help would be appreciated
here is my config file:
slowfast/demo/AVA/SLOWFAST_32x2_R101_50_50.yaml
```yaml TRAIN: ENABLE: False DATASET: ava BATCH_SIZE: 16 EVAL_PERIOD: 1 CHECKPOINT_PERIOD: 1 AUTO_RESUME: True CHECKPOINT_FILE_PATH: "./SLOWFAST_32x2_R101_50_50.pkl" #path to pretrain model CHECKPOINT_TYPE: pytorch DATA: NUM_FRAMES: 32 SAMPLING_RATE: 2 TRAIN_JITTER_SCALES: [256, 320] TRAIN_CROP_SIZE: 224 TEST_CROP_SIZE: 256 INPUT_CHANNEL_NUM: [3, 3] DETECTION: ENABLE: True ALIGNED: False AVA: BGR: False DETECTION_SCORE_THRESH: 0.8 TEST_PREDICT_BOX_LISTS: ["person_box_67091280_iou90/ava_detection_val_boxes_and_labels.csv"] SLOWFAST: ALPHA: 4 BETA_INV: 8 FUSION_CONV_CHANNEL_RATIO: 2 FUSION_KERNEL_SZ: 5 RESNET: ZERO_INIT_FINAL_BN: True WIDTH_PER_GROUP: 64 NUM_GROUPS: 1 DEPTH: 101 TRANS_FUNC: bottleneck_transform STRIDE_1X1: False NUM_BLOCK_TEMP_KERNEL: [[3, 3], [4, 4], [6, 6], [3, 3]] SPATIAL_DILATIONS: [[1, 1], [1, 1], [1, 1], [2, 2]] SPATIAL_STRIDES: [[1, 1], [2, 2], [2, 2], [1, 1]] NONLOCAL: LOCATION: [[[], []], [[], []], [[6, 13, 20], []], [[], []]] GROUP: [[1, 1], [1, 1], [1, 1], [1, 1]] INSTANTIATION: dot_product POOL: [[[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]], [[2, 2, 2], [2, 2, 2]]] MODEL: NUM_CLASSES: 80 ARCH: slowfast LOSS_FUNC: bce DROPOUT_RATE: 0.5 HEAD_ACT: sigmoid TEST: ENABLE: False DATASET: ava BATCH_SIZE: 8 DATA_LOADER: NUM_WORKERS: 2 PIN_MEMORY: True DEMO: ENABLE: True LABEL_FILE_PATH: "./demo/AVA/ava.names" DATA_SOURCE: "./fight2.mp4" # INPUT_VIDEO: "./fight2.mp4" OUTPUT_FILE: "./output.mp4" # DISPLAY_WIDTH: 640 # DISPLAY_HEIGHT: 480 DETECTRON2_OBJECT_DETECTION_MODEL_CFG: "COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml" DETECTRON2_OBJECT_DETECTION_MODEL_WEIGHTS: "/home/nd/slowfast/demo/AVA/model_final_280758.pkl" # DETECTRON2_OBJECT_DETECTION_MODEL_WEIGHTS: "detectron2://COCO-Detection/faster_rcnn_R_50_FPN_3x/137849458/model_final_280758.pkl" NUM_GPUS: 1 NUM_SHARDS: 1 RNG_SEED: 0 OUTPUT_DIR: . ```any help would be appreciated!