search

dxli94 / WLASL

WACV 2020 "Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison"
https://dxli94.github.io/WLASL/
838 stars 111 forks source link

Testing the mode #9

Closed abhiray92 closed 3 years ago

abhiray92 commented 3 years ago

Hi there, I was trying to test the trained model, but I am getting the below error while testing - 

('11772',) 0.6267942583732058 0.8133971291866029 0.8516746411483254
('17024',) 0.631578947368421 0.8181818181818182 0.8564593301435407
('17020',) 0.631578947368421 0.8181818181818182 0.861244019138756
('17730',) 0.6363636363636364 0.8229665071770335 0.8660287081339713
('62987',) 0.6411483253588517 0.8277511961722488 0.8708133971291866
('05727',) 0.6411483253588517 0.8277511961722488 0.8708133971291866
('38531',) 0.6411483253588517 0.8325358851674641 0.8755980861244019
('63219',) 0.6411483253588517 0.8373205741626795 0.8803827751196173
('06845',) 0.645933014354067 0.8421052631578947 0.8851674641148325
('57273',) 0.645933014354067 0.8421052631578947 0.8851674641148325
('27194',) 0.6507177033492823 0.84688995215311 0.8899521531100478
('28201',) 0.6555023923444976 0.8516746411483254 0.8947368421052632
top-k average per class acc: nan, nan, nan
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:98: RuntimeWarning: invalid value encountered in true_divide
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:99: RuntimeWarning: invalid value encountered in true_divide
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:100: RuntimeWarning: invalid value encountered in true_divide
dxli94 commented 3 years ago

I cannot comment on what's going on as your log is partial.

One thing I can think of is if you are NOT testing over all the video samples (instead you test part of them), you have to make sure at least one video per class is present. Otherwise, you will get NAN for all the classes with no testing videos.

abhiray92 commented 3 years ago

@dxli94 I trained my model on 100 glosses and I used the same nslt_100.json file for testing. Yet it still gives me NaN. Moreover, if I want to check for just 1 video (.mp4 file) in particular, it wouldn't process that either.

dxli94 commented 3 years ago

The testing code should work as expected, as verified by other users. I could not reproduce the issue, either. Instead, I suggest you set checkpoints and step into the code, in order to understand what is going on.

abhiray92 commented 3 years ago

@dxli94 Thanks, I believe I didn't understand the code well. However, can you please let me know how to use the function 'load_rgb_frames_from_video'? Reason being, I would like to predict the label for a specific video. I don't think the function is being used if i provide the following arguments - 

mode = 'rgb'
num_classes = 100
save_model = '/content/drive/MyDrive/I3D_WLASL/code/checkpoints/'
root = '/content/drive/MyDrive/I3D_WLASL/code/data/WLASL2000/'
train_split = '/content/drive/MyDrive/I3D_WLASL/code/preprocess/nslt_{}.json'.format(num_classes)
weights = '/content/drive/MyDrive/I3D_WLASL/code/archived/asl100/FINAL_nslt_100_iters=896_top1=65.89_top5=84.11_top10=89.92.pt'

run(mode=mode, root=root, save_model=save_model, train_split=train_split, weights=weights)

Any help would be highly appreciated! Thanks alot!