search

sczhou / ProPainter

[ICCV 2023] ProPainter: Improving Propagation and Transformer for Video Inpainting
https://shangchenzhou.com/projects/ProPainter/
Other
5.53k stars 655 forks source link

Davis数据集指标有差异 #55

Open mhq1065 opened 1 year ago

mhq1065 commented 1 year ago

python scripts/compute_flow.py -i datasets/davis/JPEGImages_432_240 -o datasets/datasets/davis/Flows_flo python scripts/evalute_propainter.py --dataset davis --video_rooot datasets/davis/JPEGImages_432_240 --mask_root datasets/davis/test_masks --load_flow True --flow_root datasets/davis/Flows_flo

结果为 PSNR/SSIM/VFID 34.17/0.9771/0.099 ｜ Time 0.1105 与论文结果有较大差距，img和mask均为根据readme下载

sczhou commented 11 months ago

这个结果好像不太对，请问你是在什么resolution下运行的？能不能share一下你的log print内容

LingyiHongfd commented 8 months ago

Hi, I also meet with the same problem. In addition to the misalignment of final score, I am also experiencing problems with codes. I just run the commands: python scripts/evaluate_propainter.py --dataset davis --video_root ./datasets/davis/JPEGImages_432_240 --mask_root ./datasets/davis/test_masks --save_results

(1) The skimage api for ssim calculation. My skimage version is 0.22.0, and I meet with ImportError: cannot import name 'structural_similarity' from 'skimage.measure'' in metrics.py. So, I just modify the origin code by using from skimage.metrics import structural_similarity as ssim to replace the origin ssim calculation codes. (2) VIFD error. I just run the evaluation codes with the official weights. And I meet with raise ValueError('Imaginary component {}'.format(m)) ValueError: Imaginary component 2.454988693213829e+55 error. I don't know the reason for this error. (3) Result misalignment. The following is the pring log. Pretrained flow completion model has loaded... Pretrained ProPainter has loaded... Network [InpaintGenerator] was created. Total number of parameters: 39.4 million. To see the architecture, do print(network). Start evaluation ... [Loading I3D model from weights/i3d_rgb_imagenet.pt for FID score ..] Processing: bear [ 1/50] Name: bear | PSNR/SSIM: 32.7588/0.9878 | Avg PSNR/SSIM: 32.7588/0.9878 | Time: 0.1295 Processing: blackswan [ 2/50] Name: blackswan | PSNR/SSIM: 31.3981/0.9560 | Avg PSNR/SSIM: 32.2434/0.9758 | Time: 0.1131 Processing: bmx-bumps [ 3/50] Name: bmx-bumps | PSNR/SSIM: 39.0668/0.9926 | Avg PSNR/SSIM: 35.0096/0.9826 | Time: 0.1116 Processing: bmx-trees [ 4/50] Name: bmx-trees | PSNR/SSIM: 43.0930/0.9982 | Avg PSNR/SSIM: 37.1509/0.9867 | Time: 0.1096 Processing: boat [ 5/50] Name: boat | PSNR/SSIM: 28.0041/0.9491 | Avg PSNR/SSIM: 35.3313/0.9792 | Time: 0.1092 Processing: breakdance [ 6/50] Name: breakdance | PSNR/SSIM: 25.7639/0.9540 | Avg PSNR/SSIM: 33.5880/0.9746 | Time: 0.1088 Processing: breakdance-flare [ 7/50] Name: breakdance-flare | PSNR/SSIM: 30.5418/0.9659 | Avg PSNR/SSIM: 33.1814/0.9735 | Time: 0.1092 Processing: bus [ 8/50] Name: bus | PSNR/SSIM: 35.0134/0.9952 | Avg PSNR/SSIM: 33.4209/0.9763 | Time: 0.1082 Processing: camel [ 9/50] Name: camel | PSNR/SSIM: 26.1284/0.9327 | Avg PSNR/SSIM: 32.4860/0.9707 | Time: 0.1083 Processing: car-roundabout [ 10/50] Name: car-roundabout | PSNR/SSIM: 39.6992/0.9987 | Avg PSNR/SSIM: 33.1822/0.9734 | Time: 0.1080 Processing: car-shadow [ 11/50] Name: car-shadow | PSNR/SSIM: 47.9661/0.9959 | Avg PSNR/SSIM: 33.9060/0.9745 | Time: 0.1072 Processing: car-turn [ 12/50] Name: car-turn | PSNR/SSIM: 33.5470/0.9827 | Avg PSNR/SSIM: 33.8740/0.9752 | Time: 0.1068 Processing: cows [ 13/50] Name: cows | PSNR/SSIM: 30.1905/0.9873 | Avg PSNR/SSIM: 33.4913/0.9765 | Time: 0.1071 Processing: dance-jump [ 14/50] Name: dance-jump | PSNR/SSIM: 33.6857/0.9735 | Avg PSNR/SSIM: 33.5023/0.9763 | Time: 0.1066 Processing: dance-twirl [ 15/50] Name: dance-twirl | PSNR/SSIM: 31.6373/0.9858 | Avg PSNR/SSIM: 33.3565/0.9771 | Time: 0.1066 Processing: dog [ 16/50] Name: dog | PSNR/SSIM: 36.6663/0.9813 | Avg PSNR/SSIM: 33.5205/0.9773 | Time: 0.1062 Processing: dog-agility [ 17/50] Name: dog-agility | PSNR/SSIM: 24.6029/0.9336 | Avg PSNR/SSIM: 33.3401/0.9764 | Time: 0.1055 Processing: drift-chicane [ 18/50] Name: drift-chicane | PSNR/SSIM: 31.7403/0.9281 | Avg PSNR/SSIM: 33.2755/0.9744 | Time: 0.1053 Processing: drift-straight [ 19/50] Name: drift-straight | PSNR/SSIM: 35.9390/0.9928 | Avg PSNR/SSIM: 33.3750/0.9751 | Time: 0.1049 Processing: drift-turn [ 20/50] Name: drift-turn | PSNR/SSIM: 34.0064/0.9895 | Avg PSNR/SSIM: 33.4039/0.9758 | Time: 0.1049 Processing: elephant [ 21/50] Name: elephant | PSNR/SSIM: 35.4201/0.9900 | Avg PSNR/SSIM: 33.5127/0.9765 | Time: 0.1048 Processing: flamingo [ 22/50] Name: flamingo | PSNR/SSIM: 34.9168/0.9867 | Avg PSNR/SSIM: 33.5846/0.9771 | Time: 0.1046 Processing: goat [ 23/50] Name: goat | PSNR/SSIM: 32.3627/0.9796 | Avg PSNR/SSIM: 33.5180/0.9772 | Time: 0.1046 Processing: hike [ 24/50] Name: hike | PSNR/SSIM: 39.6231/0.9935 | Avg PSNR/SSIM: 33.8000/0.9780 | Time: 0.1045 Processing: hockey [ 25/50] Name: hockey | PSNR/SSIM: 44.2030/0.9936 | Avg PSNR/SSIM: 34.2318/0.9786 | Time: 0.1044 Processing: horsejump-high [ 26/50] Name: horsejump-high | PSNR/SSIM: 26.3283/0.9564 | Avg PSNR/SSIM: 34.0190/0.9780 | Time: 0.1042 Processing: horsejump-low [ 27/50] Name: horsejump-low | PSNR/SSIM: 29.8700/0.9642 | Avg PSNR/SSIM: 33.8892/0.9776 | Time: 0.1039 Processing: kite-surf [ 28/50] Name: kite-surf | PSNR/SSIM: 35.1511/0.9843 | Avg PSNR/SSIM: 33.9212/0.9777 | Time: 0.1038 Processing: kite-walk [ 29/50] Name: kite-walk | PSNR/SSIM: 43.1813/0.9953 | Avg PSNR/SSIM: 34.2831/0.9784 | Time: 0.1039 Processing: libby [ 30/50] Name: libby | PSNR/SSIM: 44.8440/0.9971 | Avg PSNR/SSIM: 34.5300/0.9789 | Time: 0.1038 Processing: lucia [ 31/50] Name: lucia | PSNR/SSIM: 32.7876/0.9851 | Avg PSNR/SSIM: 34.4737/0.9791 | Time: 0.1038 Processing: mallard-fly [ 32/50] Name: mallard-fly | PSNR/SSIM: 32.2577/0.9582 | Avg PSNR/SSIM: 34.4043/0.9784 | Time: 0.1037 Processing: mallard-water [ 33/50] Name: mallard-water | PSNR/SSIM: 29.9082/0.9501 | Avg PSNR/SSIM: 34.2490/0.9774 | Time: 0.1038 Processing: motocross-bumps [ 34/50] Name: motocross-bumps | PSNR/SSIM: 36.8686/0.9824 | Avg PSNR/SSIM: 34.3152/0.9776 | Time: 0.1038 Processing: motocross-jump [ 35/50] Name: motocross-jump | PSNR/SSIM: 33.0214/0.9748 | Avg PSNR/SSIM: 34.2938/0.9775 | Time: 0.1036 Processing: motorbike [ 36/50] Name: motorbike | PSNR/SSIM: 38.7371/0.9948 | Avg PSNR/SSIM: 34.3715/0.9778 | Time: 0.1035 Processing: paragliding [ 37/50] Name: paragliding | PSNR/SSIM: 40.4857/0.9873 | Avg PSNR/SSIM: 34.5407/0.9781 | Time: 0.1034 Processing: paragliding-launch [ 38/50] Name: paragliding-launch | PSNR/SSIM: 40.8609/0.9934 | Avg PSNR/SSIM: 34.7345/0.9785 | Time: 0.1034 Processing: parkour [ 39/50] Name: parkour | PSNR/SSIM: 27.9332/0.9488 | Avg PSNR/SSIM: 34.4834/0.9775 | Time: 0.1037 Processing: rhino [ 40/50] Name: rhino | PSNR/SSIM: 26.9992/0.9556 | Avg PSNR/SSIM: 34.2428/0.9767 | Time: 0.1037 Processing: rollerblade [ 41/50] Name: rollerblade | PSNR/SSIM: 27.9731/0.9463 | Avg PSNR/SSIM: 34.1653/0.9764 | Time: 0.1037 Processing: scooter-black [ 42/50] Name: scooter-black | PSNR/SSIM: 34.4413/0.9947 | Avg PSNR/SSIM: 34.1695/0.9766 | Time: 0.1035 Processing: scooter-gray [ 43/50] Name: scooter-gray | PSNR/SSIM: 29.9107/0.9763 | Avg PSNR/SSIM: 34.0613/0.9766 | Time: 0.1037 Processing: soapbox [ 44/50] Name: soapbox | PSNR/SSIM: 40.9410/0.9965 | Avg PSNR/SSIM: 34.2845/0.9773 | Time: 0.1037 Processing: soccerball [ 45/50] Name: soccerball | PSNR/SSIM: 23.8108/0.9365 | Avg PSNR/SSIM: 34.1223/0.9767 | Time: 0.1036 Processing: stroller [ 46/50] Name: stroller | PSNR/SSIM: 33.4445/0.9861 | Avg PSNR/SSIM: 34.1029/0.9769 | Time: 0.1038 Processing: surf [ 47/50] Name: surf | PSNR/SSIM: 38.4391/0.9899 | Avg PSNR/SSIM: 34.1764/0.9771 | Time: 0.1038 Processing: swing [ 48/50] Name: swing | PSNR/SSIM: 35.1501/0.9866 | Avg PSNR/SSIM: 34.1941/0.9773 | Time: 0.1037 Processing: tennis [ 49/50] Name: tennis | PSNR/SSIM: 39.6107/0.9901 | Avg PSNR/SSIM: 34.3065/0.9776 | Time: 0.1037 Processing: train [ 50/50] Name: train | PSNR/SSIM: 29.3233/0.9599 | Avg PSNR/SSIM: 34.1911/0.9772 | Time: 0.1038 Traceback (most recent call last): The result is not the same as the performance reported in the paper.

Sharpiless commented 3 months ago

Same question: Pretrained flow completion model has loaded... Pretrained ProPainter has loaded... Network [InpaintGenerator] was created. Total number of parameters: 39.4 million. To see the architecture, do print(network). Start evaluation ... [Loading I3D model from weights/i3d_rgb_imagenet.pt for FID score ..] Processing: bear [ 1/50] Name: bear | PSNR/SSIM: 32.9378/0.9739 | Avg PSNR/SSIM: 32.9378/0.9739 | Time: 0.1183 Processing: blackswan [ 2/50] Name: blackswan | PSNR/SSIM: 31.2961/0.9438 | Avg PSNR/SSIM: 32.3160/0.9625 | Time: 0.0909 Processing: bmx-bumps [ 3/50] Name: bmx-bumps | PSNR/SSIM: 38.9452/0.9797 | Avg PSNR/SSIM: 35.0035/0.9695 | Time: 0.0831 Processing: bmx-trees [ 4/50] Name: bmx-trees | PSNR/SSIM: 43.2590/0.9916 | Avg PSNR/SSIM: 37.1904/0.9753 | Time: 0.0788 Processing: boat [ 5/50] Name: boat | PSNR/SSIM: 28.0768/0.8987 | Avg PSNR/SSIM: 35.3773/0.9601 | Time: 0.0772 Processing: breakdance [ 6/50] Name: breakdance | PSNR/SSIM: 25.5336/0.9496 | Avg PSNR/SSIM: 33.5837/0.9582 | Time: 0.0759 Processing: breakdance-flare [ 7/50] Name: breakdance-flare | PSNR/SSIM: 30.4086/0.9576 | Avg PSNR/SSIM: 33.1599/0.9581 | Time: 0.0755 Processing: bus [ 8/50] Name: bus | PSNR/SSIM: 34.9377/0.9864 | Avg PSNR/SSIM: 33.3923/0.9618 | Time: 0.0743 Processing: camel [ 9/50] Name: camel | PSNR/SSIM: 26.1054/0.8963 | Avg PSNR/SSIM: 32.4581/0.9534 | Time: 0.0739 Processing: car-roundabout [ 10/50] Name: car-roundabout | PSNR/SSIM: 39.7409/0.9950 | Avg PSNR/SSIM: 33.1611/0.9574 | Time: 0.0734 Processing: car-shadow [ 11/50] Name: car-shadow | PSNR/SSIM: 48.1064/0.9918 | Avg PSNR/SSIM: 33.8928/0.9591 | Time: 0.0721 Processing: car-turn [ 12/50] Name: car-turn | PSNR/SSIM: 33.6250/0.9670 | Avg PSNR/SSIM: 33.8689/0.9598 | Time: 0.0716 Processing: cows [ 13/50] Name: cows | PSNR/SSIM: 30.0791/0.9633 | Avg PSNR/SSIM: 33.4752/0.9602 | Time: 0.0715 Processing: dance-jump [ 14/50] Name: dance-jump | PSNR/SSIM: 33.8284/0.9680 | Avg PSNR/SSIM: 33.4951/0.9606 | Time: 0.0707 Processing: dance-twirl [ 15/50] Name: dance-twirl | PSNR/SSIM: 31.4384/0.9861 | Avg PSNR/SSIM: 33.3343/0.9626 | Time: 0.0704 Processing: dog [ 16/50] Name: dog | PSNR/SSIM: 36.7435/0.9727 | Avg PSNR/SSIM: 33.5032/0.9631 | Time: 0.0697 Processing: dog-agility [ 17/50] Name: dog-agility | PSNR/SSIM: 24.7486/0.9299 | Avg PSNR/SSIM: 33.3262/0.9624 | Time: 0.0694 Processing: drift-chicane [ 18/50] Name: drift-chicane | PSNR/SSIM: 31.6650/0.9379 | Avg PSNR/SSIM: 33.2591/0.9614 | Time: 0.0693 Processing: drift-straight [ 19/50] Name: drift-straight | PSNR/SSIM: 35.9370/0.9814 | Avg PSNR/SSIM: 33.3592/0.9622 | Time: 0.0689 Processing: drift-turn [ 20/50] Name: drift-turn | PSNR/SSIM: 33.9535/0.9829 | Avg PSNR/SSIM: 33.3863/0.9631 | Time: 0.0690 Processing: elephant [ 21/50] Name: elephant | PSNR/SSIM: 35.5645/0.9764 | Avg PSNR/SSIM: 33.5039/0.9638 | Time: 0.0688 Processing: flamingo [ 22/50] Name: flamingo | PSNR/SSIM: 34.8372/0.9735 | Avg PSNR/SSIM: 33.5722/0.9643 | Time: 0.0688 Processing: goat [ 23/50] Name: goat | PSNR/SSIM: 32.3722/0.9731 | Avg PSNR/SSIM: 33.5068/0.9648 | Time: 0.0688 Processing: hike [ 24/50] Name: hike | PSNR/SSIM: 39.8412/0.9823 | Avg PSNR/SSIM: 33.7994/0.9656 | Time: 0.0691 Processing: hockey [ 25/50] Name: hockey | PSNR/SSIM: 43.8794/0.9915 | Avg PSNR/SSIM: 34.2177/0.9667 | Time: 0.0690 Processing: horsejump-high [ 26/50] Name: horsejump-high | PSNR/SSIM: 26.2650/0.9510 | Avg PSNR/SSIM: 34.0036/0.9663 | Time: 0.0687 Processing: horsejump-low [ 27/50] Name: horsejump-low | PSNR/SSIM: 29.7927/0.9614 | Avg PSNR/SSIM: 33.8718/0.9661 | Time: 0.0684 Processing: kite-surf [ 28/50] Name: kite-surf | PSNR/SSIM: 35.0413/0.9672 | Avg PSNR/SSIM: 33.9015/0.9661 | Time: 0.0683 Processing: kite-walk [ 29/50] Name: kite-walk | PSNR/SSIM: 43.2598/0.9859 | Avg PSNR/SSIM: 34.2673/0.9669 | Time: 0.0683 Processing: libby [ 30/50] Name: libby | PSNR/SSIM: 44.6968/0.9945 | Avg PSNR/SSIM: 34.5111/0.9676 | Time: 0.0681 Processing: lucia [ 31/50] Name: lucia | PSNR/SSIM: 32.7621/0.9827 | Avg PSNR/SSIM: 34.4546/0.9681 | Time: 0.0681 Processing: mallard-fly [ 32/50] Name: mallard-fly | PSNR/SSIM: 32.3197/0.9481 | Avg PSNR/SSIM: 34.3877/0.9674 | Time: 0.0681 Processing: mallard-water [ 33/50] Name: mallard-water | PSNR/SSIM: 29.7411/0.9547 | Avg PSNR/SSIM: 34.2272/0.9670 | Time: 0.0681 Processing: motocross-bumps [ 34/50] Name: motocross-bumps | PSNR/SSIM: 36.7541/0.9791 | Avg PSNR/SSIM: 34.2910/0.9673 | Time: 0.0679 Processing: motocross-jump [ 35/50] Name: motocross-jump | PSNR/SSIM: 32.8460/0.9675 | Avg PSNR/SSIM: 34.2671/0.9673 | Time: 0.0678 Processing: motorbike [ 36/50] Name: motorbike | PSNR/SSIM: 38.7673/0.9903 | Avg PSNR/SSIM: 34.3458/0.9677 | Time: 0.0677 Processing: paragliding [ 37/50] Name: paragliding | PSNR/SSIM: 40.6973/0.9835 | Avg PSNR/SSIM: 34.5216/0.9681 | Time: 0.0677 Processing: paragliding-launch [ 38/50] Name: paragliding-launch | PSNR/SSIM: 41.1072/0.9789 | Avg PSNR/SSIM: 34.7236/0.9685 | Time: 0.0677 Processing: parkour [ 39/50] Name: parkour | PSNR/SSIM: 27.9517/0.9416 | Avg PSNR/SSIM: 34.4736/0.9675 | Time: 0.0678 Processing: rhino [ 40/50] Name: rhino | PSNR/SSIM: 27.0301/0.9521 | Avg PSNR/SSIM: 34.2342/0.9670 | Time: 0.0678 Processing: rollerblade [ 41/50] Name: rollerblade | PSNR/SSIM: 27.7262/0.9529 | Avg PSNR/SSIM: 34.1539/0.9668 | Time: 0.0679 Processing: scooter-black [ 42/50] Name: scooter-black | PSNR/SSIM: 34.4450/0.9896 | Avg PSNR/SSIM: 34.1582/0.9671 | Time: 0.0680 Processing: scooter-gray [ 43/50] Name: scooter-gray | PSNR/SSIM: 29.8826/0.9670 | Avg PSNR/SSIM: 34.0496/0.9671 | Time: 0.0681 Processing: soapbox [ 44/50] Name: soapbox | PSNR/SSIM: 41.0077/0.9935 | Avg PSNR/SSIM: 34.2754/0.9680 | Time: 0.0681 Processing: soccerball [ 45/50] Name: soccerball | PSNR/SSIM: 23.7201/0.9212 | Avg PSNR/SSIM: 34.1119/0.9673 | Time: 0.0679 Processing: stroller [ 46/50] Name: stroller | PSNR/SSIM: 33.3572/0.9753 | Avg PSNR/SSIM: 34.0903/0.9675 | Time: 0.0681 Processing: surf [ 47/50] Name: surf | PSNR/SSIM: 38.2647/0.9858 | Avg PSNR/SSIM: 34.1611/0.9678 | Time: 0.0680 Processing: swing [ 48/50] Name: swing | PSNR/SSIM: 35.1041/0.9812 | Avg PSNR/SSIM: 34.1782/0.9681 | Time: 0.0678 Processing: tennis [ 49/50] Name: tennis | PSNR/SSIM: 39.6474/0.9883 | Avg PSNR/SSIM: 34.2917/0.9685 | Time: 0.0678 Processing: train [ 50/50] Name: train | PSNR/SSIM: 28.9433/0.9326 | Avg PSNR/SSIM: 34.1678/0.9676 | Time: 0.0678 Finish evaluation... Average Frame PSNR/SSIM/VFID: 34.17/0.9676/0.098 | Time: 0.0678