search

vt-vl-lab / 3d-photo-inpainting

[CVPR 2020] 3D Photography using Context-aware Layered Depth Inpainting
https://shihmengli.github.io/3D-Photo-Inpainting/
Other
6.9k stars 1.11k forks source link

[Colab] Only ever get dolly zoom videos #164

Open CrimsonCuttle opened 2 years ago

CrimsonCuttle commented 2 years ago

I only ever get one of the four video types as a result, and that is always the dolly zoom one. The only argument in argument.yml relating to video output that I can find is video_postfix, which has ['dolly-zoom-in', 'zoom-in', 'circle', 'swing'] as the set value. By the looks of it, all four are there? Why am I only getting dolly-zoom-in as a result? Here is the log for execution:

running on device 0
  0% 0/1 [00:00<?, ?it/s]Current Source ==>  fuck
Running depth extraction at 1657177463.9234424
BoostingMonocularDepth/inputs/*.png
BoostingMonocularDepth/outputs/*.png
device: cuda
Namespace(Final=True, R0=False, R20=False, colorize_results=False, data_dir='inputs/', depthNet=0, max_res=inf, net_receptive_field_size=None, output_dir='outputs', output_resolution=1, pix2pixsize=1024, savepatchs=0, savewholeest=0)
----------------- Options ---------------
                    Final: True                             [default: False]
                       R0: False                         
                      R20: False                         
             aspect_ratio: 1.0                           
               batch_size: 1                             
          checkpoints_dir: ./pix2pix/checkpoints         
         colorize_results: False                         
                crop_size: 672                           
                 data_dir: inputs/                          [default: None]
                 dataroot: None                          
             dataset_mode: depthmerge                    
                 depthNet: 0                                [default: None]
                direction: AtoB                          
          display_winsize: 256                           
                    epoch: latest                        
                     eval: False                         
            generatevideo: None                          
                  gpu_ids: 0                             
                init_gain: 0.02                          
                init_type: normal                        
                 input_nc: 2                             
                  isTrain: False                            [default: None]
                load_iter: 0                                [default: 0]
                load_size: 672                           
         max_dataset_size: 10000                         
                  max_res: inf                           
                    model: pix2pix4depth                 
               n_layers_D: 3                             
                     name: void                          
                      ndf: 64                            
                     netD: basic                         
                     netG: unet_1024                     
 net_receptive_field_size: None                          
                      ngf: 64                            
               no_dropout: False                         
                  no_flip: False                         
                     norm: none                          
                 num_test: 50                            
              num_threads: 4                             
               output_dir: outputs                          [default: None]
                output_nc: 1                             
        output_resolution: None                          
                    phase: test                          
              pix2pixsize: None                          
               preprocess: resize_and_crop               
                savecrops: None                          
             savewholeest: None                          
           serial_batches: False                         
                   suffix:                               
                  verbose: False                         
----------------- End -------------------
initialize network with normal
loading the model from ./pix2pix/checkpoints/mergemodel/latest_net_G.pth
Loading weights:  midas/model.pt
Using cache found in /root/.cache/torch/hub/facebookresearch_WSL-Images_main
start processing
processing image 0 : fuck
     wholeImage being processed in : 2880
         DEBUG| GPU THRESHOLD REACHED 2880 ---> 1568
Adjust factor is: 1.0
Selecting patchs ...
Target resolution:  (5760, 5760, 3)
Dynamicly change merged-in resolution; scale: 0.17777777777777778
     Resulted depthmap res will be : (1024, 1024)
patchs to process: 135
     processing patch 0 | [ 34  34 990 990]
     processing patch 1 | [  0   0 956 956]
     processing patch 2 | [  0  51 956 956]
     processing patch 3 | [ 51   0 956 956]
     processing patch 4 | [ 34 137 887 887]
     processing patch 5 | [ 85 137 887 887]
     processing patch 6 | [137  34 887 887]
     processing patch 7 | [137  85 887 887]
     processing patch 8 | [137 137 887 887]
     processing patch 9 | [  0   0 853 853]
     processing patch 10 | [  0  51 853 853]
     processing patch 11 | [  0 102 853 853]
     processing patch 12 | [  0 154 853 853]
     processing patch 13 | [ 51   0 853 853]
     processing patch 14 | [102   0 853 853]
     processing patch 15 | [154   0 853 853]
     processing patch 16 | [ 34 239 785 785]
     processing patch 17 | [ 85 239 785 785]
     processing patch 18 | [137 239 785 785]
     processing patch 19 | [188 239 785 785]
     processing patch 20 | [239  34 785 785]
     processing patch 21 | [239  85 785 785]
     processing patch 22 | [239 137 785 785]
     processing patch 23 | [239 188 785 785]
     processing patch 24 | [239 239 785 785]
     processing patch 25 | [  0  51 751 751]
     processing patch 26 | [  0 102 751 751]
     processing patch 27 | [  0 154 751 751]
     processing patch 28 | [  0 205 751 751]
     processing patch 29 | [  0 256 751 751]
     processing patch 30 | [256   0 751 751]
     processing patch 31 | [ 34 341 683 683]
     processing patch 32 | [ 85 341 683 683]
     processing patch 33 | [137 341 683 683]
     processing patch 34 | [188 341 683 683]
     processing patch 35 | [239 341 683 683]
     processing patch 36 | [290 341 683 683]
     processing patch 37 | [341  34 683 683]
     processing patch 38 | [341  85 683 683]
     processing patch 39 | [341 137 683 683]
     processing patch 40 | [341 188 683 683]
     processing patch 41 | [341 239 683 683]
     processing patch 42 | [341 290 683 683]
     processing patch 43 | [341 341 683 683]
     processing patch 44 | [  0  51 649 649]
     processing patch 45 | [  0 102 649 649]
     processing patch 46 | [  0 154 649 649]
     processing patch 47 | [  0 205 649 649]
     processing patch 48 | [  0 256 649 649]
     processing patch 49 | [  0 307 649 649]
     processing patch 50 | [290 444 580 580]
     processing patch 51 | [444  85 580 580]
     processing patch 52 | [444 137 580 580]
     processing patch 53 | [444 188 580 580]
     processing patch 54 | [444 239 580 580]
     processing patch 55 | [444 290 580 580]
     processing patch 56 | [444 341 580 580]
     processing patch 57 | [444 393 580 580]
     processing patch 58 | [  0 154 546 546]
     processing patch 59 | [  0 205 546 546]
     processing patch 60 | [  0 256 546 546]
     processing patch 61 | [  0 307 546 546]
     processing patch 62 | [  0 358 546 546]
     processing patch 63 | [546 188 478 478]
     processing patch 64 | [546 239 478 478]
     processing patch 65 | [546 290 478 478]
     processing patch 66 | [546 341 478 478]
     processing patch 67 | [546 393 478 478]
     processing patch 68 | [546 444 478 478]
     processing patch 69 | [  0 205 444 444]
     processing patch 70 | [  0 256 444 444]
     processing patch 71 | [  0 307 444 444]
     processing patch 72 | [  0 358 444 444]
     processing patch 73 | [  0 410 444 444]
     processing patch 74 | [649 239 375 375]
     processing patch 75 | [649 290 375 375]
     processing patch 76 | [649 341 375 375]
     processing patch 77 | [649 393 375 375]
     processing patch 78 | [649 444 375 375]
     processing patch 79 | [  0 256 341 341]
     processing patch 80 | [  0 307 341 341]
     processing patch 81 | [  0 358 341 341]
     processing patch 82 | [  0 410 341 341]
     processing patch 83 | [751 290 273 273]
     processing patch 84 | [751 341 273 273]
     processing patch 85 | [751 393 273 273]
     processing patch 86 | [751 444 273 273]
     processing patch 87 | [751 495 273 273]
     processing patch 88 | [407 612 245 245]
     processing patch 89 | [  0 307 239 239]
     processing patch 90 | [  0 358 239 239]
     processing patch 91 | [  0 410 239 239]
     processing patch 92 | [  0 461 239 239]
     processing patch 93 | [629  14 210 210]
     processing patch 94 | [683  17 205 205]
     processing patch 95 | [589  26 188 188]
     processing patch 96 | [794  26 188 188]
     processing patch 97 | [643  80 182 182]
     processing patch 98 | [646 134 176 176]
     processing patch 99 | [646 236 176 176]
     processing patch 100 | [597  85 171 171]
     processing patch 101 | [853  34 171 171]
     processing patch 102 | [853  85 171 171]
     processing patch 103 | [853 137 171 171]
     processing patch 104 | [853 239 171 171]
     processing patch 105 | [853 290 171 171]
     processing patch 106 | [853 341 171 171]
     processing patch 107 | [853 393 171 171]
     processing patch 108 | [853 444 171 171]
     processing patch 109 | [853 495 171 171]
     processing patch 110 | [853 546 171 171]
     processing patch 111 | [242 242 165 165]
     processing patch 112 | [856 191 165 165]
     processing patch 113 | [247  43 154 154]
     processing patch 114 | [199 711 148 148]
     processing patch 115 | [404 660 148 148]
     processing patch 116 | [609 148 148 148]
     processing patch 117 | [609 250 148 148]
     processing patch 118 | [609 814 148 148]
     processing patch 119 | [253 202 142 142]
     processing patch 120 | [612 765 142 142]
     processing patch 121 | [663 202 142 142]
     processing patch 122 | [  0 358 137 137]
     processing patch 123 | [  0 410 137 137]
     processing patch 124 | [  0 461 137 137]
     processing patch 125 | [  0 512 137 137]
     processing patch 126 | [  0 563 137 137]
     processing patch 127 | [  0 614 137 137]
     processing patch 128 | [205   0 137 137]
     processing patch 129 | [256   0 137 137]
     processing patch 130 | [614   0 137 137]
     processing patch 131 | [666   0 137 137]
     processing patch 132 | [717   0 137 137]
     processing patch 133 | [819   0 137 137]
     processing patch 134 | [870   0 137 137]
finished
Start Running 3D_Photo ...
Loading edge model at 1657177590.1380754
Loading depth model at 1657177593.1767423
Loading rgb model at 1657177594.6196961
Writing depth ply (and basically doing everything) at 1657177595.989959
Writing mesh file mesh/fuck.ply ...
Making video at 1657177943.9706874
fov: 53.13010235415598
Moviepy - Building video video/fuck_dolly-zoom-in.mp4.
Moviepy - Writing video video/fuck_dolly-zoom-in.mp4

  0% 0/1 [08:40<?, ?it/s]
t:   0% 0/241 [00:00<?, ?it/s, now=None]
t:   1% 3/241 [00:00<00:13, 18.14it/s, now=None]
t:   8% 19/241 [00:00<00:02, 82.76it/s, now=None]
t:  15% 35/241 [00:00<00:01, 110.86it/s, now=None]
t:  20% 48/241 [00:00<00:02, 80.57it/s, now=None] 
t:  24% 58/241 [00:00<00:03, 53.03it/s, now=None]
t:  27% 66/241 [00:01<00:04, 42.04it/s, now=None]
t:  30% 72/241 [00:01<00:04, 37.97it/s, now=None]
t:  32% 77/241 [00:01<00:04, 35.76it/s, now=None]
t:  34% 82/241 [00:01<00:04, 34.64it/s, now=None]
t:  36% 86/241 [00:01<00:04, 32.70it/s, now=None]
t:  37% 90/241 [00:02<00:04, 30.88it/s, now=None]
t:  39% 94/241 [00:02<00:04, 29.53it/s, now=None]
t:  41% 98/241 [00:02<00:05, 28.11it/s, now=None]
t:  42% 101/241 [00:02<00:04, 28.32it/s, now=None]
t:  43% 104/241 [00:02<00:04, 27.42it/s, now=None]
t:  44% 107/241 [00:02<00:05, 26.13it/s, now=None]
t:  46% 111/241 [00:02<00:04, 26.80it/s, now=None]
t:  48% 115/241 [00:03<00:04, 26.67it/s, now=None]
t:  49% 119/241 [00:03<00:04, 27.25it/s, now=None]
t:  51% 123/241 [00:03<00:04, 27.40it/s, now=None]
t:  53% 127/241 [00:03<00:04, 27.41it/s, now=None]
t:  54% 131/241 [00:03<00:03, 27.65it/s, now=None]
t:  56% 135/241 [00:03<00:03, 26.90it/s, now=None]
t:  58% 139/241 [00:03<00:03, 27.18it/s, now=None]
t:  59% 143/241 [00:04<00:03, 27.14it/s, now=None]
t:  61% 147/241 [00:04<00:03, 27.29it/s, now=None]
t:  63% 151/241 [00:04<00:03, 27.37it/s, now=None]
t:  64% 155/241 [00:04<00:03, 27.21it/s, now=None]
t:  66% 159/241 [00:04<00:03, 27.14it/s, now=None]
t:  68% 163/241 [00:04<00:02, 26.87it/s, now=None]
t:  69% 167/241 [00:04<00:02, 26.84it/s, now=None]
t:  71% 171/241 [00:05<00:02, 27.06it/s, now=None]
t:  73% 175/241 [00:05<00:02, 27.29it/s, now=None]
t:  74% 179/241 [00:05<00:02, 27.34it/s, now=None]
t:  76% 182/241 [00:05<00:02, 27.93it/s, now=None]
t:  77% 186/241 [00:05<00:01, 30.21it/s, now=None]
t:  79% 190/241 [00:05<00:01, 27.57it/s, now=None]
t:  80% 193/241 [00:05<00:01, 26.85it/s, now=None]
t:  81% 196/241 [00:06<00:01, 26.42it/s, now=None]
t:  83% 199/241 [00:06<00:01, 25.42it/s, now=None]
t:  84% 203/241 [00:06<00:01, 25.41it/s, now=None]
t:  86% 207/241 [00:06<00:01, 25.77it/s, now=None]
t:  88% 211/241 [00:06<00:01, 26.75it/s, now=None]
t:  89% 215/241 [00:06<00:01, 25.73it/s, now=None]
t:  91% 219/241 [00:06<00:00, 26.14it/s, now=None]
t:  93% 223/241 [00:07<00:00, 27.03it/s, now=None]
t:  94% 227/241 [00:07<00:00, 26.77it/s, now=None]
t:  96% 231/241 [00:07<00:00, 27.22it/s, now=None]
t:  98% 235/241 [00:07<00:00, 27.95it/s, now=None]
t:  99% 239/241 [00:07<00:00, 27.48it/s, now=None]
Moviepy - Done !
Moviepy - video ready video/fuck_dolly-zoom-in.mp4
  0% 0/1 [08:49<?, ?it/s]^C
CrimsonCuttle commented 2 years ago

Downscaled the image and tried again and confirmed it has to do with image size? Why is this? It's a shame I can only get past dolly-zoom-in if I'm using a 500x500p image

Michaelwhite34 commented 2 years ago

Downscaled the image and tried again and confirmed it has to do with image size? Why is this? It's a shame I can only get past dolly-zoom-in if I'm using a 500x500p image

The script stopped "^C", maybe it runs too long, you can run it again.

CrimsonCuttle commented 2 years ago

Downscaled the image and tried again and confirmed it has to do with image size? Why is this? It's a shame I can only get past dolly-zoom-in if I'm using a 500x500p image

The script stopped "^C", maybe it runs too long, you can run it again.

Doing so just gives me the dolly zoom video once more and then stops again