search

etched-ai / open-oasis

Inference script for Oasis 500M
MIT License
1.53k stars 128 forks source link

Run with Video as prompt Error：channels not match #37

Open Kevin-Lee1299 opened 1 week ago

Kevin-Lee1299 commented 1 week ago

python generate.py --prompt-path sample_data/snippy-chartreuse-mastiff-f79998db196d-20220401-224517.chunk_001.mp4 --actions-path sample_data/snippy-chartreuse-mastiff-f79998db196d-20220401-224517.chunk_001.one_hot_actions.pt

Traceback (most recent call last): File "D:\text\open-oasis\generate_origin.py", line 199, in main(args) File "D:\text\open-oasis\generate_origin.py", line 76, in main x = vae.encode(x 2 - 1).mean scaling_factor File "D:\text\open-oasis\vae.py", line 283, in encode x = self.patch_embed(x) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl return forward_call(*args, *kwargs) File "D:\text\open-oasis\dit.py", line 65, in forward x = self.proj(x) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\module.py", line 1736, in _wrapped_call_impl return self._call_impl(args, kwargs) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\conv.py", line 554, in forward return self._conv_forward(input, self.weight, self.bias) File "D:\text\open-oasis\ENV_DIR\lib\site-packages\torch\nn\modules\conv.py", line 549, in _conv_forward return F.conv2d( RuntimeError: Given groups=1, weight of size [1024, 3, 20, 20], expected input[1, 360, 360, 640] to have 3 channels, but got 360 channels instead