ali-vilab / videocomposer

Official repo for VideoComposer: Compositional Video Synthesis with Motion Controllability
https://videocomposer.github.io
MIT License
902 stars 81 forks source link

Error while processing rearrange-reduction pattern "(i j) c f h w -> c f (i h) (j w)". Input tensor shape: torch.Size([3, 16, 256, 256]). Additional info: {'i': 1}. #12

Open Yaomingshuai opened 1 year ago

Yaomingshuai commented 1 year ago
image
mumukawayi commented 1 year ago

same problem

wuleibegreat commented 1 year ago

same problem

mumukawayi commented 1 year ago

I guess this may caused by unappropriate torch version. I met this problem and solved it by refering #15 to rebuild the environment.

SSUHan commented 1 year ago

Is this issue solved? I encounter the same problem. If we downgrade torch version from 2.0 to 1.13 <, xformers are not compatible with it :(

mumukawayi commented 1 year ago

Is this issue solved? I encounter the same problem. If we downgrade torch version from 2.0 to 1.13 <, xformers are not compatible with it :(

I solved it by following #15 to downgrade torch and rebuild the environment. xformers will conflict with the downgraded torch but anyway the code can still process and output the gif.

SSUHan commented 1 year ago

In my case, the model could output a gif but it is just blacked out like this: rank_1-0 Is it OK in your case? @mumukawayi

mumukawayi commented 1 year ago

In my case, the model could output a gif but it is just blacked out like this: rank_1-0 Is it OK in your case? @mumukawayi

In my case, the model can successfully produce the moving scene of the moonlight. rank_1-0

SSUHan commented 1 year ago

I finally find a way,

def _i(tensor, t, x):
    r"""Index tensor using t and format the output according to x.
    """
    shape = (x.size(0), ) + (1, ) * (x.ndim - 1)
    return tensor.to(x)[t].view(shape).to(x)

This works for me. But I still wonder why the below code does not work:

def _i(tensor, t, x):
    r"""Index tensor using t and format the output according to x.
    """
    shape = (x.size(0), ) + (1, ) * (x.ndim - 1)
    tensor = tensor.to(t)
    return tensor[t].view(shape).to(x)
handsomeme001 commented 1 year ago

After I made modifications to the code, the problem was resolved

def video_tensor_to_gif(tensor, path, duration = 120, loop = 0, optimize = True):
    tensor = tensor.permute(1,2,3,0)
    images = tensor.unbind(dim = 0)
    images = [(image.numpy()*255).astype('uint8') for image in images]
    # imageio.mimwrite(path, images, fps=8)
    # return images
    imageio.mimsave(path, images, 'GIF', duration = duration)
    return path

This works for me.

Yinhance commented 10 months ago

I finally find a way,

def _i(tensor, t, x):
    r"""Index tensor using t and format the output according to x.
    """
    shape = (x.size(0), ) + (1, ) * (x.ndim - 1)
    return tensor.to(x)[t].view(shape).to(x)

This works for me. But I still wonder why the below code does not work:

def _i(tensor, t, x):
    r"""Index tensor using t and format the output according to x.
    """
    shape = (x.size(0), ) + (1, ) * (x.ndim - 1)
    tensor = tensor.to(t)
    return tensor[t].view(shape).to(x)

I met the same error, would you mind tell me where you change this function?