Doubiiu
commented
4 months ago Hi! Thanks for your comments!
- We will resize and crop the input image into the resolution of 576x1024 and will fill gray pixel to missing pixels. You can crop the original image (1152x832) into the size of (1152x648), then the aspect ratio of the input image should be perfectly 16:9 (larger than 16:9 is also OK, i.e. longer image) and we will not be filled with gray pixels.
- We train the model only with 16 frames video clips (2s, FPS=8). You can try to change the
video_lengthparameter from 16 to 32 inrun.shto synthesize longer videos (4s, FPS=8), however the performance is not guaranteed. - Thanks for your comments! Could you share your code? I will add that parameter to the config.
I provide you the code to automatically preprocess your input image to a specific size (i.e. 1024x576) by center cropping to avoid filling gray pixels by our method:
from PIL import Image
import os
def resize_and_crop(img, size, path):
img_width, img_height = img.size
width, height = size
if img_width / width < img_height / height:
new_width = width
new_height = img_height * width // img_width
if new_height < height:
new_width = img_width * height // img_height
new_height = height
else:
new_width = img_width * height // img_height
new_height = height
if new_width < width:
new_width = width
new_height = img_height * width // img_width
img = img.resize((new_width, new_height))
left = (new_width - width) / 2
top = (new_height - height) / 2
right = (new_width + width) / 2
bottom = (new_height + height) / 2
img = img.crop((left, top, right, bottom))
img.save(path)
def process_images(directory,save_dir,size):
os.makedirs(save_dir, exist_ok=True)
for filename in os.listdir(directory):
if filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"):
img = Image.open(os.path.join(directory, filename))
resize_and_crop(img, size, os.path.join(save_dir, filename.replace('.jpg', '.png').replace('.jpeg', '.png')))
# process_images("./your_folder_name_of_images", "./img2video_case320_512", (512,320))
process_images("./your_folder_name_of_images", "./img2video_case576_1024", (1024,576))
shirubei
alexandrtsk
Hello, thanks for your great job and open source. Just created 2s video from 1152*832 image with a 2080Ti 22GB VR, using 560s.
The motion part is perfect, but sometimes imperfection can be found like fames in this video. Two gray stripes in both side of the frame. Original image:
https://github.com/Doubiiu/DynamiCrafter/assets/37259748/7559553c-507b-4c63-b62a-77f219edae70
And one more question, can I create more longer videos? such as 3s or 4s. Finally, I have change the format of H264 to H265 just by modified func.py for H264 videos look like still images (no motion effect） So maybe you can make video format as a parameter that can be set in the config file.