Improvements, and a bug

[ghost]

Thanks for getting it working, love it!

Using it I've come up with a couple of improvements that could enhance the experience :

1- Option to export as video with ffmpeg (with options for FPS, export location/name, format) 2- Enable moving the timeline marker by clicking on the timeline, instead of just by dragging the marker 3- Adding settings tab, for options like defaults (sampler, cfg scale, steps), local data clean-up, model weights selection (from the list that A1111's end creates) 4- Enable batch creation, to speed up generation and interpolation in higher end GPUs (have the option of the batch size also in settings) 5- Adding the option to remove generated frames in playback.

Thinking of some heavier improvements: 6- Have different keyframes interpolations (exponential, quadratic, sinusoidal) 7- Integrate frame interpolation like https://github.com/google-research/frame-interpolation - on export time 8- Integrate super-resolution - also on export time

And a little bug, when dragging keyframes to the left (frame<0) causes a crash and forces reload.

Much love!

[amotile]

Thanks for the feedback here's some comments:

1. Yes, having the video generation part of the the system would be nice. But since you can collect all the frame in one folder in the correct order you not "locked" into any particular way to generate the video. But as a first step some instructions for how I do it would be good.
2. Good Suggestion
3. Good Suggestion
4. Good Suggestion
5. Why? To not include them in "collect"?
6. Not very hard to implement I think, but wasn't sure it would be very useful.
7. Same as 1. This is the one I use to make my videos.
8. Same as 1.

Bug: Opps, will fix: also if you drag the line between keyframes you can move it <0 (without crashing this time...)

[ghost]

1- I've found https://www.npmjs.com/package/fluent-ffmpeg to be the most popular wrapper for ffmpeg in node. In this case the command for the image sequence would be like this: var ffmpeg = require('fluent-ffmpeg'); var command = ffmpeg().addInput('/data/collected/frame%03d.png') .videoCodec(CODEC) .videoBitrate(BITRATE) .outputFPS(FPS) ...etc .save('/path/to/output.mp4');

5- I've found myself generating interpolated frames and then wanting to tune something and regenerate them. I'm thinking of the option to individually select the frames and having a trash bin button to erase them, because there's no other way to undo the process as of now. Maybe a checkbox that indicates the interpolator to create keyframes with the new generated frames? Maybe that's too messy.

6- Sometimes the interpolation is really abrupt while on the same timeline it gets slow. I know it's got something to do with the latent space not being smooth. Having controls for the interpolation speed would help, something like the curves that video editors have. I know implementing something like that would get a bit complicated 😅

[amotile]

5: even if you could regenerate them you would get the same result. Because all the inputs are the same. If the inputs are not the same it would not be there. (unless there's a bug)

[ghost]

Sorry, I didn't explain myself. I was saying deleting even the old keyframe to regenerate with a new one

[amotile]

I'm not sure I understand. Do you mean the "frames" aka the images generated by SD or the "keyframes" aka the "diamonds that set a value for a property" ?

[ghost]

Sorry, I mean both. If I wanted a different output in the interpolation, I'd need to change the keyframes like you said. Also I'd need to delete the generated interpolation

[amotile]

You should not have to delete the generated interpolation, if you for instance change the "seed" keyframes, all generated frames should "go away".

[ghost]

Oh i see, didn't know that! Then what i would suggest is to have the option to keep a generated frame (i.e. keyframe it) in case the user wants to keep any of the interpolated frames. One more thing I just thought of, a play and pause button on the playback tab!

[amotile]

The are "keept" in the sense that if you retern to the old settings you don't have to regenerate them. Play/Pause is "planned".