Add callback option.

[ZeroCool940711]

Maybe I didn't see it and it is already implemented but it would be nice to have a callback option which is executed every X number of steps, it would be useful for doing something like previewing the frame while it's being generated, on our WebUI we have a tab called "Text-To-Video" which is based on the script before this repo was made, we want to migrate that old code to use things from here as its better and more up to date than what we have but the problem we face is that we show the user a preview image every X number of steps, like for example every 10 steps, while it slows down the generations a bit this makes it so the user can see how the current frame looks during generation before it has finished so they can have a better idea of how things look instead of wasting a lot of time generating stuff without having any idea how it will look, it also helps with just showing progress, it seems like users prefer to be able to see how things look rather than having things go faster during the generation, so, a callback returning the current image or anything we can use would be enough for us, without it, we would probably need to break our heads around in order to make things work. I would appreciate if this feature is added or if it is already part of the code then I would appreciate some pointers on how to use it. Thanks for your time.

[nateraw]

Can for sure do or add something like this. There is a callback option in the pipeline call fn already that I copied over from diffusers. I haven't used it/really looked at it much though so will check to see if it will fit your use case.

[nateraw]

So there is a callback in the __call__ fn of the pipeline. What I imagine you really need is a yield there instead...

Let me know if the callback works for you or if you actually need the latter. :)

[nateraw]

If we do an optional yield, some code will need to be reworked to make it still make the video. would be preferable not to do it, or to have a separate fn/script for this where ya copy paste the pipeline and add what you need. That's why I shoved most of everything in the one file to begin with, so folks could do something like that.

btw here's the callback param and here's where it gets called.

[ZeroCool940711]

@nateraw sorry I keep bothering with this topic. I tried using the callback you mentioned but im not good enough with python and OOP to use it, I would appreciate if you could just make it two simpe arguments on the pipe.walk() function so I can just pass it there with the rest of the arguments during inference and then it call internally the __call__ function which seems to be the one doing exactly what I need, if I can just use something like this;

pipe.walk(
        prompts=['a cat'],
        seeds=[42],
        num_interpolation_steps=30,
        height=512,
        width=512,
        batch_size=4,
        fps=15,
        output_dir='dreams',
        name='animals_test',
        guidance_scale=7.5,
        num_inference_steps=50,
        callback=my_callback_function, # function which will be called during inference with the arguments: `callback(step: int, timestep: int, latents: torch.FloatTensor)`.
        callback_steps=10, # steps at which the callback will be called during inference.
    )

Another issue I found which im not sure if I should open a new issue here for that is the folder on the name argument from the previous example, I think when setting it to None it should not create a random folder name with the timestamp and just dump everything on the output_dir, we should also ahve an option to specify the video ouput name and format as well as the location where it should be saved in case we want it to be saved somewhere else other than the output_dir, this will allow us to have the frames output in one place and the video result in another, better control over those things is never a bad thing in my opinion.

[nateraw]

no worries! totally didn't realize I would need to surface this up in walk, my bad! yea we can do that.

Can you break down the other issues you mention in 2nd paragraph into separate new issues here, please? Preferably very specific and limited to one feature each so they're easier to track. They don't seem very difficult to add so I'll see what I can do.

Just as a heads up, I'm not allocating as much time to this project these days as I have other work to do, so if I'm a bit slow, just ping me in issues asking for update :)

[ZeroCool940711]

can you break down the other issues you mention in 2nd paragraph into separate new issues here, please?

@nateraw sure, will create a new issue for it, will probably make it with my specific use case but it will also be useful for other people and other use cases. Just give me a couple of minutes to do that issue before going to bed.

[nateraw]

No rush! Thank you