search

ColorfulSoft / DeOldify.NET

C# implementation of Jason Antic's DeOldify
Apache License 2.0
299 stars 57 forks source link

Include new Video Model #26

Closed janbl closed 1 year ago

janbl commented 1 year ago

Jason has trained a new model for Videos. Can we get the new Model as well?

Maybe you can also throw some light on the conversion process from the pytorch pth file to your model file and why you have chosen to put some of the file content into code.

GlebSBrykin commented 1 year ago

DeOldify.NET it does not support video processing, so there are no models for video in the list. The conversion of pth to custom files takes place in several stages: 1) spectral_form and weight_form are removed from the model in PyTorch (in fact, they are not just removed, but the weights and biases of convolutional layers are calculated). 2) All weights from pth are sequentially stored in a binary file without separators. At the same time, C# code is generated to read weights from this file. Since the data in the binary file is not structured, the weights must be read in the same sequence in which they were written. The generated C# code allows you to get a dictionary with named weights (like state_dict in PyTorch).