DeOldify.NET

C# implementation of Jason Antic's DeOldify(https://github.com/jantic/DeOldify) Only for photos for now!

Paper "DeOldify.NET: cross-platform application for coloring black and white photos" was accepted to poster session of Neuroinformatics - 2022 conference. The paper describes technical details of managed C# implementation of the original DeOldify and contains some comparisons with different other image colorization products.

How to run

On Windows 7, 8, 8.1, 10, 11

• Make sure that .NET Framework 4.5+ (4.6+ for SIMD-accelerated version) or higher is installed on your computer.

• You can use any bit depth (x32 or x64), but on a 32-bit system you will not be able to process large images due to the limited amount of memory.

• SIMD and Stable model are supported only in 64-bit mode. On a 32-bit machine, you should use the regular artistic version.

• At least 3 GB 1.5 GB with new convolution algorithm of free RAM is required to run Artistic model. About 3 GB is required for Stable model.

• Select a version of DeOldify.NET the one you want to build. Versions with and without simd are available, with float32 typed weights (higher accuracy) and float16 typed weights (lower accuracy and smaller file size), with a stable model and an artistic model, as in the original DeOldify. Further actions depend on your choice.

• Download and unpack the repository, then download model from the releases (https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights) and place it in Implementation\src\Resources.

• Select compilation script and run it.

• The executable file will appear in the Implementation\Release folder. The application is ready to work!

• Use!

On Linux (Tested on Mint)

• We recommend that the first step is to update everything. It may take time, but it's worth it:

  sudo apt-get update
  sudo apt-get upgrade

• Install Mono:

  sudo apt-get install mono-complete

• Get sources; select and download model

• Select and run compilation script

• The executable file will appear in the Implementation/Release folder. The application is ready to work!

• Run application using mono <build name>.exe command in terminal or double click as in Windows.

• Use!

Please note, that DeOldify.NET using Mono is a bit slower, than using .NET Framework

Examples

Original	Artistic	Stable

New algorithms

DeOldify.NET has become a platform for testing the latest highly optimized algorithms, which will then be applied in the System.AI project. In this section you can find some information about the results of the experiments.

Patch2Vec Conv2d

The meaning of most fast convolution algorithms, such as im2col or im2row, involves bringing the convolution to matrix multiplication, which allows optimizing memory access operations by using the processor cache. However, such methods either require a buffer for srcC * kernelY * kernelH * dstH * dstW elements, which is extremely irrational. The proposed patch2vec method unwraps each patch of the input image on the fly, and then applies all convolution filters to it. This implementation is not inferior in efficiency to classical algorithms like im2col, and in practice even surpasses them. The buffer for this algorithm will have the size of srcC * kernelY * kernelX, which is much smaller than in the case of similar methods. Moreover, patch2vec does not impose restrictions on the convolution parameters, unlike, for example, the Shmuel Vinograd method. The proposed algorithm is difficult to fit into classical machine learning frameworks due to the fact that they are focused on using GEMM as the core. Pure C#-based implementations make it easy to do this.

For more detailed information, please see official Patch2Vec repository: https://github.com/GlebSBrykin/Patch2Vec

In DeOldify.NET two versions of the patch2vec conv2d algorithm are implemented - with and without SIMD support. You can choose, which version to use by executing the corresponding compilation command file. Vectorization is implemented through the Vector4 structure of the System.Numerics namespace. Vectorization is only available for x86-64 processors at version .NET Framework 4.6 and higher, or when using Mono newer than 2008.

All tests was done in Windows 7 x64 laptop with Intel(R) Core(TM) i5-6300HQ CPU and 32 GB of RAM

Updates

• [10.09.2022] - Version 2.1 has been released. The ability to drag and drop images into the application has been added, error handling has been improved, and the process of saving the result has been simplified.
• [20.07.2022] - The Windows and Linux versions are combined.
• [11.07.2022] - DeOldify.NET now supports Stable model from original DeOldify. Added the ability to build DeOldify.NET with uncompressed original weights to obtain the maximum quality of coloring.
• [27.04.2022] - DeOldify.NET has become a testing ground for the latest optimized algorithms. The Conv2d layer has been optimized and now requires significantly less memory. Support for SIMD vectorization will allow you to get about a fourfold increase in performance.
• [29.10.2021] - Big refactoring and code clean up
• [29.10.2021] - Linux support
• [16.09.2021] - Fixed a memory leak issue