search

nagadomi / waifu2x

Image Super-Resolution for Anime-Style Art
http://waifu2x.udp.jp/
MIT License
27.54k stars 2.71k forks source link

How to prepare input data when settings.method=user? #193

Open BroJackfeely opened 7 years ago

BroJackfeely commented 7 years ago

I have paired dataset {x,y}, where x is the low-resolution images and y the corresponding high-resolution ones. I want to train my own model to achieve a super-resolution task. Therefore, I set the method as USER. How can I prepare my .t7 input data file?

nagadomi commented 7 years ago

-method user supports 1x and 2x mapping x -> y. First, prepare the directory where each image is stored, for example, ./x and ./y. tools/make_pairwise_list.lua is a helper script for generating the paired image list.

mkdir user_data
th tools/make_pairwise_list.lua -x_dir ./x -y_dir ./y > ./user_data/image_list.txt

make images.t7

th convert_data.lua -data_dir ./user_data

train (if your dataset x,y is the same size, specify -scale 1)

th train.lua -data_dir ./user_data -scale 2 -method user -test test_input_image.png  -name your_model_name -model_dir ./models/user

And, I recommend you to use dev branch.

EDIT: Sorry, I did not mention to -model option. When use -scale 2, specify -model upconv_7 or -model upcunet. When use -scale 1, specify -model vgg_7 or -model cunet.

Archit01 commented 6 years ago

i created two folders one with noisy and low resoultion images 512512 in x folder and there corresponding 2x high resoultion image 10241024 noise free in y folder and run the same commands as above but got this error `th train.lua -data_dir ./user_data -scale 2 -method user -test test.jpg -name photo2 -model_dir ./models/user -downsampling_filters "Box,Sinc,Catrom" -style photo -color rgb -random_unsharp_mask_rate 0.5 -thread 3 -nr_rate 1 -resize_blur_min 1 -resize_blur_max 1 { grayscale : false thread : 3 name : "photo2" loss : "huber" use_transparent_png : false model : "vgg_7" patches : 64 random_pairwise_negate_x_rate : 0 downsampling_filters : { 1 : "Box" 2 : "Sinc" 3 : "Catrom" } resume : "" crop_size : 48 random_pairwise_rotate_min : -6 random_blur_size : "3,5" random_pairwise_scale_max : 1.176 random_blur_rate : 0 random_pairwise_negate_rate : 0 random_pairwise_rotate_max : 6 oracle_drop_rate : 0.5 inner_epoch : 4 invert_x : false epoch : 50 update_criterion : "mse" random_overlay_rate : 0 jpeg_chroma_subsampling_rate : 0.5 image_list : "./user_data/image_list.txt" oracle_rate : 0.1 model_dir : "./models/user" backend : "cunn" random_pairwise_scale_min : 0.85 active_cropping_rate : 0.5 batch_size : 16 method : "user" max_size : 256 padding_y_zero : false plot : false save_history : false resize_blur_max : 1 gpu : { 1 : 1 } random_pairwise_scale_rate : 0 random_pairwise_rotate_rate : 0 random_color_noise_rate : 0 validation_filename_split : false resize_blur_min : 1 learning_rate_decay : 3e-07 images : "./user_data/images.t7" random_unsharp_mask_rate : 0.5 random_blur_sigma_min : 0.5 color : "rgb" pairwise_y_binary : false active_cropping_tries : 10 style : "photo" data_dir : "./user_data" random_half_rate : 0 model_file : "./models/user/photo2_model.t7" validation_rate : 0.05 pairwise_flip : true max_training_image_size : -1 noise_level : 1 random_blur_sigma_max : 1 validation_crops : 200 nr_rate : 1 learning_rate : 0.00025 scale : 2 seed : 11 test : "test.jpg" padding : 0 }

make validation-set

load .. 1

1

resampling

/home/archit/torch/install/bin/luajit: ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:179: [thread 3 endcallback] train.lua:377: inconsistent tensor size, expected tensor [3 x 48 x 48] and src [3 x 24 x 24] to have the same number of elements, but got 6912 and 1728 elements respectively at /home/archit/torch/pkg/torch/lib/TH/generic/THTensorCopy.c:86 stack traceback: [C]: in function 'copy' train.lua:377: in function [C]: in function 'xpcall' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:174: in function 'dojob' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:264: in function 'synchronize' train.lua:395: in function 'resampling' train.lua:557: in function 'train' train.lua:642: in main chunk [C]: in function 'dofile' ...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x55720ffac560 stack traceback: [C]: in function 'error' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:179: in function 'dojob' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:264: in function 'synchronize' train.lua:395: in function 'resampling' train.lua:557: in function 'train' train.lua:642: in main chunk [C]: in function 'dofile' ...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x55720ffac560 i tried changeing the orders of file high resolution images in x folder and low ones in y and got this errorth train.lua -data_dir ./user_data -scale 2 -method user -test test.jpg -name photo2 -model_dir ./models/user -downsampling_filters "Box,Sinc,Catrom" -style photo -color rgb -random_unsharp_mask_rate 0.5 -thread 3 -nr_rate 1 -resize_blur_min 1 -resize_blur_max 1 { grayscale : false thread : 3 name : "photo2" loss : "huber" use_transparent_png : false model : "vgg_7" patches : 64 random_pairwise_negate_x_rate : 0 downsampling_filters : { 1 : "Box" 2 : "Sinc" 3 : "Catrom" } resume : "" crop_size : 48 random_pairwise_rotate_min : -6 random_blur_size : "3,5" random_pairwise_scale_max : 1.176 random_blur_rate : 0 random_pairwise_negate_rate : 0 random_pairwise_rotate_max : 6 oracle_drop_rate : 0.5 inner_epoch : 4 invert_x : false epoch : 50 update_criterion : "mse" random_overlay_rate : 0 jpeg_chroma_subsampling_rate : 0.5 image_list : "./user_data/image_list.txt" oracle_rate : 0.1 model_dir : "./models/user" backend : "cunn" random_pairwise_scale_min : 0.85 active_cropping_rate : 0.5 batch_size : 16 method : "user" max_size : 256 padding_y_zero : false plot : false save_history : false resize_blur_max : 1 gpu : { 1 : 1 } random_pairwise_scale_rate : 0 random_pairwise_rotate_rate : 0 random_color_noise_rate : 0 validation_filename_split : false resize_blur_min : 1 learning_rate_decay : 3e-07 images : "./user_data/images.t7" random_unsharp_mask_rate : 0.5 random_blur_sigma_min : 0.5 color : "rgb" pairwise_y_binary : false active_cropping_tries : 10 style : "photo" data_dir : "./user_data" random_half_rate : 0 model_file : "./models/user/photo2_model.t7" validation_rate : 0.05 pairwise_flip : true max_training_image_size : -1 noise_level : 1 random_blur_sigma_max : 1 validation_crops : 200 nr_rate : 1 learning_rate : 0.00025 scale : 2 seed : 11 test : "test.jpg" padding : 0 }

make validation-set

/home/archit/torch/install/bin/luajit: ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:183: [thread 1 callback] bad argument #2 to '?' (end index out of bound at /home/archit/torch/pkg/torch/generic/Tensor.c:992) stack traceback: [C]: at 0x7f3d7b75b5b0 [C]: in function '__index' lib/iproc.lua:16: in function 'crop' lib/pairwise_transform_utils.lua:130: in function 'active_cropping' lib/pairwise_transform_user.lua:34: in function 'transformer' train.lua:261: in function [C]: in function 'xpcall' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:234: in function 'callback' /home/archit/torch/install/share/lua/5.1/threads/queue.lua:65: in function </home/archit/torch/install/share/lua/5.1/threads/queue.lua:41> [C]: in function 'pcall' /home/archit/torch/install/share/lua/5.1/threads/queue.lua:40: in function 'dojob' [string " local Queue = require 'threads.queue'..."]:15: in main chunk stack traceback: [C]: in function 'error' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:183: in function 'dojob' ...e/archit/torch/install/share/lua/5.1/threads/threads.lua:223: in function 'addjob' train.lua:259: in function 'make_validation_set' train.lua:503: in function 'train' train.lua:642: in main chunk [C]: in function 'dofile' ...chit/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x55cdb9b7e560 `

nagadomi commented 6 years ago

Sorry, in the above command, i did not mention to -model option. When use -scale 2, specify -model upconv_7 or -model resnet_14l(renset_14l is only available dev branch). When use -scale 1, specify -model vgg_7(it is default).

78Alpha commented 4 years ago

Just tried this out and it did not work.

/root/torch/install/bin/luajit: bad argument #1 to '?' (must be strictly positive at /root/torch/pkg/torch/lib/TH/generic/THTensorMath.c:2033) stack traceback: [C]: at 0x7f5482c32c90 [C]: in function 'randperm' tools/make_pairwise_list.lua:54: in function 'output' tools/make_pairwise_list.lua:108: in main chunk [C]: in function 'dofile' /root/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x56477add4570

Will be tinkering with it, but it should have just been copy and paste commands...

nagadomi commented 4 years ago

No paired image is found. ./x and ./y must contain paired images with the same name. like

x/1.png
x/abcd.png
...
y/1.png
y/abcd.png
78Alpha commented 4 years ago

They are indeed pairs. However, i am using the docker image, so it must be different. The docker image has a REALLY hard time finding and understanding what files are available...

Also, managed to reach the convert lua step, but the script is not able to read the directories it made earlier. It tries to access /some/directory/other/ instead of ./some/directory/other/

Hopefully I can fix it without editing 100's of lines of the image list

EDIT:

Decided to just sh into the thing and build everything from there, it all works now, but for some reason it couldn't find the very first image without altering the file list. A very random thing.

ErrorRaffyline0 commented 3 years ago

I am a complete noob at C++ and cmd stuff, so please don't cringe. For me "th" is not recognised as a command when "cd'ded" into the waifu2x-caffe-master or waifu2x-caffe folders. Or can you only do this on the non-caffe version (bc no linux (and i'm having issues with the windows subsystem for linux))

nagadomi commented 3 years ago

waifu2x-caffe does not support training the model.