search

Yaoyi-Li / GCA-Matting

Official repository for Natural Image Matting via Guided Contextual Attention
MIT License
390 stars 62 forks source link

training with new dataset #15

Closed javismiles closed 3 years ago

javismiles commented 3 years ago

Good day I want to try to train the model with a new dataset I´m creating,

In this new dataset, for each input image, I have a ground truth alpha mask.

In here: train_image_file = ImageFileTrain(alpha_dir=CONFIG.data.train_alpha, fg_dir=CONFIG.data.train_fg, bg_dir=CONFIG.data.train_bg)

It seems to ask for 3 folders, one for the ground truth alpha it seems, and then two for fg and bg? what are these?

if I have input images and ground truth alphas, how can I adapt them to work with the input of your model? thank you very much

Yaoyi-Li commented 3 years ago

Hi, fg means 'foreground images' and bg is 'background'. Your input images are foreground images and you need one more folder for background images. You can refer to our paper or Deep Image Matting for more details about the composition.

javismiles commented 3 years ago

Thank you very much Li so maybe you are doing the same procedure than they do for example at HAttmatting? like this?:

but in your model are you doing the compositing before the training or during the training? maybe thats why you ask as input the foreground and the background images?

On Wed, Oct 14, 2020 at 3:07 PM Li Yaoyi notifications@github.com wrote:

Hi, fg means 'foreground images' and bg is 'background'. Your input images are foreground images and you need one more folder for background images. You can refer to our paper or Deep Image Matting for more details about the composition.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yaoyi-Li/GCA-Matting/issues/15#issuecomment-708389815, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHUIBFDZHFM2STMVHPXVATSKWPBTANCNFSM4SB5QUYQ .

Yaoyi-Li commented 3 years ago

Yes, almost the same as you said, and we composite images during the training. The number of samples in each epoch is the same as background images.

javismiles commented 3 years ago

thank you Li, that´s interesting, in HAttmatting they first do the compositing ahead of training, but you do it during the training, interesting, in theory that should slow down the training right?

"and we composite images during the training "

On Mon, Oct 19, 2020 at 3:27 PM Li Yaoyi notifications@github.com wrote:

Yes, almost the same as you said, and we composite images during the training. The number of samples in each epoch is the same as background images.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yaoyi-Li/GCA-Matting/issues/15#issuecomment-712156055, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHUIBCRHJQYLPYUBRIUVCLSLQ5FTANCNFSM4SB5QUYQ .

Yaoyi-Li commented 3 years ago

It highly depends on your CPUs and the complexity of your augmentations. In our training, we are doing the compositing after the resize operation, which is faster than merging high-resolution images off-line. It is a little slower but totally acceptable.

javismiles commented 3 years ago

thank you Li, makes sense

On Mon, Oct 19, 2020 at 3:43 PM Li Yaoyi notifications@github.com wrote:

It highly depends on your CPUs and the complexity of your augmentations. In our training, we are doing the compositing after the resize operation, which is faster than merging high-resolution images off-line. It is a little slower but totally acceptable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yaoyi-Li/GCA-Matting/issues/15#issuecomment-712165914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHUIBBVMT4DF3AFUH5L3V3SLQ7BPANCNFSM4SB5QUYQ .

javismiles commented 3 years ago

quick question if I may, when designing a dataset to extract people like the one you used, would you also include people holding objects? (books, hats, fruit, etc) or better people not holding anything? thank you for any advice, best :)

On Mon, Oct 19, 2020 at 3:46 PM Javier Ideami ideami@gmail.com wrote:

thank you Li, makes sense

On Mon, Oct 19, 2020 at 3:43 PM Li Yaoyi notifications@github.com wrote:

It highly depends on your CPUs and the complexity of your augmentations. In our training, we are doing the compositing after the resize operation, which is faster than merging high-resolution images off-line. It is a little slower but totally acceptable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yaoyi-Li/GCA-Matting/issues/15#issuecomment-712165914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHUIBBVMT4DF3AFUH5L3V3SLQ7BPANCNFSM4SB5QUYQ .

javismiles commented 3 years ago

and also if I may, while creating the dataset I have the doubt if it would be important to have people from different races, caucasian, asian, black, etc; I don´t know if this is relevant when we are just extracting shapes; is it relevant for this type of dataset or could the dataset be using just caucasian people for example; I have doubts because for example networks that extract saliency maybe wouldn´t care about it, or maybe they do care, any tips are welcome, thank you :)

On Mon, Oct 19, 2020 at 5:40 PM Javier Ideami ideami@gmail.com wrote:

quick question if I may, when designing a dataset to extract people like the one you used, would you also include people holding objects? (books, hats, fruit, etc) or better people not holding anything? thank you for any advice, best :)

On Mon, Oct 19, 2020 at 3:46 PM Javier Ideami ideami@gmail.com wrote:

thank you Li, makes sense

On Mon, Oct 19, 2020 at 3:43 PM Li Yaoyi notifications@github.com wrote:

It highly depends on your CPUs and the complexity of your augmentations. In our training, we are doing the compositing after the resize operation, which is faster than merging high-resolution images off-line. It is a little slower but totally acceptable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Yaoyi-Li/GCA-Matting/issues/15#issuecomment-712165914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHUIBBVMT4DF3AFUH5L3V3SLQ7BPANCNFSM4SB5QUYQ .

javismiles commented 3 years ago

Dear friends, couple of quick questions:

in your gca repo, what is the difference between the gca-dist.toml and the gca-dist-all-data.toml files?

and then in the config file, here:

train_fg = "/home/liyaoyi/dataset/Adobe/all/fg" train_alpha = "/home/liyaoyi/dataset/Adobe/all/alpha" train_bg = "/home/liyaoyi/dataset/coco_bg" test_merged = "/home/liyaoyi/dataset/Adobe/Combined_Dataset/Test_set/merged" test_alpha = "/home/liyaoyi/dataset/Adobe/Combined_Dataset/Test_set/alpha_copy" test_trimap = "/home/liyaoyi/dataset/Adobe/Combined_Dataset/Test_set/trimaps"

If i want to retrain the net with the distinctions-646 dataset, it's clear what I should put in the first 3, just foregrounds, just alpha and just coco backgrounds, but in regards to the 3 test folders, the distinctions dataset does not have trimaps in separate files, would it be possible to use part of the alphamatting dataset at http://alphamatting.com/datasets.php as the test folders here? (they have separate trimaps

thank you for any tips

javismiles commented 3 years ago

Good day friends,

I am running well the training code of GCA, however using a single 16GB RAM gpu of colab pro, just to try it, I get Memory error:

RuntimeError: CUDA out of memory. Tried to allocate 3.29 GiB (GPU 0; 15.75 GiB total capacity; 11.80 GiB already allocated; 2.16 GiB free; 12.43 GiB reserved in total by PyTorch) [ ]

How much GPU memory is required to train GCA? thank you

javismiles commented 3 years ago

good day, changing the images from png to jpeg format eliminates the memory problem thats good, and the training part goes well, however the test part fails with error: " ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,2) and requested shape (0,2)"

for the training part im using the distinction-646 dataset, for the test part I am using the alphamatting dataset because it has trimaps, could that be the reason? what should be shapes and features of the images used for the test part?

thank you :)

Ideami Studios Augmented A.I / Deep Learning: https://losslandscape.com Augmented Innovation: https://torch4ideas.com Augmented Design: https://posterini.com Augmented Ideami: https://ideami.com

Twitter: https://twitter.com/ideami

On Wed, Oct 28, 2020 at 5:41 PM Javier Ideami ideami@gmail.com wrote:

Good day friends,

I am running well the training code of GCA, however using a single 16GB RAM gpu of colab pro, just to try it, I get Memory error:

RuntimeError: CUDA out of memory. Tried to allocate 3.29 GiB (GPU 0; 15.75 GiB total capacity; 11.80 GiB already allocated; 2.16 GiB free; 12.43 GiB reserved in total by PyTorch) [ ]

How much GPU memory is required to train GCA? thank you