search

wkentaro / pytorch-fcn

PyTorch Implementation of Fully Convolutional Networks. (Training code to reproduce the original result is available.)
MIT License
1.74k stars 480 forks source link

CPU memory usage is high #139

Open LiheYoung opened 5 years ago

LiheYoung commented 5 years ago

The cpu memory usage is high when training on Cityscapes. I didn't crop the image, and input size is 1024x2048. Larger size may lead to higher GPU memory usage but my GPU memory usage is okay(around 11G), the cpu memory usage is high(around 80%). Sometimes it may be killed because it cannot allocate memory.

wkentaro commented 5 years ago

Maybe this visualized image is too large in your case? https://github.com/wkentaro/pytorch-fcn/blob/e86a0f08bbc6fa4033b1e9d15c8ddb70a9537bb5/torchfcn/trainer.py#L126-L129

Those visualized images are tiled to create another large image. https://github.com/wkentaro/pytorch-fcn/blob/e86a0f08bbc6fa4033b1e9d15c8ddb70a9537bb5/torchfcn/trainer.py#L133-L137