Memory Error

[NamburiSrinath]

Hi Jackie-Chou,

I was working on this repository and it was showing "Memory Error" when I am running match.py

I printed the width, height, max_num of the np array that was about to form just before this error pop up

In the line 642 to 646, the output "Here" got printed but "Here too" didn't. Could you help me in resolving this issue?

P.S: I have 2GB graphics card, NVIDIA 940 and 8GB RAM. Thanks a lot in advance

Srinath

[Jackie-Chou]

Hi Jackie-Chou,

I was working on this repository and it was showing "Memory Error" when I am running match.py

I printed the width, height, max_num of the np array that was about to form just before this error pop up

In the line 642 to 646, the output "Here" got printed but "Here too" didn't. Could you help me in resolving this issue?

P.S: I have 2GB graphics card, NVIDIA 940 and 8GB RAM. Thanks a lot in advance

Srinath

Hi Srinath,

It seems to me that the ndarray union_region needs more RAM to be created, sadly. Using the values printed in your first picture, we can calculate that a union_region with shape (948,1440,784,2) and dtype int32 would need 948x1440x784x2x4 = 8,562,032,640 B = 8,361,360 KB = 8165 MB = 7.97 GB*, so a 8GB RAM would be dangerous.

Sincerely, this is my fault. When I worte this code, I was working on a server with a big 126GB RAM, so I didn't pay much effort to optimize the code and try to reduce memory consumption. I will do the optimization in the future. Sorry for my laziness. ):

And for your problem at hand, I would suggest: 1.use smaller images 2.skip some processing procedure that comsumes large RAM 3.try to do code optimization yourself.

Hope this can be useful. Apologize for my laziness again.

Jackie Chou

[NamburiSrinath]

Thanks for your reply Jackie Chou,

1.Use smaller images --> We want to run MC-CNN on KITTI stereo dataset. So, do u mean to resize the images before feeding into the network.

2.Skip some processing steps that consumes large RAM ---> Can you please be more specific as in what steps can be avoided/ removed. As in giving the names of the functions or placing code snippets etc etc;

3.Try to do code optimization yourself ---> I have reduced the np.int32 to np.int8 and it gave a bit improvement but that is not enough. Can you please suggest in addition to that some other tips

4.How to access GPU for training? I have given the parse argument -gpu 4 but couldn't observe any improvements in training speed. So, I doubt whether the training uses my GPU card. Any tips for that

5.Is 16GB RAM good enough to train the entire MC-CNN on KITTI stereo dataset?

6.How much time will it take(appx) to train the entire KITTI stereo dataset(with and without GPU)?

Thanks in advance Jackie Chou

[NamburiSrinath]

Hello again Jackie Chou,

As you suggested, we have tried to optimise the code and upgraded to a 32GB system and started running but finally ended up with an error

We got this error after approximately the system ran for 15+hrs

The code corresponding to this is process_functional.py with disparity_prediction function(check line number 259 in attached screenshot below

1. Can you please help us out what we should do to avoid the assertion error?

2. Also, I have given this command to run the match.py code. What does those tag, start and end numbers actually should be?

_python3 match.py --list_file '/media/srinath/Major Project/cod es/MC-CNN-python-master/data/list/test.txt' --data_dir '/media/srinath/Major Project/codes/MC-CNN-python-master/data/MiddEval3/testH/' --save_dir '/media/srinath/Major Project/codes/MC-CNN-python-master/save1/' -t 1 -s 0 -e 1_

3. If I want to include gpus, can I write -g 0,1,2,3? How to reduce time as in to get a disparity map within a couple of hours?

Very thankful in advance Jackie

[Jackie-Chou]

Hello again Jackie Chou,

As you suggested, we have tried to optimise the code and upgraded to a 32GB system and started running but finally ended up with an error

We got this error after approximately the system ran for 15+hrs

The code corresponding to this is process_functional.py with disparity_prediction function(check line number 259 in attached screenshot below

1. Can you please help us out what we should do to avoid the assertion error?
2. Also, I have given this command to run the match.py code. What does those tag, start and end numbers actually should be?

_python3 match.py --list_file '/media/srinath/Major Project/cod es/MC-CNN-python-master/data/list/test.txt' --data_dir '/media/srinath/Major Project/codes/MC-CNN-python-master/data/MiddEval3/testH/' --save_dir '/media/srinath/Major Project/codes/MC-CNN-python-master/save1/' -t 1 -s 0 -e 1_

1. If I want to include gpus, can I write -g 0,1,2,3? How to reduce time as in to get a disparity map within a couple of hours?

Very thankful in advance Jackie

1. For the assertion error, I'd suggest check the value of ndisp and content in left_disparity_map. It shouldn't happen in normal.

2.The tag is only used to mark one particular execution, so simply "-t 1" is ok; -s and -e are used to limit the processing range of images, only images with index between [-s, -e] will be processed and others skipped.

3. You could write -g 0,1,2,3 (see the explanation for flag -g). About the running time, I'm afraid it's inevitably long for the python implementation; try using more low-level language like C and cuda for better efficiency, just like the authors of the MC-CNN paper did.