search

tangxianfeng / STDN

Code for our Spatiotemporal Dynamic Network
275 stars 98 forks source link

Load File Question #4

Open skyWalker1997 opened 5 years ago

skyWalker1997 commented 5 years ago

Hello！I have already read your paper and tried to run your code. But I got a wired problem here. When I tried to run this code on my CPU server(CentOS 7,8G memory),it always killed by system because of out of memory.And it would be kill in sampling stage.（Ubuntu 18 can`t run successfully either,killed when it sampling）. But when I run this code on my macbook,it will not be killed by macos.(Just little slow when train one epoch，about 1 hour) I tried to turn up the virtual memory in CentOS (etc. 10g),it was killed when it smapling 1600 timeslots(when virtual memory is zero,it just can make 900). So how could I fixed this problem?Or must I use GPU to run your model?

BTW, have you ever put the compare experiment code in your project.I guess there is a single "if model_name[2:] == "stdn"" in your code.Maybe here was a "if model_name[2:] == "lstn""before LOL.(If you would offer the compare experiment code I will really appreciate it)

Thx a lot!

LeeDoYup commented 5 years ago

I have same problem on preprocessing. When the sampler runs, the program is killed. I have 16 GB memory.

tangxianfeng commented 5 years ago

@skyWalker1997 @LeeDoYup If the memory is not enough, you need to add more memory (or swap). I use a machine with 128G memory with a 128G swap. So I simply load all data to the memory. You can also rewrite the loading part so that it can load data on each batch. This can essentially solve the problem.

LeeDoYup commented 5 years ago

I think it has many problems in preprocessing codes, because the original dataset (NYC-taxi) has very small size (< 100 MB). I will review it.

zwd2016 commented 5 years ago

Hi @tangxianfeng, I also encountered a similar problem, why do timeslots start from 500. The second is how to set up the code to load samples one batch by one batch as you said? Is the training also carried out batch by batch? This is my error: `runfile('D:/test/stdn/main.py', wdir='D:/test/stdn') Now sampling at 500 timeslots. Now sampling at 600 timeslots. Now sampling at 700 timeslots. Now sampling at 800 timeslots. Now sampling at 900 timeslots. Now sampling at 1000 timeslots. Now sampling at 1100 timeslots. Now sampling at 1200 timeslots. Now sampling at 1300 timeslots. Now sampling at 1400 timeslots. Now sampling at 1500 timeslots. Now sampling at 1600 timeslots. Now sampling at 1700 timeslots. Now sampling at 1800 timeslots. Now sampling at 1900 timeslots. Traceback (most recent call last):

File "", line 1, in runfile('D:/test/stdn/main.py', wdir='D:/test/stdn')

File "D:\Users\w\Anaconda3\envs\tensorflow1.4-gpu\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile execfile(filename, namespace)

File "D:\Users\w\Anaconda3\envs\tensorflow1.4-gpu\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/test/stdn/main.py", line 128, in main(batch_size = batch_size, max_epochs = max_epochs, early_stop = stop)

File "D:/test/stdn/main.py", line 94, in main nbhd_size = nbhd_size, cnn_nbhd_size = cnn_nbhd_size)

File "D:\test\stdn\file_loader.py", line 234, in sample_stdn lstm_att_features[i] = np.array(lstm_att_features[i])

MemoryError ` Thank you very much!