Don't know everyone will encounter this problem, the model run many times, before running structure, will find that model can only load data, at the time of loading model process will be killed, but before is completely can run normally, and ten times in one time and can run normally, but some batches of training time will be very long before. I watched the system monitor and found that the memory was completely (99%+) used during load so it was killed, whereas the memory usage was usually only around 20% when the program was running properly. By the way, my computer is 64 gigabytes
不知道大家会不会遇到这样的问题,模型运行很多次后,再运行之前的结构,会发现模型只能加载数据,在加载模型的时候进程会被杀死,但之前是完全能正常运行的,而且十次中有一次又能正常运行,只不过前一些批次的训练时长会很长。我观察系统监视器发现加载时内存完全(99%+)被使用所以才会被kill,然而程序正常运行时内存的使用经常只占据20%左右。顺便说一下我的电脑是64G的.请问大家有遇到这个问题吗,都是怎么解决的呢?
Don't know everyone will encounter this problem, the model run many times, before running structure, will find that model can only load data, at the time of loading model process will be killed, but before is completely can run normally, and ten times in one time and can run normally, but some batches of training time will be very long before. I watched the system monitor and found that the memory was completely (99%+) used during load so it was killed, whereas the memory usage was usually only around 20% when the program was running properly. By the way, my computer is 64 gigabytes 不知道大家会不会遇到这样的问题,模型运行很多次后,再运行之前的结构,会发现模型只能加载数据,在加载模型的时候进程会被杀死,但之前是完全能正常运行的,而且十次中有一次又能正常运行,只不过前一些批次的训练时长会很长。我观察系统监视器发现加载时内存完全(99%+)被使用所以才会被kill,然而程序正常运行时内存的使用经常只占据20%左右。顺便说一下我的电脑是64G的.请问大家有遇到这个问题吗,都是怎么解决的呢?