search

twitter-archive / torch-ipc

A set of primitives for parallel computation in Torch
Apache License 2.0
95 stars 28 forks source link

WARN: torch-ipc: CUDA IPC evicting 0x431ee20000 due to ptr reuse, performance will drastically suffer #27

Open chienlinhuang1116 opened 8 years ago

chienlinhuang1116 commented 8 years ago

Hi,

Sometimes, I got messages like these. The jobs are still running and results seem correct. Do you have any idea about this? How can we avoid this situation?

WARN: torch-ipc: CUDA IPC evicting 0x431ee20000 due to ptr reuse, performance will drastically suffer
WARN: torch-ipc: CUDA IPC evicting 0x431ef20000 due to ptr reuse, performance will drastically suffer
WARN: torch-ipc: CUDA IPC evicting 0x431ee20000 due to ptr reuse, performance will drastically suffer
WARN: torch-ipc: CUDA IPC evicting 0x431ef20000 due to ptr reuse, performance will drastically suffer
WARN: torch-ipc: CUDA IPC evicting 0x431ee20000 due to ptr reuse, performance will drastically suffer

Thank you, Chien-Lin

xternalz commented 8 years ago

I'm experiencing this too, even with stableGradients = true. Anyone has idea why?