Oneflow-Inc / oneflow

OneFlow is a deep learning framework designed to be user-friendly, scalable and efficient.
http://www.oneflow.org
Apache License 2.0
5.78k stars 658 forks source link

Aborted (core dumped) in `flow.nn.AdaptiveAvgPool1d/AdaptiveAvgPool2d/AdaptiveAvgPool3d` #10522

Open x0w3n opened 1 month ago

x0w3n commented 1 month ago

Summary

When input some boundary values to the parameters of oneflow.nn.AdaptiveAvgPool1d/AdaptiveAvgPool2d/AdaptiveAvgPool3d, it triggers the crash

Code to reproduce bug

AdaptiveAvgPool1d:

m = flow.nn.AdaptiveAvgPool1d(-5)
input = flow.Tensor(np.random.randn(1, 64, 8))
m(input)

output:

terminate called after throwing an instance of 'oneflow::RuntimeException'
  what():  Error: Error! : Out of memory when allocate size : 18446744073709550592.
 The total_memory_bytes allocated by this BinAllocator is : 2097152
You can set ONEFLOW_DEBUG or ONEFLOW_PYTHON_STACK_GETTER to 1 to get the Python stack of the error.
Stack trace (most recent call last) in thread 285823:
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1574249, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1573847, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff156f368, in vm::ThreadCtx::TryReceiveAndRun()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1506818, in vm::EpStreamPolicyBase::Run(vm::Instruction*) const
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150abe6, in vm::Instruction::Compute()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510d2a, in vm::OpCallInstructionPolicy::Compute(vm::Instruction*)
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510a1f, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150d182, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7fffed910082, in 

Aborted (Signal sent by tkill() 285582 0)
Aborted (core dumped)

AdaptiveAvgPool2d:

import oneflow as flow
m = flow.nn.AdaptiveAvgPool2d((-5,7))
input = flow.Tensor(np.random.randn(1, 64, 8, 9))
output = m(input)

output:

terminate called after throwing an instance of 'oneflow::RuntimeException'
  what():  Error: Error! : Out of memory when allocate size : 18446744073709542912.
 The total_memory_bytes allocated by this BinAllocator is : 2097152
You can set ONEFLOW_DEBUG or ONEFLOW_PYTHON_STACK_GETTER to 1 to get the Python stack of the error.
Stack trace (most recent call last) in thread 286359:
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1574249, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1573847, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff156f368, in vm::ThreadCtx::TryReceiveAndRun()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1506818, in vm::EpStreamPolicyBase::Run(vm::Instruction*) const
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150abe6, in vm::Instruction::Compute()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510d2a, in vm::OpCallInstructionPolicy::Compute(vm::Instruction*)
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510a1f, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150d182, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7fffed910082, in 

Aborted (Signal sent by tkill() 286118 0)
Aborted (core dumped)

AdaptiveAvgPool3d:

m = flow.nn.AdaptiveAvgPool3d((-5,5,7))
input = flow.Tensor(np.random.randn(1, 64, 8, 9,2))
output = m(input)

output:

terminate called after throwing an instance of 'oneflow::RuntimeException'
  what():  Error: Error! : Out of memory when allocate size : 18446744073709507072.
 The total_memory_bytes allocated by this BinAllocator is : 2097152
You can set ONEFLOW_DEBUG or ONEFLOW_PYTHON_STACK_GETTER to 1 to get the Python stack of the error.
Stack trace (most recent call last) in thread 286740:
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1574249, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1573847, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff156f368, in vm::ThreadCtx::TryReceiveAndRun()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1506818, in vm::EpStreamPolicyBase::Run(vm::Instruction*) const
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150abe6, in vm::Instruction::Compute()
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510d2a, in vm::OpCallInstructionPolicy::Compute(vm::Instruction*)
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff1510a1f, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7ffff150d182, in 
   Object "/home/temp/oneflow-1.0.0/build/liboneflow.so", at 0x7fffed910082, in 

Aborted (Signal sent by tkill() 286499 0)
Aborted (core dumped)

System Information