search

Jittor / jittor

Jittor is a high-performance deep learning framework based on JIT compiling and meta-operators.
https://cg.cs.tsinghua.edu.cn/jittor/
Apache License 2.0
3.09k stars 311 forks source link

Crash with error happend during compilatio in `jt.nn.AvgPool2d/avg_pool2d/AvgPool3d` #520

Closed x0w3n closed 5 months ago

x0w3n commented 5 months ago

Describe the bug

Crash occurs when passing unexpected values to jittor's jt.nn.AvgPool2d/avg_pool2d/avg_pool3d/AvgPool3d, such as kernel_size=-1, resulting in error happend during compilatio.

Full Log

AvgPool2d:

[i 0508 13:45:41.607793 52 compiler.py:956] Jittor(1.3.9.6) src: /jittor/python/jittor
[i 0508 13:45:41.610825 52 compiler.py:957] g++ at /usr/bin/g++(8.3.0)
[i 0508 13:45:41.610968 52 compiler.py:958] cache_path: /root/.cache/jittor/jt1.3.9/g++8.3.0/py3.7.4/Linux-5.15.153x56/13thGenIntelRCx56/8c5a/master
[i 0508 13:45:41.615182 52 __init__.py:412] Found addr2line(2.31.1) at /usr/bin/addr2line.
[i 0508 13:45:41.838718 52 __init__.py:227] Total mem: 15.43GB, using 5 procs for compiling.
[i 0508 13:45:41.909309 52 jit_compiler.cc:28] Load cc_path: /usr/bin/g++
Traceback (most recent call last):
  File "test.py", line 12, in <module>
    a = jt.nn.AvgPool2d(kernel_size=-1,stride=1,padding=-1)(jt.random((1,3,5,5))) # crash
  File "/jittor/python/jittor/__init__.py", line 1184, in __call__
    return self.execute(*args, **kw)
  File "/jittor/python/jittor/pool.py", line 524, in execute
    return self.layer(x)
  File "/jittor/python/jittor/__init__.py", line 1184, in __call__
    return self.execute(*args, **kw)
  File "/jittor/python/jittor/pool.py", line 184, in execute
    f"i3*{self.stride[1]}-{self.padding[1]}+i5", # Wid
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.reindex)).

Types of your inputs are:
 self   = Var,
 args   = (list, list, ),

The function declarations are:
 VarHolder* reindex(VarHolder* x,  NanoVector shape,  vector<string>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={}, vector<VarHolder*>&& extras={})
 VarHolder* reindex_(VarHolder* x, vector<VarHolder*>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={})

Failed reason:[f 0508 13:45:43.762340 52 parallel_compiler.cc:331] Error happend during compilation:
 [Error] source file location:/root/.cache/jittor/jt1.3.9/g++8.3.0/py3.7.4/Linux-5.15.153x56/13thGenIntelRCx56/8c5a/master/jit/__opkey0_reindex__Tx_float32__XDIM_4__YDIM_6__OVERFLOW_itof_0x0___INDEX0_i0__INDEX1_i1__IN___hash_c67001a5e6ed8c7f_op.cc
Compile fused operator(1/2)failed:[Op(3:1:1:1:i1:o1:s0:g1,reindex->4),]

Reason: [f 0508 13:45:43.758751 04:C1 expr.cc:394] Op -- 1 1 not support

avg_pool2d:

[i 0508 13:46:20.687796 32 compiler.py:956] Jittor(1.3.9.6) src: /jittor/python/jittor
[i 0508 13:46:20.690899 32 compiler.py:957] g++ at /usr/bin/g++(8.3.0)
[i 0508 13:46:20.691011 32 compiler.py:958] cache_path: /root/.cache/jittor/jt1.3.9/g++8.3.0/py3.7.4/Linux-5.15.153x56/13thGenIntelRCx56/8c5a/master
[i 0508 13:46:20.695856 32 __init__.py:412] Found addr2line(2.31.1) at /usr/bin/addr2line.
[i 0508 13:46:20.922624 32 __init__.py:227] Total mem: 15.43GB, using 5 procs for compiling.
[i 0508 13:46:20.994343 32 jit_compiler.cc:28] Load cc_path: /usr/bin/g++
Traceback (most recent call last):
  File "test.py", line 13, in <module>
    a = jt.nn.avg_pool2d(x=jt.random((1,3,3,5)),kernel_size=-1,stride=1,padding=-1) # crash
  File "/jittor/python/jittor/pool.py", line 534, in avg_pool2d
    return AvgPool2d(kernel_size, stride, padding, ceil_mode, count_include_pad)(x)
  File "/jittor/python/jittor/__init__.py", line 1184, in __call__
    return self.execute(*args, **kw)
  File "/jittor/python/jittor/pool.py", line 524, in execute
    return self.layer(x)
  File "/jittor/python/jittor/__init__.py", line 1184, in __call__
    return self.execute(*args, **kw)
  File "/jittor/python/jittor/pool.py", line 184, in execute
    f"i3*{self.stride[1]}-{self.padding[1]}+i5", # Wid
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.reindex)).

Types of your inputs are:
 self   = Var,
 args   = (list, list, ),

The function declarations are:
 VarHolder* reindex(VarHolder* x,  NanoVector shape,  vector<string>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={}, vector<VarHolder*>&& extras={})
 VarHolder* reindex_(VarHolder* x, vector<VarHolder*>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={})

Failed reason:[f 0508 13:46:21.135019 32 parallel_compiler.cc:331] Error happend during compilation:
 [Error] source file location:/root/.cache/jittor/jt1.3.9/g++8.3.0/py3.7.4/Linux-5.15.153x56/13thGenIntelRCx56/8c5a/master/jit/__opkey0_reindex__Tx_float32__XDIM_4__YDIM_6__OVERFLOW_itof_0x0___INDEX0_i0__INDEX1_i1__IN___hash_c67001a5e6ed8c7f_op.cc
Compile fused operator(1/2)failed:[Op(3:1:1:1:i1:o1:s0:g1,reindex->4),]

Reason: [f 0508 13:46:21.133978 08:C1 expr.cc:394] Op -- 1 1 not support

AvgPool3d:

Traceback (most recent call last):
  File "test.py", line 7, in <module>
    a = jt.nn.AvgPool3d(kernel_size=-1,stride=1,padding=-1)(jt.random((1,3,5,5,5))) # crash
  File "/root/anaconda3/envs/jt/lib/python3.7/site-packages/jittor/__init__.py", line 1172, in __call__
    return self.execute(*args, **kw)
  File "/root/anaconda3/envs/jt/lib/python3.7/site-packages/jittor/pool.py", line 531, in execute
    return self.layer(x)
  File "/root/anaconda3/envs/jt/lib/python3.7/site-packages/jittor/__init__.py", line 1172, in __call__
    return self.execute(*args, **kw)
  File "/root/anaconda3/envs/jt/lib/python3.7/site-packages/jittor/pool.py", line 382, in execute
    f"i4*{self.stride[2]}-{self.padding[2]}+i7", # Hid
RuntimeError: Wrong inputs arguments, Please refer to examples(help(jt.reindex)).

Types of your inputs are:
 self   = Var,
 args   = (list, list, ),

The function declarations are:
 VarHolder* reindex(VarHolder* x,  NanoVector shape,  vector<string>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={}, vector<VarHolder*>&& extras={})
 VarHolder* reindex_(VarHolder* x, vector<VarHolder*>&& indexes,  float64 overflow_value=0,  vector<string>&& overflow_conditions={})

Failed reason:[f 0508 08:50:37.874728 36 parallel_compiler.cc:331] Error happend during compilation:
 [Error] source file location:/root/.cache/jittor/jt1.3.8/g++11.4.0/py3.7.0/Linux-6.5.0-27xd2/IntelRXeonRGolx8d/default/cu12.1.105_sm_86/jit/__opkey0_reindex__Tx_float32__XDIM_5__YDIM_8__OVERFLOW_itof_0x0___INDEX0_i0__INDEX1_i1__IN___hash_e419dfc3066ba767_op.cc
Compile fused operator(1/2)failed:[Op(3:1:1:1:i1:o1:s0,reindex->4),]

Reason: [f 0508 08:50:37.872340 16:C1 expr.cc:394] Op -- 1 1 not support

Minimal Reproduce

import jittor as jt
from jittor import *
AvgPool2d = jt.nn.AvgPool2d(kernel_size=-1,stride=1,padding=-1)(jt.random((1,3,5,5))) # crash
# avg_pool2d = jt.nn.avg_pool2d(x=jt.random((1,3,3,5)),kernel_size=-1,stride=1,padding=-1) # crash
# AvgPool3d = jt.nn.AvgPool3d(kernel_size=-1,stride=1,padding=-1)(jt.random((1,3,5,5,5))) # crash

Expected behavior

The main reason for these crashes may be that the underlying processing functions of these api's don't do bounds checking on parameters such as kernel_size=1,stride=1, etc.