search

taichi-dev / taichi

Productive, portable, and performant GPU programming in Python.
https://taichi-lang.org
Apache License 2.0
25.42k stars 2.27k forks source link

[Bug] [type] [autodiff] "examples/ad_gravity.py": RuntimeError: Assertion failure: primitive #1924

Open archibate opened 4 years ago

archibate commented 4 years ago

Describe the bug Running examples/ad_gravity.py results in RuntimeError: Assertion failure: primitive in latest master. But I can't reproduce this in v0.6.38 (latest stable).

To Reproduce Just run examples/ad_gravity.py.

Log/Screenshots Please post the full log of the program (instead of just a few lines around the error message, unless the log is > 1000 lines). This will help us diagnose what's happening. For example:

(gui-zerocopy) [bate@archit taichi]$ p examples/ad_gravity.py 
[Taichi] mode=development
[Taichi] preparing sandbox at /tmp/taichi-_vw6230o
[Taichi] <dev mode>, llvm 10.0.0, commit 91507a98, linux, python 3.8.5
[Taichi] materializing...
[E 10/05/20 20:40:40.488] [lang_util.cpp:to_primitive_type@390] Assertion failure: primitive

***********************************
* Taichi Compiler Stack Traceback *
***********************************                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::Logger::error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)                                  
/tmp/taichi-_vw6230o/taichi_core.so(+0x418319) [0x7fc761ec4319]                              
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::promoted_type(taichi::lang::DataType, taichi::lang::DataType)                                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::BinaryOpStmt*)                                                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::Block*)    
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::Block*)    
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::type_check(taichi::lang::IRNode*) 
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::compile_to_offloads(taichi::lang::IRNode*, taichi::lang::CompileConfig const&, bool, bool, bool, bool)                         
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::compile_to_executable(taichi::lang::IRNode*, taichi::lang::CompileConfig const&, bool, bool, bool, bool, bool, bool, bool)     
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::lower(bool)                       
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Program::compile(taichi::lang::Kernel&)   
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::compile()                         
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::operator()(taichi::lang::Kernel::LaunchContextBuilder&)                                                                        
/tmp/taichi-_vw6230o/taichi_core.so(+0x480694) [0x7fc761f2c694]                              
/tmp/taichi-_vw6230o/taichi_core.so(+0x39b8c2) [0x7fc761e478c2]                              
/usr/lib/libpython3.8.so.1.0: PyCFunction_Call                                               
/usr/lib/libpython3.8.so.1.0: _PyObject_MakeTpCall                                           
/usr/lib/libpython3.8.so.1.0(+0x13ecf6) [0x7fc766064cf6]                                     
/usr/lib/libpython3.8.so.1.0: PyObject_Call                                                  
/usr/lib/libpython3.8.so.1.0(+0x9904e) [0x7fc765fbf04e]                                      
/usr/lib/libpython3.8.so.1.0: _PyObject_MakeTpCall                                           
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall                                         
/usr/lib/libpython3.8.so.1.0: PyObject_Call                                                  
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall                                         
/usr/lib/libpython3.8.so.1.0: _PyObject_FastCallDict                                         
/usr/lib/libpython3.8.so.1.0: _PyObject_Call_Prepend                                         
/usr/lib/libpython3.8.so.1.0(+0x1f7a39) [0x7fc76611da39]                                     
/usr/lib/libpython3.8.so.1.0: PyObject_Call                                                  
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall                                         
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall                                         
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: PyEval_EvalCode                                                
/usr/lib/libpython3.8.so.1.0(+0x1d9c18) [0x7fc7660ffc18]                                     
/usr/lib/libpython3.8.so.1.0(+0x1d3e33) [0x7fc7660f9e33]                                     
/usr/lib/libpython3.8.so.1.0: PyRun_FileExFlags                                              
/usr/lib/libpython3.8.so.1.0: PyRun_SimpleFileExFlags                                        
/usr/lib/libpython3.8.so.1.0: Py_RunMain                                                     
/usr/lib/libpython3.8.so.1.0: Py_BytesMain                                                   
/usr/lib/libc.so.6: __libc_start_main                                                        
python(_start+0x2e) [0x564d159fc04e]                                                         

Internal Error occurred, check this page for possible solutions:                             
https://taichi.readthedocs.io/en/stable/install.html#troubleshooting                         
[E 10/05/20 20:40:40.496] [lang_util.cpp:to_primitive_type@390] Assertion failure: primitive

***********************************
* Taichi Compiler Stack Traceback *                                                          
***********************************                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::Logger::error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)                                  
/tmp/taichi-_vw6230o/taichi_core.so(+0x418319) [0x7fc761ec4319]                              
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::promoted_type(taichi::lang::DataType, taichi::lang::DataType)                                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::BinaryOpStmt*)                                                                                          
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::Block*)    
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::TypeCheck::visit(taichi::lang::Block*)    
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::type_check(taichi::lang::IRNode*) 
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::compile_to_offloads(taichi::lang::IRNode*, taichi::lang::CompileConfig const&, bool, bool, bool, bool)                         
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::irpass::compile_to_executable(taichi::lang::IRNode*, taichi::lang::CompileConfig const&, bool, bool, bool, bool, bool, bool, bool)     
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::lower(bool)                       
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Program::compile(taichi::lang::Kernel&)   
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::compile()                         
/tmp/taichi-_vw6230o/taichi_core.so: taichi::lang::Kernel::operator()(taichi::lang::Kernel::LaunchContextBuilder&)                                                                        
/tmp/taichi-_vw6230o/taichi_core.so(+0x480694) [0x7fc761f2c694]                              
/tmp/taichi-_vw6230o/taichi_core.so(+0x39b8c2) [0x7fc761e478c2]                              
/usr/lib/libpython3.8.so.1.0: PyCFunction_Call                                               
/usr/lib/libpython3.8.so.1.0: _PyObject_MakeTpCall                                           
/usr/lib/libpython3.8.so.1.0(+0x13ecf6) [0x7fc766064cf6]                                     
/usr/lib/libpython3.8.so.1.0: PyObject_Call                                                  
/usr/lib/libpython3.8.so.1.0(+0x9904e) [0x7fc765fbf04e]                                      
/usr/lib/libpython3.8.so.1.0: _PyObject_MakeTpCall                                           
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall                                         
/usr/lib/libpython3.8.so.1.0: PyObject_Call                                                  
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault                                       
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName                                       
/usr/lib/libpython3.8.so.1.0: _PyObject_FastCallDict                                         
/usr/lib/libpython3.8.so.1.0: _PyObject_Call_Prepend                                         
/usr/lib/libpython3.8.so.1.0(+0x1f7a39) [0x7fc76611da39]                                     
/usr/lib/libpython3.8.so.1.0: PyObject_Call
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall
/usr/lib/libpython3.8.so.1.0(+0x13eb3c) [0x7fc766064b3c]
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault
/usr/lib/libpython3.8.so.1.0: _PyFunction_Vectorcall
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalFrameDefault
/usr/lib/libpython3.8.so.1.0: _PyEval_EvalCodeWithName
/usr/lib/libpython3.8.so.1.0: PyEval_EvalCode
/usr/lib/libpython3.8.so.1.0(+0x1d9c18) [0x7fc7660ffc18]
/usr/lib/libpython3.8.so.1.0(+0x1d3e33) [0x7fc7660f9e33]
/usr/lib/libpython3.8.so.1.0: PyRun_FileExFlags
/usr/lib/libpython3.8.so.1.0: PyRun_SimpleFileExFlags
/usr/lib/libpython3.8.so.1.0: Py_RunMain
/usr/lib/libpython3.8.so.1.0: Py_BytesMain
/usr/lib/libc.so.6: __libc_start_main
python(_start+0x2e) [0x564d159fc04e]

Internal Error occurred, check this page for possible solutions:
https://taichi.readthedocs.io/en/stable/install.html#troubleshooting
Traceback (most recent call last):
  File "examples/ad_gravity.py", line 33, in substep
    compute_U()  # will also computes dU/dx and save in x.grad
  File "/home/bate/Develop/taichi/python/taichi/lang/kernel.py", line 574, in wrapped
    return primal(*args, **kwargs)
  File "/home/bate/Develop/taichi/python/taichi/lang/kernel.py", line 502, in __call__
    return self.compiled_functions[key](*args)
  File "/home/bate/Develop/taichi/python/taichi/lang/kernel.py", line 460, in func__
    t_kernel(launch_ctx)
RuntimeError: [lang_util.cpp:to_primitive_type@390] Assertion failure: primitive

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "examples/ad_gravity.py", line 47, in <module>
    substep()
  File "examples/ad_gravity.py", line 33, in substep
    compute_U()  # will also computes dU/dx and save in x.grad
  File "/home/bate/Develop/taichi/python/taichi/lang/tape.py", line 18, in __exit__
    self.grad()
  File "/home/bate/Develop/taichi/python/taichi/lang/tape.py", line 27, in grad
    func.grad(*args)
  File "/home/bate/Develop/taichi/python/taichi/lang/kernel.py", line 502, in __call__
    return self.compiled_functions[key](*args)
  File "/home/bate/Develop/taichi/python/taichi/lang/kernel.py", line 460, in func__
    t_kernel(launch_ctx)
RuntimeError: [lang_util.cpp:to_primitive_type@390] Assertion failure: primitive

Additional comments If possible, please also consider attaching the output of command ti diagnose. This produces the detailed environment information and hopefully helps us diagnose faster.

If you have local commits (e.g. compile fixes before you reproduce the bug), please make sure you first make a PR to fix the build errors and then report the bug.

archibate commented 4 years ago

The bug is gone after reverting #1906 @yuanming-hu Would you revert it and remerge it into an experimental branch (like @Rullec did in rullec-field) so it won't break master completely?

archibate commented 4 years ago

It's likely you didn't copy the primitive ptr in somewhere in ad system :( We'd better set up the test_examples asap to prevent hidden issues like this.

archibate commented 4 years ago

mrp:

import taichi as ti

# ti.init()  # OK
# ti.reset()  # OK

x = ti.field(float, (), needs_grad=True)
y = ti.field(float, (), needs_grad=True)

@ti.kernel
def func():
    y[None] = 1 / x[None]
    #y[None] = 1.0 / x[None]  # OK
    #y[None] = 1 + x[None]  # OK
    #y[None] = x[None]  # OK

func.grad()

Why adding ti.reset() fix the problem? Why 1.0 / x[None] fix the problem? Isn't promote_type working well?

archibate commented 3 years ago

I wonder if we actually should put ti.reset() in the last line of taichi/__init__.py?