Could support using taichi in jupyter notebook?

[zhugw]

Hope could use taichi in jupyter notebook directly, now run sample code in cell failed with below exception

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:150, in our_findsource(object)
    149 try:
--> 150     return inspect._si_old_findsource(object)
    151 except IOError:

File /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/inspect.py:798, in findsource(object)
    797 if not lines:
--> 798     raise OSError('could not get source code')
    800 if ismodule(object):

OSError: could not get source code

During handling of the above exception, another exception occurred:

ModuleNotFoundError                       Traceback (most recent call last)
File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:96, in blender_findsourcetext(object)
     95 try:
---> 96     import bpy
     97 except ImportError:

ModuleNotFoundError: No module named 'bpy'

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:153, in our_findsource(object)
    152 try:
--> 153     return blender_findsource(object)
    154 except IOError:

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:110, in blender_findsource(object)
    109 def blender_findsource(object):
    --> 110     lines, text_name = blender_findsourcetext(object)
    112     try:

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:98, in blender_findsourcetext(object)
     97 except ImportError:
---> 98     raise IOError('Not in Blender environment!')
    100 file = inspect.getfile(object)

OSError: Not in Blender environment!

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:156, in our_findsource(object)
    155 try:
--> 156     return remote_findsource(object)
    157 except IOError:

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:42, in remote_findsource(object)
     40     raise IOError(f'Object `{repr(object)}` does not come from console')
---> 42 lines = remote_findsourcelines(object)
     43 for lnum, line in reversed(list(enumerate(lines))):

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:28, in remote_findsourcelines(object)
     27     (message)
---> 28     raise IOError(f'Cannot find file {file}!' + message)
     30 with open(file, 'r') as f:

OSError: Cannot find file /var/folders/x0/fk_d04j9307fs178ghhgb7b80000gn/T/SI_IPC_81596.py!
To make Taichi functional in IDLE, please append the following line:

__import__("sourceinspect").remote_hack(globals())

to file "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/code.py".

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:159, in our_findsource(object)
    158 try:
--> 159     return dill_findsource(object)
    160 except IOError:

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:145, in dill_findsource(object)
    141     raise IOError(
    142         'Run `python3 -m pip install dill` to make SourceInspect functional in Python native shell. '
    143         'Additionally you should run `python3 -m pip install pyreadline` if you are on Windows.'
    144         )
--> 145 return dill.source.findsource(object)

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/dill/source.py:154, in findsource(object)
    153 if not lines:
--> 154     raise IOError('could not extract source code')
    156 #FIXME: all below may fail if exec used (i.e. exec('f = lambda x:x') )

OSError: could not extract source code

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
File <timed exec>:24, in <module>

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:918, in _kernel_impl.<locals>.wrapped(*args, **kwargs)
    915 @functools.wraps(_func)
    916 def wrapped(*args, **kwargs):
    917     try:
--> 918         return primal(*args, **kwargs)
    919     except (TaichiCompilationError, TaichiRuntimeError) as e:
    920         raise type(e)('\n' + str(e)) from None

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/shell.py:37, in _shell_pop_print.<locals>.new_call(*args, **kwargs)
     35 @functools.wraps(old_call)
     36 def new_call(*args, **kwargs):
---> 37     ret = old_call(*args, **kwargs)
     38     # print's in kernel won't take effect until ti.sync(), discussion:
     39     # https://github.com/taichi-dev/taichi/pull/1303#discussion_r444897102
40     print(_ti_core.pop_python_print_buffer(), end='')

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:844, in Kernel.__call__(self, *args, **kwargs)
840     _logging.warn(
841         """opt_level = 1 is enforced to enable gradient computation."""
842     )
843     impl.current_cfg().opt_level = 1
--> 844 key = self.ensure_compiled(*args)
845 return self.runtime.compiled_functions[key](*args)

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:819, in Kernel.ensure_compiled(self, *args)
817 instance_id, arg_features = self.mapper.lookup(args)
818 key = (self.func, instance_id, self.autodiff_mode)
--> 819 self.materialize(key=key, args=args, arg_features=arg_features)
820 return key

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:507, in Kernel.materialize(self, key, args, arg_features)
504 kernel_name = f"{self.func.__name__}_c{self.kernel_counter}_{key[1]}{grad_suffix}"
505 _logging.trace(f"Compiling kernel {kernel_name}...")
--> 507 tree, ctx = _get_tree_and_ctx(
508     self,
509     args=args,
510     excluded_parameters=self.template_slot_locations,
511     arg_features=arg_features)
513 if self.autodiff_mode != AutodiffMode.NONE:
514     KernelSimplicityASTChecker(self.func).visit(tree)

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/taichi/lang/kernel_impl.py:110, in _get_tree_and_ctx(self, excluded_parameters, is_kernel, arg_features, args, ast_builder, is_real_function)
102 def _get_tree_and_ctx(self,
103                       excluded_parameters=(),
104                       is_kernel=True,
(...)
107                       ast_builder=None,
108                       is_real_function=False):
109     file = oinspect.getsourcefile(self.func)
--> 110     src, start_lineno = oinspect.getsourcelines(self.func)
111     src = [textwrap.fill(line, tabsize=4, width=9999) for line in src]
   112     tree = ast.parse(textwrap.dedent("\n".join(src)))

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:193, in getsourcelines(object)
   191 def getsourcelines(object):
   192     with InspectMock():
--> 193         return inspect.getsourcelines(object)

File /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/inspect.py:967, in getsourcelines(object)
   959 """Return a list of source lines and starting line number for an object.
   960 
   961 The argument may be a module, class, method, function, traceback, frame,
  (...)
   964 original source file the first line of code was found.  An OSError is
   965 raised if the source code cannot be retrieved."""
   966 object = unwrap(object)
--> 967 lines, lnum = findsource(object)
   969 if istraceback(object):
   970     object = object.tb_frame

File ~/PycharmProjects/3i/venv/lib/python3.8/site-packages/sourceinspect/__init__.py:161, in our_findsource(object)
   159     return dill_findsource(object)
   160 except IOError:
--> 161     raise IOError(f'Could not get source code for object: {repr(object)}')

[xuhao1]

I am fine with using taichi in Jupyter environments inside VSCode Jupyterlab and also Jupyter notebook. Maybe you can try to use Jupyter lab with anaconda?

[zhugw]

Maybe it's pycharm's problem, I will submit bug to pycharm

[victoriacity]

Your stack trace tries to import bpy which is the Python interface of Blender. I would recommend to double check your Python environment and see whether it involves Blender Python at any point.

[zhugw]

"double check your Python environment and see whether it involves Blender Python at any point." how to check like this

 pip list | grep -i Blender

And run the same code in Python console is ok

(venv) ➜  3i git:(master) ✗ python
Python 3.8.2 (default, Apr  8 2021, 23:19:18) 
[Clang 12.0.5 (clang-1205.0.22.9)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import taichi as ti
[Taichi] version 1.1.2, llvm 10.0.0, commit f25cf4a2, osx, python 3.8.2
>>> ti.init()
[Taichi] Starting on arch=x64
>>> @ti.func
... def is_prime2(n: int):
...     result = True
...     for k in range(2, int(n ** 0.5) + 1):
...         if n % k == 0:
...             result = False
...             break
...     return result

>>> @ti.kernel
... def count_primes2(n: int) -> int:
...     count = 0
...     for k in range(2, n):
...         if is_prime2(k):
...             count += 1
...     return count
... 
>>> print(count_primes2(1000000))
78498

but run it in Pycharm jupyter notebook failed

[CakeOrRiot]

I was having the same issue, but I've found a workaround. Try commenting %%time in your code or moving it to a cell, where you call count_primes2.

[zhugw]

@CakeOrRiot when removed %%time it's ok. Amazing!