numba / numba

NumPy aware dynamic Python compiler using LLVM
https://numba.pydata.org/
BSD 2-Clause "Simplified" License
10.01k stars 1.13k forks source link

object mode fails sometimes #5094

Open haimivan opened 4 years ago

haimivan commented 4 years ago

Hi,

this snippet gets called several times in my main Numba processing function:

@nb.jit(boundscheck=True, debug=True, forceobj=True)
def printing4Numba(runner, datetimeInt, par0, par1, par2, par3, par4):
   with nb.objmode():
        print('%8s' % runner, datetime.datetime.utcfromtimestamp(datetimeInt), par0, par1, par2, par3, par4)
        1 + 1

Using the directives @nb.jit(boundscheck=True, debug=True, forceobj=True) leads to this error (at least sometimes):

warning: ignoring debug info with an invalid version (0) in <string>
Traceback (most recent call last):
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/errors.py", line 717, in new_error_context
    yield
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/lowering.py", line 288, in lower_block
    self.lower_inst(inst)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/pylowering.py", line 176, in lower_inst
    raise NotImplementedError(type(inst), inst)
NotImplementedError: (<class 'numba.ir.EnterWith'>, <numba.ir.EnterWith object at 0x7f7ea4f010f0>)

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
.
.
.
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 420, in _compile_for_args
    raise e
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 353, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 768, in compile
    cres = self._compiler.compile(args, return_type)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 77, in compile
    status, retval = self._compile_cached(args, return_type)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 91, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/dispatcher.py", line 109, in _compile_core
    pipeline_class=self.pipeline_class)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler.py", line 551, in compile_extra
    return pipeline.compile_extra(func)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler.py", line 331, in compile_extra
    return self._compile_bytecode()
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler.py", line 393, in _compile_bytecode
    return self._compile_core()
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler.py", line 373, in _compile_core
    raise e
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler.py", line 364, in _compile_core
    pm.run(self.state)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_machinery.py", line 347, in run
    raise patched_exception
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_machinery.py", line 338, in run
    self._runPass(idx, pass_inst, state)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_machinery.py", line 302, in _runPass
    mutated |= check(pss.run_pass, internal_state)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/compiler_machinery.py", line 275, in check
    mangled = func(compiler_state)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/object_mode_passes.py", line 145, in run_pass
    lowered = backend_object_mode()
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/object_mode_passes.py", line 143, in backend_object_mode
    state.flags)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/object_mode_passes.py", line 100, in _py_lowering_stage
    lower.lower()
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/lowering.py", line 195, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/lowering.py", line 248, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/lowering.py", line 273, in lower_function_body
    self.lower_block(block)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/lowering.py", line 288, in lower_block
    self.lower_inst(inst)
  File "/home/user/anaconda3/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/errors.py", line 725, in new_error_context
    six.reraise(type(newerr), newerr, tb)
  File "/home/user/venv/numba/lib/python3.6/site-packages/numba/six.py", line 669, in reraise
    raise value
numba.errors.LoweringError: Failed in object mode pipeline (step: object mode backend)
(<class 'numba.ir.EnterWith'>, <numba.ir.EnterWith object at 0x7f7ea4f010f0>)

File "t_a0340_aBig_B.py", line 437:
def printing4Numba(runner, datetimeInt, par0, par1, par2, par3, par4):
   with nb.objmode():
   ^

[1] During: lowering "enter_with $phi8.1" at /home/user/PycharmProjects/0650_r_b/t_a0340_aBig_B.py (437)

-------------------------------------------------------------------------------
This should not have happened, a problem has occurred in Numba's internals.
You are currently using Numba version 0.47.0.

Please see also https://github.com/numba/numba/issues/5093

Unfortunately, I cannot share my code. Extracting it to a minimal reproducable code snippet did not work neither.

stuartarchibald commented 4 years ago

Thanks for the report. This shouldn't have happened, but equally I'm not sure compiling this code makes sense? The @jit decorator has the forceobj=True kwarg supplied which will force the compiler to compile in object mode, there's then an object mode block defined, but the compiler is already in object mode, so I'm not sure what this is trying to achieve?

To learn more about nopython vs object mode see: http://numba.pydata.org/numba-doc/latest/user/5minguide.html#what-is-nopython-mode

haimivan commented 4 years ago

Hi @stuartarchibald ,

I came across this issue when fiddling around with https://github.com/numba/numba/issues/5093 .

I am with you that it does not make really sense, but my thought was that it maybe might help to analyse that issue or others.

stuartarchibald commented 4 years ago

Thanks, whilst unusual, am labeling this as a bug as LoweringErrors should not end up in user space.