search

numba / numba

NumPy aware dynamic Python compiler using LLVM
https://numba.pydata.org/
BSD 2-Clause "Simplified" License
9.91k stars 1.12k forks source link

numba.errors.LoweringError: Failed at nopython (nopython mode backend) #5415

Closed MARMOTatZJU closed 4 years ago

MARMOTatZJU commented 4 years ago

I encounter the following problem and hereby I report it to Numba Dev. team.

Traceback (most recent call last):        
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/errors.py", line 662, in new_error_context
    yield                                          
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 258, in lower_block  
    self.lower_inst(inst)          
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 301, in lower_inst                                                                                                                                                                       
    val = self.lower_assign(ty, inst)                                                                                 
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 459, in lower_assign                         
    return self.lower_expr(ty, value)                                                                                         
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 919, in lower_expr               
    res = self.lower_call(resty, expr)                                                                           
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 711, in lower_call
    res = self._lower_call_normal(fnty, expr, signature)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 890, in _lower_call_normal
    res = impl(self.builder, argvals, self.loc)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/targets/base.py", line 1132, in __call__
    res = self._imp(self._context, builder, self._sig, args, loc=loc)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/targets/base.py", line 1157, in wrapper
    return fn(*args, **kwargs)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/targets/arrayobj.py", line 3375, in numpy_zeros_nd
    _zero_fill_array(context, builder, ary)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/targets/arrayobj.py", line 3305, in _zero_fill_array
    cgutils.memset(builder, ary.data, builder.mul(ary.itemsize, ary.nitems), 0)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/cgutils.py", line 866, in memset
    builder.call(fn, [ptr, value, size, int32_t(0), bool_t(0)])
  File "/home/xuyinda/.local/lib/python3.6/site-packages/llvmlite/ir/builder.py", line 841, in call
    cconv=cconv, tail=tail, fastmath=fastmath)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/llvmlite/ir/instructions.py", line 84, in __init__
    raise TypeError(msg)
TypeError: Type of #4 arg mismatch: i1 != i32

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./main/test.py", line 51, in <module>
    tester.test()
  File "/home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/engine/tester/tester_impl/vot.py", line 89, in test
    self.evaluation()
  File "/home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/engine/tester/tester_impl/vot.py", line 191, in evaluation
    ret = benchmark.eval(self.tracker_name)
  File "/home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/evaluation/vot_benchmark/pysot/evaluation/eao_benchmark.py", line 53, in eval
    eao = self._calculate_eao(tracker_name, self.tags)
  File "/home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/evaluation/vot_benchmark/pysot/evaluation/eao_benchmark.py", line 245, in _calculate_eao
    expected_overlaps = calculate_expected_overlap(fragments, fweights)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 395, in _compile_for_args
    raise e
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 352, in _compile_for_args
    return self.compile(tuple(argtypes))
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 693, in compile
    cres = self._compiler.compile(args, return_type)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 76, in compile
    status, retval = self._compile_cached(args, return_type)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 90, in _compile_cached
    retval = self._compile_core(args, return_type)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/dispatcher.py", line 108, in _compile_core
    pipeline_class=self.pipeline_class)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 972, in compile_extra
    return pipeline.compile_extra(func)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 390, in compile_extra
    return self._compile_bytecode()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 903, in _compile_bytecode
    return self._compile_core()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 890, in _compile_core
    res = pm.run(self.status)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler_lock.py", line 32, in _acquire_compile_lock
    return func(*args, **kwargs)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 266, in run
    raise patched_exception
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 257, in run
    stage()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 764, in stage_nopython_backend
    self._backend(lowerfn, objectmode=False)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 703, in _backend
    lowered = lowerfn()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 690, in backend_nopython_mode
    self.metadata)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/compiler.py", line 1143, in native_lowering_stage
    lower.lower()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 177, in lower
    self.lower_normal_function(self.fndesc)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 218, in lower_normal_function
    entry_block_tail = self.lower_function_body()
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 243, in lower_function_body
    self.lower_block(block)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/lowering.py", line 258, in lower_block
    self.lower_inst(inst)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/errors.py", line 670, in new_error_context
    six.reraise(type(newerr), newerr, tb)
  File "/home/xuyinda/.local/lib/python3.6/site-packages/numba/six.py", line 659, in reraise
    raise value
numba.errors.LoweringError: Failed in nopython mode pipeline (step: nopython mode backend)
Type of #4 arg mismatch: i1 != i32

File "videoanalyst/evaluation/vot_benchmark/pysot/utils/statistics.py", line 166:
def calculate_expected_overlap(fragments, fweights):
    <source elided>
    max_len = fragments.shape[1]
    expected_overlaps = np.zeros((max_len), np.float32)
    ^

[1] During: lowering "$0.10 = call $0.6(max_len, $0.9, func=$0.6, args=[Var(max_len, /home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/evaluation/vot_benchmark/pysot/utils/statistics.py (165)), Var($0.9, /home/xuyinda/Documents/Projects/video_analyst-
debug/videoanalyst/evaluation/vot_benchmark/pysot/utils/statistics.py (166))], kws=(), vararg=None)" at /home/xuyinda/Documents/Projects/video_analyst-debug/videoanalyst/evaluation/vot_benchmark/pysot/utils/statistics.py (166)
-------------------------------------------------------------------------------
This should not have happened, a problem has occurred in Numba's internals.
You are currently using Numba version 0.45.0.

Please report the error message and traceback, along with a minimal reproducer
at: https://github.com/numba/numba/issues/new

If more help is needed please feel free to speak to the Numba core developers
directly at: https://gitter.im/numba/numba

Thanks in advance for your help in improving Numba!

Awaiting for your reply and solution. Thanks in advance!

MARMOTatZJU commented 4 years ago

More details:

Version

I have tried versions including 0.39.0 / 0. 42.0 0.45.0. None of them fixed this problem

Code

Error line

    expected_overlaps = np.zeros((max_len), np.float32)

Context

@jit(nopython=True)
def calculate_expected_overlap(fragments, fweights):
    max_len = fragments.shape[1]
    expected_overlaps = np.zeros((max_len), np.float32)
    expected_overlaps[0] = 1

    # TODO Speed Up
    for i in range(1, max_len):
        mask = np.logical_not(np.isnan(fragments[:, i]))
        if np.any(mask):
            fragment = fragments[mask, 1:i + 1]
            seq_mean = np.sum(fragment, 1) / fragment.shape[1]
            expected_overlaps[i] = np.sum(seq_mean * fweights[mask]) / np.sum(
                fweights[mask])
    return expected_overlaps

A simple solution: comment the jit line

# @jit(nopython=True)

This solve the problem

sklam commented 4 years ago

The bug has been fixed since 0.47.0.

reproducer ```python from numba import njit import numpy as np @njit def calculate_expected_overlap(fragments, fweights): max_len = fragments.shape[1] expected_overlaps = np.zeros((max_len), np.float32) expected_overlaps[0] = 1 # TODO Speed Up for i in range(1, max_len): mask = np.logical_not(np.isnan(fragments[:, i])) if np.any(mask): fragment = fragments[mask, 1:i + 1] seq_mean = np.sum(fragment, 1) / fragment.shape[1] expected_overlaps[i] = np.sum(seq_mean * fweights[mask]) / np.sum( fweights[mask]) return expected_overlaps fragments = np.random.random((10, 10)) fweights = np.random.random(10) r = calculate_expected_overlap(fragments, fweights) print(r) ```