`binwalk -E` - NameError: name 'np' is not defined

[rflmota]

Hello,

When running latest Kali 2021.1 and using binwalk -E for checking the entropy graph of a file, I get the following error:

DECIMAL       HEXADECIMAL     ENTROPY
--------------------------------------------------------------------------------

Entropy Exception: Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'np' is not defined
----------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1+80a519b-py3.9.egg/binwalk/core/module.py", line 595, in main
    retval = self.run()
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1+80a519b-py3.9.egg/binwalk/modules/entropy.py", line 136, in run
    self._run()
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1+80a519b-py3.9.egg/binwalk/modules/entropy.py", line 158, in _run
    self.calculate_file_entropy(fp)
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1+80a519b-py3.9.egg/binwalk/modules/entropy.py", line 199, in calculate_file_entropy
    entropy = self.algorithm(data[i:i + block_size])
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1+80a519b-py3.9.egg/binwalk/modules/entropy.py", line 256, in shannon_numpy
    return self._shannon_numpy(bytes2str(data))
  File "/usr/lib/python3/dist-packages/numba/core/dispatcher.py", line 414, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/usr/lib/python3/dist-packages/numba/core/dispatcher.py", line 357, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'np' is not defined
----------------------------------------------------------------------------------------------------

I ingenuously thought that could be due to missing dependency like numpy as np and tried installing it with pip but I still get the same error...

I also cloned the source from this repo and tried installing it according to instructions provided on Wiki but I still get with same error, both with master and last tagged version v2.3.1.

I've also noted that although checking out tagged version v.2.3.1, install script setup.py still contains MODULE_VERSION = "2.2.1", is this expected?

Thank you for your attention. KR

[Max5561]

Same problem here.

DECIMAL       HEXADECIMAL     ENTROPY
--------------------------------------------------------------------------------

Entropy Exception: Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'np' is not defined
----------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1-py3.9.egg/binwalk/core/module.py", line 595, in main
    retval = self.run()
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1-py3.9.egg/binwalk/modules/entropy.py", line 136, in run
    self._run()
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1-py3.9.egg/binwalk/modules/entropy.py", line 158, in _run
    self.calculate_file_entropy(fp)
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1-py3.9.egg/binwalk/modules/entropy.py", line 199, in calculate_file_entropy
    entropy = self.algorithm(data[i:i + block_size])
  File "/usr/local/lib/python3.9/dist-packages/binwalk-2.2.1-py3.9.egg/binwalk/modules/entropy.py", line 256, in shannon_numpy
    return self._shannon_numpy(bytes2str(data))
  File "/home/kali/.local/lib/python3.9/site-packages/numba/core/dispatcher.py", line 420, in _compile_for_args
    error_rewrite(e, 'typing')
  File "/home/kali/.local/lib/python3.9/site-packages/numba/core/dispatcher.py", line 361, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
NameError: name 'np' is not defined
----------------------------------------------------------------------------------------------------

[init-js]

I get the same issue, but only the second time I use the module.

import binwalk.modules
# this works and I can retrieve results
e = binwalk.Modules().execute("-E", "-J", "--log=foo.csv", "corpus.zip")
# this second invocation fails to resolve 'np'
e = binwalk.Modules().execute("-E", "-J", "--log=foo.csv", "corpus.zip")

I get the same error as @Max5561

Using revision 74b2a6f9de7493141c28dfecddfbb5a2250416c5, and python 3.9.5 on macos, inside a virtual environment. installed by cloning the git repo and doing pip install -e binwalk.

[init-js]

I believe the bug is due to another module importing numpy, and then the bit that decides which shannon implementation to use breaks:

if self.use_zlib:
            self.algorithm = self.gzip
        else:
            if 'numpy' in sys.modules:
                self.algorithm = self.shannon_numpy
            else:
                self.algorithm = self.shannon

first time, numpy is not available, and it's all good. second time (probably because I've plotted something and imported numpy as a side effect, the broken codepath is used).

as a workaround, we can either load the modules in a different order (-E before -J), or just patch out this section to use the non-numpy implementation.

[yueqi77]

I also had this problem, and then I used -e and it was solved

[edheliel]

I had the same problem. However after some reading on the problem I found a temporary solution. The whole issue was due to Kali Linux having the apt package python3-numpy, which was also mixed with the pip package 🤷 .

What I did was to essentially remove the apt package and then re-install the pip one.

sudo apt remove python3-numpy
pip3 install numpy

Hope this helps someone 😸

[APOORVMAGAN]

Thanks @edheliel it worked for me

[recepbalibey]

Thank you @edheliel , it worked for me as well!

[alexdetrano]

is there a fix for mac os x? i'm having this problem with python 3.11.5 and numpy 1.24.4 installed via pip

[PennRobotics]

It's possible the dependencies for matplotlib and numba and numpy are all not currently compatible. When I uninstall numpy entirely, I get an error that matplotlib failed to import. I then install this, which brings numpy 1.26.4, but then I get an error in numba: "SystemError: initialization of _internal failed without raising an exception". If I then install numba, numpy 1.23.5 is installed.

This is on Python 3.10.12 in WSL2. The package manager numpy is 1.21.5.

Removing numpy entirely allows entropy calculation without a plot. …good enough for what I need

[xuanxuanblingbling]

python3 -m pip uninstall numba

[dlg383]

I solved it by typing pip3 install numpy numba