AttributeError: module 'magic' has no attribute 'from_buffer'

[noraj]

Analyzing a jped image I got

Type:   JPEG (ISO 10918)
Mode:   RGB

Checking Meta Data

Exif Data
=========

Checking for trailing data
Running image filters
Attempting to brute force LSB items
Trying 0.0.0
Traceback (most recent call last):
  File "/home/noraj/CTF/tools/stegoVeritas/stegoveritas.py", line 70, in <module>
    modules.image.run(fArray,args)
  File "/home/noraj/CTF/tools/stegoVeritas/modules/image/__init__.py", line 98, in run
    autoAnalysis(f,args)
  File "/home/noraj/CTF/tools/stegoVeritas/modules/image/__init__.py", line 37, in autoAnalysis
    modules.image.imageLSB.auto(f,args)
  File "/home/noraj/CTF/tools/stegoVeritas/modules/image/imageLSB.py", line 143, in auto
    testOutput(o,args)
  File "/home/noraj/CTF/tools/stegoVeritas/modules/image/imageLSB.py", line 23, in testOutput
    m = magic.from_buffer(b,mime=True)
AttributeError: module 'magic' has no attribute 'from_buffer'

All dependencies installed, sorry I cant share the pic.

[bannsec]

I'm currently re-factoring the code base for this tool that will, among other things, fix the dependency issues.

In the mean time, two things to check. First, be sure you're running it with python3. And secondly, you need to install python-magic:

pip install python-magic

[noraj]

Yeah I was using python3 and I have python-magic installed.

PS : If you pay attention it is not an ImportError but an AttributeError, meaning the module 'magic' is imported correctly but that in this version of the lib from_buffer is not available. I already tried to look at https://github.com/ahupp/python-magic to see if in a version change, from_buffer was removed but they have no changelog.

[bannsec]

Pip freeze?

On Fri, Feb 1, 2019, 3:54 AM Alexandre ZANNI <notifications@github.com wrote:

Yeah I was using python3 and I have python-magic installed.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bannsec/stegoVeritas/issues/10#issuecomment-459651289, or mute the thread https://github.com/notifications/unsubscribe-auth/AHR34LKSLnCSVxTU99CHz39cGfGh0mJVks5vJADDgaJpZM4adcnA .

[bannsec]

well i ask because i believe there are multiple python projects that register 'magic' as an import, and thus the error would be due to the magic module being a different one than expected.

[noraj]

appdirs==1.4.3
argcomplete==1.9.4
asciinema==2.0.2
asn1crypto==0.24.0
bcrypt==3.1.4
Beaker==1.10.0
beautifulsoup4==4.6.3
binwalk==2.1.1
blinker==1.4
brotlipy==0.7.0
btrfsutil==1.1.0
CacheControl==0.12.5
cached-property==1.5.1
capstone==3.0.5
catfish==1.4.6
cffi==1.11.5
chardet==3.0.4
Click==7.0
colorama==0.4.1
cryptography==2.4.2
cupshelpers==1.0
distlib==0.2.8
distro==1.3.0
dnspython==1.16.0
docker==3.7.0
docker-compose==1.23.2
docker-pycreds==0.4.0
dockerpty==0.4.1
docopt==0.6.2
drupwn==0.9.2
fake2db==0.5.4
Faker==0.7.11
file-magic==0.4.0
Flask==1.0.2
future==0.17.1
gufw==18.10.0
h11==0.8.1
h2==3.0.1
hpack==3.0.0
html5lib==1.0.1
hyperframe==5.2.0
idna==2.8
intervaltree==2.1.0
isc==2.0
itsdangerous==0.24
Jinja2==2.10
jsonschema==2.6.0
kaitaistruct==0.8
ldap3==2.5.2
lensfun==0.3.2
lightdm-gtk-greeter-settings==1.2.2
lockfile==0.12.2
louis==3.8.0
Mako==1.0.7
MarkupSafe==1.1.0
menulibre==2.2.0
mitmproxy==4.0.4
msgpack==0.6.0
mugshot==0.4.1
npyscreen==4.10.5
numpy==1.15.4
olefile==0.46
packaging==18.0
pacman-mirrors==4.14.1
paramiko==2.4.1
passlib==1.7.1
pep517==0.5.0
pexpect==4.6.0
pikaur==1.3.1
Pillow==5.4.1
pluggy==0.7.1
ply==3.11
progress==1.4
prompt-toolkit==2.0.7
protobuf==3.6.1
psutil==5.4.8
ptyprocess==0.6.0
pwntools==2.2.1
py==1.6.0
pyalpm==0.8.5
pyasn1==0.4.5
pycairo==1.18.0
pycparser==2.19
pycups==1.9.74
pycurl==7.43.0.2
pyelftools==0.25
Pygments==2.3.1
PyGObject==3.30.4
PyNaCl==1.2.1
pyOpenSSL==18.0.0
pypandoc==1.4
pyparsing==2.3.1
pyperclip==1.6.5
PyQt5==5.11.3
PyQt5-sip==4.19.13
pyserial==3.4
PySocks==1.6.8
python-dateutil==2.7.3
python-distutils-extra==2.39
python-libtorrent==1.1.11
python-ptrace==0.9.3
pytoml==0.1.20
pyxdg==0.26
PyYAML==3.13
requests==2.21.0
retrying==1.3.3
ROPGadget==5.4
ruamel.yaml==0.15.86
simplejson==3.16.1
sip==4.19.13
six==1.12.0
sortedcontainers==2.1.0
team==1.0
termcolor==1.1.0
texttable==1.6.0
tld==0.9.2
tornado==5.1.1
tox==3.2.1
ufw==0.36
unicorn==1.0.1
urllib3==1.24.1
urwid==2.0.1
veryprettytable==0.8.1
virtualenv==16.0.0
wapiti3==3.0.1
wcwidth==0.1.7
webencodings==0.5.1
websocket-client==0.54.0
Werkzeug==0.14.1
wsproto==0.12.0
yara-python==3.8.1
yaswfp==0.9.3

[bannsec]

Doesn't look like you have python-magic installed. For instance, in my installs pip freeze has the following:

python-magic==0.4.15

[noraj]

Here with python-magic package:

https://www.archlinux.org/packages/community/any/python-magic/

python-magic = file-magic

But that's the wrong dependency !! In fact we need python-python-magic from blackarch repo !

And yes this time we have python-magic==0.4.15 but this is totally conflicting with the official python-magic from arch repo:

sudo pacman -S python-python-magic 
resolving dependencies...
looking for conflicting packages...

Packages (1) python-python-magic-0.4.15-1

Total Download Size:   0,01 MiB
Total Installed Size:  0,05 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages...
 python-python-magic-0.4.15-1-any                                                                       9,0 KiB  0,00B/s 00:00 [#############################################################################] 100%
(1/1) checking keys in keyring                                                                                                 [#############################################################################] 100%
(1/1) checking package integrity                                                                                               [#############################################################################] 100%
(1/1) loading package files                                                                                                    [#############################################################################] 100%
(1/1) checking for file conflicts                                                                                              [#############################################################################] 100%
error: failed to commit transaction (conflicting files)
python-python-magic: /usr/lib/python3.7/site-packages/__pycache__/magic.cpython-37.opt-1.pyc exists in filesystem (owned by python-magic)
python-python-magic: /usr/lib/python3.7/site-packages/__pycache__/magic.cpython-37.pyc exists in filesystem (owned by python-magic)
python-python-magic: /usr/lib/python3.7/site-packages/magic.py exists in filesystem (owned by python-magic)
Errors occurred, no packages were upgraded.

python-magic is installed like this and python-python-magic like this.

This is really a problem for packaging when two lib have exactly the same name and are doing nearly the same thing. Like libtorrent and libtorrent. So we need a disambiguation notice on the readme.

[noraj]

Plus this is really a problem to rely on a library that conflicts with others. I know you'll tell me that you said pip3 install python-magic but installing package with pip is the worst idea ever. One would want to keep is system clean and only install distro package in order to avoid conflict between python distro package and pip python package on the filesystem. pip should be only use in a virtualenv.

[bannsec]

Agree with the virtualenv, which is how I run it locally. I've pushed out the re-factored version which is a python package instead of git clone. Two methods for install:

$ pip3 install stegoveritas
$ stegoveritas_install_deps

And if you're still having dependency issues, I'd suggest running the Docker version:

$ sudo docker run -it --rm bannsec/stegoveritas

[Tmanyere]

uninstall python-magic

install this 1 python-magic-bin==0.4.14

[PepoPalo]

uninstall python-magic

install this 1 python-magic-bin==0.4.14

I'm trying with this solution and doesn't work, get this:

ERROR: Could not find a version that satisfies the requirement python-magic-bin==0.4.14 (from versions: none) ERROR: No matching distribution found for python-magic-bin==0.4.14

[bannsec]

I believe all the OS tests are still passing. If you're having trouble with dependencies it's probably best to install it into it's own python virtualenv.

[bannsec]

you'll have to be more specific. your link is to langchain. also all the unit tests on the different OSes are building and passing.