search

ahupp / python-magic

A python wrapper for libmagic
Other
2.64k stars 283 forks source link

magic.MagicException: b'error reading (Invalid argument)' #194

Closed paul-nameless closed 2 years ago

paul-nameless commented 5 years ago

I got exception with some files:

>>> magic.from_file("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72")
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    magic.from_file("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72")
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 136, in from_file
    return m.from_file(filename)
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 92, in from_file
    return self._handle509Bug(e)
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 101, in _handle509Bug
    raise e
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 90, in from_file
    return maybe_decode(magic_file(self.cookie, filename))
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 247, in magic_file
    return _magic_file(cookie, coerce_filename(filename))
  File "/Users/paul/Documents/job/venv/lib/python3.7/site-packages/magic.py", line 188, in errorcheck_null
    raise MagicException(err)
magic.MagicException: b'error reading (Invalid argument)'

> file 078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72

078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter b64/ld-l, missing section headers
ahupp commented 5 years ago

This is an error from inside libmagic, rather than the python-magic wrapper. Does this only happen on some files? Does something like:

magic.from_buffer(open(fname).read(1024))

give the same result?

paul-nameless commented 5 years ago 
>>> magic.from_buffer(open("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72").read(1024))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    magic.from_buffer(open("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72").read(1024))
  File "/Users/paul/Documents/job/venv/bin/../lib/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 25: invalid continuation byte
>>> magic.from_buffer(open("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72", 'rb').read(1024))
'ELF 64-bit LSB executable, x86-64, version 1 (SYSV)'

This is what I got. Is it python level problem or libmagic one? Should I investigate further it there? Is it valid to use magic library like this?

magic.from_buffer(open("078a38e0bb7962fb145f75fbd5099321162c108ecb66123c88032f1c620aad72", 'rb').read(1024))
ahupp commented 5 years ago

Its a problem inside libmagic, which unfortunately makes it a bit harder to track down. It looks like there's some elf-specific logic that can trigger this read error, possibly here:

https://github.com/threatstack/libmagic/blob/1249b5cd02c3b6fb9b917d16c76bc76c862932b6/src/readelf.c#L310

Is the file truncated in some weird way? If you pass the entire file to from_buffer do you get the same result?