vstinner / hachoir

Hachoir is a Python library to view and edit a binary stream field by field
http://hachoir.readthedocs.io/
GNU General Public License v2.0
604 stars 70 forks source link

[bug] MemoryErrors when using subfile #87

Open agrajag9 opened 1 year ago

agrajag9 commented 1 year ago

Error output:

$ python3 -m hachoir.subfile DSN-CTL-V23R01.exe 
[+] Start search on 6475848 bytes (6.2 MB)

[+] File at 0 size=57344 (56.0 KB): Microsoft Windows Portable Executable: Intel 80386, Windows GUI
[!] Memory error!

[+] End of search -- offset=524288 (512.0 KB)
Total time: 676 ms -- global rate: 756.7 KB/sec

Can be reproduced with https://dsn-ctl.fr/DSN-CTL-V23R01.exe

$ wget https://dsn-ctl.fr/DSN-CTL-V23R01.exe
--2023-07-04 13:19:30--  https://dsn-ctl.fr/DSN-CTL-V23R01.exe
Resolving dsn-ctl.fr (dsn-ctl.fr)... 85.236.158.186
Connecting to dsn-ctl.fr (dsn-ctl.fr)|85.236.158.186|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6475848 (6.2M) [application/x-msdownload]
Saving to: ‘DSN-CTL-V23R01.exe’

DSN-CTL-V23R01.exe          100%[==========================================>]   6.18M  3.71MB/s    in 1.7s    

2023-07-04 13:19:32 (3.71 MB/s) - ‘DSN-CTL-V23R01.exe’ saved [6475848/6475848]

$ shasum -a256 DSN-CTL-V23R01.exe 
b4e855f92c4ae8cec77b9ccaf8b6e0cf53134eb47f5e668980e20afdc149d99f  DSN-CTL-V23R01.exe

Host info:

$ uname -rvm
6.2.6-76060206-generic #202303130630~1685473338~22.04~995127e SMP PREEMPT_DYNAMIC Tue M x86_64

Python version:

$ python3 -V
Python 3.10.6

Hachoir version:

$ pip show hachoir
Name: hachoir
Version: 3.2.0
Summary: Package of Hachoir parsers used to open binary files
Home-page: http://hachoir.readthedocs.io/
Author: Hachoir team (see AUTHORS file)
Author-email: 
License: GNU GPL v2
Location: /home/agrajag9/.local/lib/python3.10/site-packages
Requires: 
Required-by: 
agrajag9 commented 1 year ago

And on RHEL 7.9:

 $ python3 -m hachoir.subfile ba6981176279a3547cb62566559361d628d3f7e665f31065b5a5e420a307e5b7
[+] Start search on 6471808 bytes (6.2 MB)

[+] File at 0 size=57344 (56.0 KB): Microsoft Windows Portable Executable: Intel 80386, Windows GUI
Fatal Python error: Cannot recover from MemoryErrors while normalizing exceptions.
Python runtime state: initialized

Current thread 0x00007fdd8327f740 (most recent call first):
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 224 in guess
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 192 in findMagic
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 136 in searchSubfiles
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/core/memory.py", line 97 in limitedMemory
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/search.py", line 89 in main
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/main.py", line 73 in runSearch
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/main.py", line 96 in main
  File "/home/agrajag9/.local/lib/python3.8/site-packages/hachoir/subfile/__main__.py", line 3 in <module>
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/runpy.py", line 87 in _run_code
  File "/opt/rh/rh-python38/root/usr/lib64/python3.8/runpy.py", line 194 in _run_module_as_main
Aborted
$ uname -rvm
3.10.0-1160.92.1.el7.x86_64 #1 SMP Thu May 18 11:23:40 UTC 2023 x86_64
$ python3 -V
Python 3.8.14
agrajag9 commented 1 year ago

Verbose log with python3 -v -m hachoir.subfile DSN-CTL-V23R01.exe 2>&1 | tee hachoir.log

https://gist.github.com/agrajag9/a60a8580fbcae56999635331d1b38029