search

Cisco-Talos / clamav

ClamAV - Documentation is here: https://docs.clamav.net
https://www.clamav.net/
GNU General Public License v2.0
4.22k stars 687 forks source link

[armhf] Unaligned memory access causes clamav to crash #1025

Closed vpa1977 closed 1 week ago

vpa1977 commented 1 year ago

Describe the bug

The test suite for clamav fails on armhf platform due to the unaligned memory access[1]. Relevant part of the build log:

 =================================== FAILURES ===================================
3: _______________________________ TC.test_allmatch _______________________________
3: 
3: self = <heuristics_test.TC testMethod=test_allmatch>
3: 
3:     def test_allmatch(self):
3:         '''
3:         This test uses a ZIP that has two things:
3:         1. malformed file that will alert with  --alert-broken-media
3:         2. the clam.exe file that will alert normally.
3:         The idea is that since the malformed file is first, the heuristic alert will be encountered first.
3:     
3:         In this test we use --allmatch but we don't use --heuristic-scan-precedence.
3:         That means the NDB sig should alert first, even though the heuristic is encountered first.
3:         Note the verify_output() uses STRICT_ORDER.
3:         '''
3:         self.step_name('Test that a clam heuristic alert will alert LAST in allmatch mode without heuristic-precedence.')
3:     
3:         testfile = TC.heuristics_testfile
3:     
3:         command = '{valgrind} {valgrind_args} {clamscan} -d {clam_exe_db} {testfiles} --alert-broken-media \
3:              --allmatch'.format(
3:             valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
3:             clam_exe_db=TC.path_tmp / 'clam.ndb',
3:             testfiles=testfile,
3:         )
3:         output = self.execute_command(command)
3:     
3: >       assert output.ec == 1  # virus
3: E       AssertionError: assert 135 == 1
3: E        +  where 135 = CmdResult(ec=135, out='', err='LibClamAV Warning: PNG: Unexpected early end-of-file.\nBus error (core dumped)\n').ec
3: 
3: /<<PKGBUILDDIR>>/unit_tests/clamscan/heuristics_test.py:158: AssertionError
3: ----------------------------- Captured stdout call -----------------------------
3: 
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [INFO]: Test that a clam heuristic alert will alert LAST in allmatch mode without heuristic-precedence.
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [DEBUG]: Run command:   /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/clamscan/clamscan -d /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/clam.ndb /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip --alert-broken-media              --allmatch 
3: [DEBUG]: Exit code: 135
3: [DEBUG]: stdout: 
3: [DEBUG]: stderr: LibClamAV Warning: PNG: Unexpected early end-of-file.
3: Bus error (core dumped)
3: 
3: 
3: _________________________ TC.test_allmatch_precedence __________________________
3: 
3: self = <heuristics_test.TC testMethod=test_allmatch_precedence>
3: 
3:     def test_allmatch_precedence(self):
3:         '''
3:         This test uses a ZIP that has two things:
3:         1. malformed file that will alert with  --alert-broken-media
3:         2. the clam.exe file that will alert normally.
3:         The idea is that since the malformed file is first, the heuristic alert will be encountered first.
3:     
3:         In this test we use --allmatch AND we use --heuristic-scan-precedence.
3:         That means the heuristic is encountered first and should be treated equally, so it should alert first.
3:         Note the verify_output() uses STRICT_ORDER.
3:         '''
3:         self.step_name('Test that a clam heuristic alert will alert FIRST in allmatch mode with heuristic-precedence.')
3:     
3:         testfile = TC.heuristics_testfile
3:     
3:         command = '{valgrind} {valgrind_args} {clamscan} -d {clam_exe_db} {testfiles} --alert-broken-media \
3:              --allmatch \
3:              --heuristic-scan-precedence'.format(
3:             valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
3:             clam_exe_db=TC.path_tmp / 'clam.ndb',
3:             testfiles=testfile,
3:         )
3:         output = self.execute_command(command)
3:     
3: >       assert output.ec == 1  # virus
3: E       AssertionError: assert 135 == 1
3: E        +  where 135 = CmdResult(ec=135, out='/<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip: Heuristics.Broken.Media.PNG.EOFReadingChunk FOUND\n', err='LibClamAV Warning: PNG: Unexpected early end-of-file.\nBus error (core dumped)\n').ec
3: 
3: /<<PKGBUILDDIR>>/unit_tests/clamscan/heuristics_test.py:190: AssertionError
3: ----------------------------- Captured stdout call -----------------------------
3: 
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [INFO]: Test that a clam heuristic alert will alert FIRST in allmatch mode with heuristic-precedence.
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [DEBUG]: Run command:   /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/clamscan/clamscan -d /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/clam.ndb /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip --alert-broken-media              --allmatch              --heuristic-scan-precedence 
3: [DEBUG]: Exit code: 135
3: [DEBUG]: stdout: /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip: Heuristics.Broken.Media.PNG.EOFReadingChunk FOUND
3: 
3: [DEBUG]: stderr: LibClamAV Warning: PNG: Unexpected early end-of-file.
3: Bus error (core dumped)
3: 
3: 
3: ______________________ TC.test_hidden_by_strong_indicator ______________________
3: 
3: self = <heuristics_test.TC testMethod=test_hidden_by_strong_indicator>
3: 
3:     def test_hidden_by_strong_indicator(self):
3:         '''
3:         This test uses a ZIP that has two things:
3:         1. malformed file that will alert with  --alert-broken-media
3:         2. the clam.exe file that will alert normally.
3:         The idea is that since the malformed file is first, the heuristic alert will be encountered first.
3:     
3:         In this test the heuristic alert must not alert because neither allmatch is specified, nor --heuristic-scan-precedence
3:         '''
3:         self.step_name('Test that a clam heuristic not alert because regular sig alerts first.')
3:     
3:         testfile = TC.heuristics_testfile
3:     
3:         command = '{valgrind} {valgrind_args} {clamscan} -d {clam_exe_db} {testfiles} --alert-broken-media'.format(
3:             valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
3:             clam_exe_db=TC.path_tmp / 'clam.ndb',
3:             testfiles=testfile,
3:         )
3:         output = self.execute_command(command)
3:     
3: >       assert output.ec == 1  # virus
3: E       AssertionError: assert 135 == 1
3: E        +  where 135 = CmdResult(ec=135, out='', err='LibClamAV Warning: PNG: Unexpected early end-of-file.\nBus error (core dumped)\n').ec
3: 
3: /<<PKGBUILDDIR>>/unit_tests/clamscan/heuristics_test.py:70: AssertionError
3: ----------------------------- Captured stdout call -----------------------------
3: 
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [INFO]:  Test that a clam heuristic not alert because regular sig alerts first. 
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [DEBUG]: Run command:   /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/clamscan/clamscan -d /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/clam.ndb /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip --alert-broken-media 
3: [DEBUG]: Exit code: 135
3: [DEBUG]: stdout: 
3: [DEBUG]: stderr: LibClamAV Warning: PNG: Unexpected early end-of-file.
3: Bus error (core dumped)
3: 
3: 
3: ______________________________ TC.test_only_heur _______________________________
3: 
3: self = <heuristics_test.TC testMethod=test_only_heur>
3: 
3:     def test_only_heur(self):
3:         '''
3:         This test uses a ZIP that has two things:
3:         1. malformed file that will alert with  --alert-broken-media
3:         2. the clam.exe file that will alert normally.
3:         The idea is that since the malformed file is first, the heuristic alert will be encountered first.
3:     
3:         In this test the heuristic alert must alert because we don't use the sig for the other file.
3:         '''
3:         self.step_name('Test that a clam heuristic will alert, because it is the only detection.')
3:     
3:         testfile = TC.heuristics_testfile
3:     
3:         # Add an empty NDB file, because we need to pass in some sort of database.
3:         (TC.path_tmp / 'empty.ndb').write_text(
3:             "# Just a comment\n"
3:         )
3:     
3:         command = '{valgrind} {valgrind_args} {clamscan} -d {clam_exe_db} {testfiles} --alert-broken-media'.format(
3:             valgrind=TC.valgrind, valgrind_args=TC.valgrind_args, clamscan=TC.clamscan,
3:             clam_exe_db=TC.path_tmp / 'empty.ndb',
3:             testfiles=testfile,
3:         )
3:         output = self.execute_command(command)
3:     
3: >       assert output.ec == 1  # virus
3: E       AssertionError: assert 135 == 1
3: E        +  where 135 = CmdResult(ec=135, out='', err='LibClamAV Warning: PNG: Unexpected early end-of-file.\nBus error (core dumped)\n').ec
3: 
3: /<<PKGBUILDDIR>>/unit_tests/clamscan/heuristics_test.py:101: AssertionError
3: ----------------------------- Captured stdout call -----------------------------
3: 
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [INFO]: Test that a clam heuristic will alert, because it is the only detection.
3: [INFO]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: [DEBUG]: Run command:   /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/clamscan/clamscan -d /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/empty.ndb /<<PKGBUILDDIR>>/obj-arm-linux-gnueabihf/unit_tests/TC-gk3tl4fz/heuristics-test.zip --alert-broken-media 
3: [DEBUG]: Exit code: 135
3: [DEBUG]: stdout: 
3: [DEBUG]: stderr: LibClamAV Warning: PNG: Unexpected early end-of-file.
3: Bus error (core dumped)
3: 
3: 
3: =========================== short test summary info ============================
3: FAILED clamscan/heuristics_test.py::TC::test_allmatch - AssertionError: asser...
3: FAILED clamscan/heuristics_test.py::TC::test_allmatch_precedence - AssertionE...
3: FAILED clamscan/heuristics_test.py::TC::test_hidden_by_strong_indicator - Ass...
3: FAILED clamscan/heuristics_test.py::TC::test_only_heur - AssertionError: asse...
3: ======================== 4 failed, 45 passed in 27.13s =========================

In Ubuntu the following patch was used to resolve this particular issue[2]

How to reproduce the problem

on armhf

/usr/bin/pytest "-v" "clamd_test.py"

Attachments

If applicable, add screenshots to help explain your problem.

If the issue is reproducible only when scanning a specific file, attach it to the ticket.

[1] https://launchpadlibrarian.net/681419175/buildlog_ubuntu-mantic-armhf.clamav_1.0.1+dfsg-2ubuntu1_BUILDING.txt.gz [2] https://git.launchpad.net/ubuntu/+source/clamav/tree/debian/patches/resolve-armhf-ftbfs.patch

micahsnyder commented 1 week ago

I think this may be fixed by https://github.com/Cisco-Talos/clamav/pull/1293 which has been applied to 1.4.0, 1.3.2, 1.0.7, and 0.103.12