Clang parsing failure during patch generation (libtiff/CVE-2016-10094)

[rshariffdeen]

Repair module cannot find fix-location, ignores developer fix location

[2023-03-30 22:22:05.639] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:05.646] [info] checking location: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6
[2023-03-30 22:22:05.783] [warning] statement finder did not encounter an AST for the given file: /data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c
[2023-03-30 22:22:05.783] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:05.790] [info] checking location: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6
[2023-03-30 22:22:05.933] [warning] statement finder did not encounter an AST for the given file: /data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c
[2023-03-30 22:22:05.933] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:05.940] [info] checking location: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6
tif_unix.c:56:10: fatal error: 'tiffiop.h' file not found
#include "tiffiop.h"
         ^~~~~~~~~~~
[2023-03-30 22:22:05.977] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:05.977] [warning] statement finder did not encounter an AST for the given file: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c
[2023-03-30 22:22:05.977] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6]: unable to find statement
1 error generated.
Error while processing /data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c.
tif_unix.c:56:10: fatal error: 'tiffiop.h' file not found
#include "tiffiop.h"
         ^~~~~~~~~~~
[2023-03-30 22:22:06.001] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:06.001] [warning] statement finder did not encounter an AST for the given file: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c
[2023-03-30 22:22:06.001] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6]: unable to find statement
1 error generated.
Error while processing /data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c.
tif_unix.c:56:10: fatal error: 'tiffiop.h' file not found
#include "tiffiop.h"
         ^~~~~~~~~~~
[2023-03-30 22:22:06.033] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c:340:2]: unable to find statement
[2023-03-30 22:22:06.033] [warning] statement finder did not encounter an AST for the given file: /data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c
[2023-03-30 22:22:06.033] [error] bad fix location [/data/vulnloc/libtiff/CVE-2016-10094/src/tools/tiff2pdf.c:2901:6]: unable to find statement
1 error generated.
Error while processing /data/vulnloc/libtiff/CVE-2016-10094/src/libtiff/tif_unix.c.

[ChrisTimperley]

This is the same failure mode as #134. Dealing with this is tricky since it's an issue in Clang itself and not in the way that we interact with LibTooling. Possibly the best intervention is for us to preprocess src/libtiff/tif_unix.c. I'll take a look into it.

[ChrisTimperley]

I'm in the process of testing out a promising fix to these libtiff parsing issues 🤞🏻

[ChrisTimperley]

I manually performed the following change (this will eventually go into setup.sh):

cd src
clang -I . -I ./libtiff -E tools/tiffcrop.c > tools/tiffcrop.c.pre
mv tools/tiffcrop.c.pre tools/tiffcrop.c

Unfortunately I now get a crash during the analysis phase:

        Generating Fix Locations
        __________________________________________________________________________________________

                generating taint map
                             [info] found 6210 tainted locations
                        starting parallel computing
                        waiting for thread completion
                             [info] found 7 source files
        Runtime Error
        [Errno 2] No such file or directory: '/data/vulnloc/libtiff/CVE-2016-10092/src/tools/tools/tiffcrop.c.ast'

[rshariffdeen]

Was there additional changes? the path seems to be wrong, there is no src/tools/tools directory?

[ChrisTimperley]

Was there additional changes? the path seems to be wrong, there is no src/tools/tools directory?

Those were the only changes. You should be able to reproduce.

[rshariffdeen]

I think the preprocessor comments created changes the debug meta-data so the debug information shows the path as tools/tools/tiffcrop.c instead of tools/tiffcrop.c

If we remove the pre-processing comments this will work, which is what I did for other subjects when pre-processing.

clang -I . -I ./libtiff -E tools/tiffcrop.c > tools/tiffcrop.c.pre
grep -o '^[^#]*' tools/tiffcrop.c.pre > tools/tiffcrop.c

[rshariffdeen]

the command should be updated as below:

grep -v '^ *#' tools/tiffcrop.c.pre > tools/tiffcrop.c.post

[ChrisTimperley]

The above does the trick. I'll update the setup.sh files.