Question on debugging libewf

[mikebbt]

This is related to an earlier issue I posted about libewf having trouble reading a very large (540GB single segment) L01 file. I was directed to the libewf debug wiki page. I'm building libewf on MacOS as a library which another application then uses to open E0x and L0x files.

The debug wiki mentions the two configure items, --enable-verbose-output & --enable-debug-output, to build for debugging. It is my understanding from the next line in the wiki ("This will generate vast amounts of debug information on stderr when the tools are run with -v.") that this is only pertains to the EWFtools, is this correct?

When I run ewfverify against the large L01 on MacOS it fails but seems to verify fine on Windows, this seems inconsistent so I would like to debug w/o the ewftools.

Can I setup the libewf notify stream from my application and add debug traces to various libewf files?

I would open a log to disk, set verbose mode, add libcnotify_printf() calls where I want and then close out the log:

libewf_notify_stream_open() => with a path to a log file on disk libewf_notify_set_verbose( 1 )

libcnotify_printf("some debug here") libcnotify_printf("some other debug here") libcnotify_printf("more debug here")

libewf_notify_stream_close

[joachimmetz]

("This will generate vast amounts of debug information on stderr when the tools are run with -v.") that this is only pertains to the EWFtools, is this correct?

the -v option pertains to the ewftools

the vast amount of output is mainly generated by the library

Can I setup the libewf notify stream from my application and add debug traces to various libewf files?

yes, using libewf_notify_stream_open and libewf_notify_set_verbose should be able to facilitate this

I would open a log to disk, set verbose mode, add libcnotify_printf() calls where I want and then close out the log:

using libcnotify_printf in your application, this should work in most cases, but there might be corner cases where the notify stream set in the library might be different from the one set in your application

When I run ewfverify against the large L01 on MacOS it fails but seems to verify fine on Windows, this seems inconsistent so I would like to debug w/o the ewftools.

In what mode? verifying the embedded single file hashes? Maybe 32-bit versus 64-bit compilation of certain types like size_t ? Or some difference in use / version of a dependency?

[joachimmetz]

I assume the earlier issue you're referring to is https://github.com/libyal/libewf/issues/156 ?

[mikebbt]

Thanks for the quick response, issue #156 isn't the earlier issue, I can't seem to find it (perhaps I deleted it by mistake). In a nutshell I have a 540GB single file L01 that libewf couldn't open, I get "libewf_handle_get_root_file_entry: invalid handle - missing single files.".

As for the Mac vs Windows for ewftools, I'm just calling "ewftools -v " so I'm not sure which mode that is, should I be using other args to verify?

If I recall the original issue post you did also mention that perhaps the ltree was corrupt, I might agree but I did a test where that same L01 was compressed, the compressed L01 was able to be read by libewf. Unless I'm mistaken the compression doesn't effect the ltree section so if it was corrupt the L01 wouldn't have been readable regardless of it being compressed or not but could be wrong.

As for the logging, I did try what I suggested above, the log file was created where I expected but nothing was written into it so perhaps the notify stream in the library isn't what I think it is.

[joachimmetz]

If I recall the original issue post you did also mention that perhaps the ltree was corrupt,

Ah, this one likely https://github.com/libyal/libewf-legacy/issues/16

I'm just calling "ewftools -v " so I'm not sure which mode that is, should I be using other args to verify?

that is ewfverify -f raw ..., which is the default mode that ignores the ltree

ewfverify -f files ... to verify individual files, if I recall correctly, a typically L01 has no hash section (or equiv) so what ewfverify can mostly do in such cases is check for checksum mismatches and decompression errors

[mikebbt]

Was able to track down the cause of the issue, the access mask in the permissions items was being stored in the L01 as a signed integer which was a problem for libewf to read and thus failed when that value had its MSB set (negative). Will close this issue.

[joachimmetz]

@mikebbt can you tell a bit more what tool created this L01 or the add an example of the offending single file entry in the ltree section. L01 is a propriety format, so it is not fully known what valid / invalid format cases are.

[mikebbt]

The tool that created this L01 was Cellebrite's Inspector application (formerly BlackBag Technologies BlackLight app). As mentioned above the access mask was being stored in the ltree as a signed value, depending on the bits set in the mask a negative value could be stored in the Permissions Category entry. This is not valid when libewf is reading it so an error occurs and the L01 import/read fails. Will paste a segment of the perms category that failed, the value highlighted in the image (-1610612736) is what causes libewf to fail, creating the ltree with the unsigned number (2684354560) resolves the issue.

[joachimmetz]

Thx for the context, not sure if this is an issue where BlackBag Technologies came up with their own variant of the of format or that this is also supported by Encase.

[mikebbt]

The L01 exported by the Inspector product is supposed to be compatible should be able to be imported into EnCase. One other note, this L01 was also failing import to EnCase.

[joachimmetz]

Just out of curiosity do you know if after the changes to using an unsigned, Encase does accept the file?

[mikebbt]

I don't at this point but will find out and post here.