search

libyal / libesedb

Library and tools to access the Extensible Storage Engine (ESE) Database File (EDB) format.
GNU Lesser General Public License v3.0
341 stars 91 forks source link

Error exporting table: export_handle_export_basic_record_value: missing value string #50

Open vicodark opened 4 years ago

vicodark commented 4 years ago

Trying to parse a Windows.edb file from Windows 10 1803 file results in the following errors:

$ ./esedbexport -t EXPORTS Windows.edb esedbexport 20200102

Opening file. Database type: Windows Search. Exporting table 1 (MSysObjects) out of 29. Exporting table 2 (MSysObjectsShadow) out of 29. Exporting table 3 (MSysObjids) out of 29. Exporting table 4 (MSysLocales) out of 29. Exporting table 5 (CatalogManager_Properties) out of 29. Exporting table 6 (CatalogStorageManager) out of 29. Exporting table 7 (SystemIndex_Gthr) out of 29. Exporting table 8 (SystemIndex_GthrPth) out of 29. Exporting table 9 (SystemIndex_GthrAppOwner) out of 29. Exporting table 10 (SystemIndex_1_Properties) out of 29. Exporting table 11 (SystemIndex_1) out of 29. Exporting table 12 (SystemIndex_PropertyStore) out of 29. Unable to export file. export_handle_export_basic_record_value: missing value string. export_handle_export_record_value: unable to export basic record value: 305. export_handle_export_record: unable to export record value: 305. export_handle_export_table: unable to export record. export_handle_export_file: unable to export table: 11.

The errors are the same when running the last two experimental releases (libesedb-20181229 and libesedb-20191220).

I've recompiled 20200102 with --enable-verbose-output --enable-debug-output and re-ran just exporting the problematic table (SystemIndex_PropertyStore) and used the -v switch for debug output.

./esedbexport -T SystemIndex_PropertyStore -t EXPORTS -v Windows.edb > SystemIndex_PropertyStore.debug 2>&1

It's been running 20+ minutes and the file where I'm capturing the debug output is approaching 1GB which doesn't seem right, but maybe it is? The Windows.edb file is 224MB.

The tools are being run on macOS 10.14.6.

Is there something else I should be trying? Or something I'm missing?

Thanks!

joachimmetz commented 4 years ago

It's been running 20+ minutes and the file where I'm capturing the debug output is approaching 1GB which doesn't seem right, but maybe it is?

debug output can be very verbose, you can pipe to gzip to compress output and save space

vicodark commented 4 years ago

Understood. When it finishes I'll zip and attach it here.

joachimmetz commented 4 years ago

if the file is too large for github, I would be mainly interested in the record leading up to the error. Otherwise sharing via other platform works for me

Looking a bit closer at the error, it is actually a safe guard in the esedbexport tool https://github.com/libyal/libesedb/blob/55f6723917752b460141cfcae059fa52c4fc43f1/esedbtools/export_handle.c#L3580

The code is easily changed to handle/warn about this situation. Would be still interesting to know the context on why this is happening

joachimmetz commented 4 years ago

Give the changes in https://github.com/libyal/libesedb/commit/90184775511e57cd3300157594635ac4a1c9630d a try

vicodark commented 4 years ago

Excellent. With those changes it now runs to completion without spewing any errors. The older version is still running in debug. I'll be sure and get you context for the error I was seeing before. Thanks!

vicodark commented 4 years ago

Sorry, had to step away from this for a bit.

The verbose run on head from just before your fix that I spoke of above ran for over 40 hours before I just killed it. The output file at that time was 24GB. Surely that's not expected? I've attached as large a chunk of the beginning and end of the output file that github would allow me to upload. I'm happy to share more if you'd like it. Or I can give you the source Windows.edb privately.

head.txt.zip tail.txt.zip

Again, thanks for looking into all this.

joachimmetz commented 4 years ago

Sorry, had to step away from this for a bit.

Noworries, thx for getting back to me.

Surely that's not expected?

As I indicated the debug output can be very verbose especially if the table is large, it is typically better to isolate the issue first. So it's not always desirable but not unexpected.

Or I can give you the source Windows.edb privately.

This could work for me, please drop me mail. Sharing via Google Drive or equiv could work for me.

vicodark commented 4 years ago

Ah, it can be very, very, very verbose, lol. Now I understand better. I went ahead and just sent the edb file to you.

joachimmetz commented 4 years ago

excellent, thx. I'll have a look as soon as time permits, to see if I can pinpoint the reason for the observed behavior.