libyal / libewf-legacy

Legacy version of libewf
GNU Lesser General Public License v3.0
10 stars 5 forks source link

Should libewf_handle_close usually take quite some time to complete? #12

Closed tedsmith closed 2 years ago

tedsmith commented 3 years ago

@joachimmetz

My implementation of libewf_handle_close seems to take a very long time in my Windows 10 virtual machine with 16Gb RAM. Several minutes it seems. During this process, the memory in use according to task manager gradually drops, from a few hundred Mb to the normal startup usage of about 40Mb.

I want to check this is normal\expected (for it to take a while), or work out if I am perhaps mis-using it? I am using only these functions:

libewf_handle_open libewf_handle_get_media_size libewf_handle_read_buffer libewf_handle_get_utf8_hash_value libewf_handle_close

And the segment of my wrapper call to libewf_handle_close is :

if libewf_version='V2' then
      begin
      Result:=flibewfhandleclose (fCurEWFHandle,@err);
      if result=0 then result:=flibewfhandlefree (@fCurEWFHandle,@err);
      end;      
joachimmetz commented 3 years ago

Depends can you provide a bit more context how you are using libewf? Are you reading or writing? How many segment files? etc.

tedsmith commented 3 years ago

Reading only. In my example, it has 11 E01 segments, and the image was captured using a Falcon device but I've repeated using a test image of the same source device that was captured using X-Ways Forensics. In both cases, memory starts at around 30Mb usage, and climbs to about 400Mb, and then returns to 30Mb after close.

Total capacity: 120,034,123,776 bytes = 112 GB Bytes per sector: 512 Sector count: 234,441,648

So it all seems to work OK. It's just the close step seems to take longer than I would expect. Maybe its my virtual machine etc.

joachimmetz commented 3 years ago

In both cases, memory starts at around 30Mb usage, and climbs to about 400Mb, and then returns to 30Mb after close.

libewf needs to build up lookup table of all chunks, libewf legacy (v2) keeps a full table in memory, where as libewf experimental (v3) should throttle this, at the cost of re-reads or EWF metadata.

With this context, several minutes sounds long, how does this compare to the libewf tools? Do you have any indication what is going on e.g. using a debugger or profiler?

tedsmith commented 3 years ago

I'll try and get some further testing done on real machines instead of virtual ones. I am not too worried, but I wanted to check the behaviour was not extremely out of character. By the sounds of it, there's a lot that goes on resource wise, so its perhaps not unexpected to take a while in a not so hot VM. Like everything, the proof will come when the users start using it. Thanks for taking the time to provide further context for me.