search

sleuthkit / autopsy

Autopsy® is a digital forensics platform and graphical interface to The Sleuth Kit® and other digital forensics tools. It can be used by law enforcement, military, and corporate examiners to investigate what happened on a computer. You can even use it to recover photos from your camera's memory card.
http://www.sleuthkit.org/autopsy/
2.41k stars 597 forks source link

Table view takes a long time to sort #830

Open kefir- opened 10 years ago

kefir- commented 10 years ago

I did a file search in autopsy that returned quite a lot of files. I tried to search the results by size, but after thinking for a number of minutes the results were not sorted. I clicked the Size column again, and then the sort indicator (little arrow) appeared on the heading, but the results are still not sorted.

Here's a screenshot:

no-sorting

rcordovano commented 10 years ago

I did a file attributes search that returned 12,265 files. I was unable to reproduce this problem.

kefir- commented 10 years ago

I ran some more tests to reproduce this here. This time I ran in a fresh instance of Autopsy and I reopened the already ingested case. I searched for files with size larger than X, and then checked if sorting worked. I started timing the sorting operation at one point, as it took some time (so did the search, but I didn't time the search operation):

kefir- commented 10 years ago

I thought I'd create an index on the size column and retry after doing that, but simply running sqlite3 shell manually on the database and executing commands like this:

select count(*) from tsk_files where size > 1000;

they returned the same numbers as I got from the search results in the GUI, but they all seemed to return after less than a second. So if it's sqlite taking time, it must be in a more complex join operation, but I can't figure out a way to intercept the actual query so that I can try it manually and then attempt to optimize it. (As a side note, it'd be really nice if it were possible to enable sql query logging, and even nicer if I could enable logging of how long each query took.)

rcordovano commented 10 years ago

Interesting results, thanks. If I increase the number of results to be sorted in my tests, I expect I will be able to reproduce the problem.

However, the sorting in question is not done by SQL statements sent to the SQLite database. Instead, the data for the matching files is obtained from the database, then wrapped in NetBeans Node objects and displayed in a NetBeans OutlineView. After that, sorting the displayed data by column is done by NetBeans library code, not Autopsy code. I have found the NetBeans source in the past when I went looking for it, but the sorting in this case is essentially a black box to Autopsy.

kefir- commented 10 years ago

Strange that netbeans sorting is so slow, sorting these numbers of records ought to be pretty fast.

Just found this article on customising sorting in netbeans outlineview, sharing in case it could be useful for something.

http://netbeans.dzone.com/nbm-advanced-sorting-customization

Apparently the comparator can be overridden as well, if that's what is taking so long.

bcarrier commented 10 years ago

Thanks for the link. There are several places that we could use paging in the table results, but haven't because the sorting would be all messed up. That link may help that.