Timeout on search

[yajo]

Today I opened a DB dump (~16MiB) and I gladly saw Enki handling it with syntax highlihgt at a reasonable speed and RAM.

I had to search for the word "TGSS", so I pressed Ctrl+F and typed that.

When I typed the "T", Enki hanged for a while searching for all "T" in the document. Following letters gave a similar result (quicker each time I typed more).

Maybe if you set a timeout when searching (like 0.5s or so), Enki would start searching after I typed all characters, and thus it would not get stuck. Another solution could be stopping the search when I type something new in the search box instead of waiting for the results to start again.

I know Enki is no DB dump viewer, but responsiveness is always a Good Thing™.

[andreikop]

Can you give me your file? Which CPU do you have?

I tested 32 Mb 1M LOC C++ file - Enki refuses to highlight it (to do not consume whole system memory) but search takes less than a second on my not the most powerful i3.

[andreikop]

How long was the freeze? If not too long, I'd like to leave as is. Because delay big enough to do not expire between key pressings (i.e. 0.5 sec) will be noticeable by eye even when editing small files (99.9 % of cases).

[yajo]

Can you give me your file?

I'm sorry but not, it's a DB dump with sensitive data.

Which CPU do you have?

Intel® Pentium(R) CPU G645 @ 2.90GHz × 2

I tested 32 Mb 1M LOC C++ file - Enki refuses to highlight it (to do not consume whole system memory) but search takes less than a second on my not the most powerful i3.

My file:

$ wc --lines file.sql 
2083 file.sql
$ wc --words file.sql 
851272 file.sql
$ wc --chars file.sql 
16019317 file.sql

As you can see, there were not too much LOC, but yes many words, because MySQL dumps make looooong lines.

How long was the freeze?

~ 15 sec. For the first letter, ~3 sec. for the second, and almost nothing for the rest.

If not too long, I'd like to leave as is. Because delay big enough to do not expire between key pressings (i.e. 0.5 sec) will be noticeable by eye even when editing small files (99.9 % of cases).

Well instead of a delay I think the point is that the process should be asynchronous and killable.

Imagine: I write "T", then the searching process spawns and the word "Searching" appears on the top. When the process ends, the results display. If in the meanwhile I write the second letter "G", then the previous process gets killed and a new one is spawned, and so on.

This way the current UX in small files would be instant too (spawning and waiting to end should not take more than a couple of milliseconds more, probably you would not even have time to see the word "Searching" in small files) and with big files the UI is not freezed and no resources are wasted in searching something that is not what the user actually needs.

[andreikop]

I tried to generate a file similar to yours. 39 MB in 5K LOC, but having less than a second search time. If you are still having your dump, could you please checkout searchtest enki branch, try to search something and paste result. This branch contains a debug print. Do sudo pip2 install contexttimer to use the branch.

[yajo]

This is the output:

libpng warning: iCCP: Not recognizing known sRGB profile that has been edited
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited
get text time 1.00135803223e-05 search time 0.282513856888 found 196941 items
get text time 6.91413879395e-06 search time 0.212423086166 found 108 items
get text time 5.00679016113e-06 search time 0.198891878128 found 3 items
get text time 5.96046447754e-06 search time 0.198921918869 found 3 items

[andreikop]

Hmm. I see only 300 ms lags here. This is what I precept when searching in my big file. Probably the long lag happens not on search, but on some other action (highlighting results, moving cursor :-/)

[yajo]

I just did another test. I did not move the cursor. Also, I used a chronometer and it was about 10s for the first letter, and almost nothing for the rest. Maybe you're right and it's in the highlighting process...

[andreikop]

Hmm. If the first time is so slow, maybe you could put such a timer here. Qutepart caches text as a Python object, but the first access could be slow.

[andreikop]

The first freeze is likely to be caused by long conversion from Qt internal representation to huge Python text object. In this case I can't fix this problem with reasonable effort.

[yajo]

OK, I guess this is a corner case anyway.