search

earshinov / vscode-filter-lines

Extension for Visual Studio Code allowing to quickly find all lines matching a string or regular expression
https://marketplace.visualstudio.com/items?itemName=earshinov.filter-lines
MIT License
13 stars 3 forks source link

Practical Limit to the number of Filterable Lines? #29

Closed larray closed 11 months ago

larray commented 1 year ago

Is there a practical limit to the number of lines that the tool should be able to filter? I've expanded a log to roughly 700k lines, but I cannot get "Filter Lines: Include Lines with Regex and Context" or "Filter Lines: Include Lines with Regex" to engage. Likely not a fault of the tool - just looking for approximate "don't try to filter over 500K lines" etc.

earshinov commented 1 year ago

@larray , sorry, are you saying that the extension doesn't work? Or it works when it shouldn't?

I didn't analyze extension's performance, but it should be linear of the number of lines in the document. So even 700k should be fine, although you might need to give the extension some time.

larray commented 1 year ago

Hello and thank you!
This was not a result of the changes for 1.1.0. I noted it on 1.0.0 as well.

At 700,000 lines I cannot get the extension to "engage" when I CTRL-P > Filter LInes

I have not cut down the log size to find out where the number of lines or size of the log allows for Filter Lines to work (yet). (I will do so) I wanted to raise the experience and see if it was expected that after 250k or 500lk lines it was too complex to work on. Your confirmation is that it should work on 700k.

It isn't a matter that it takes a long time to process (that I could understand) but that the interface doesn't present after I CTRL-P > Filter Lines ..... It never brings up the dialog to enter the search term. I will try to find the upper limit where it works and report back.

earshinov commented 1 year ago

I don't have a repro. Tried "Filter Lines: Include Lines With String..." on this file with 700k lines: 700k.txt.

image

I must admit, on my machine it takes a few seconds for the prompt to appear, but this amount of time is taken by VS Code, not by the extension (which by the time the prompt appears doesn't really do anything).

larray commented 11 months ago

Sincerest of apologies - I saw a response to this, fully intended to test further and was swamped by a tsunami :)

I did identify that indeed filter-lines works on large data sets... just not on my Mac. I took it to a PC and tested there and the prompt popped up just like expected. I have not yet isolated if it is interference from another plugin in VSCode, memory utilization differences, platform differences, or something draconian like AV interfering. For now I did want to report back that I never should have cast aspersions on this tool that 1) I dearly love, 2) has saved me HOURS of dissection.

Thanks again for looking into it and this tool's development!

earshinov commented 11 months ago

@larray , you are welcome! I am going to close the ticket. Feel free to reopen if needed.