Redesign search dialog

[pkulchenko]

The search dialog is currently shown as a popup window, but it would be useful to (1) make it dockable, (2) make it more compact, and (3) add support for incremental search.

Other search-related tickets: #398, #146, #162, #140, #132, #127, and #147.

[pkulchenko]

The updated dialog window is always docked. I initially tried to make it floating and dockable, but there is an issue on OSX and Linux with getting focus to components on floating windows that have wxFRAME_TOOL_WINDOW style, which is exactly the style that wxAuiManager is using for its floating windows (and it's not configurable).

Another wxwidgets issue made it difficult to use wxComboBox components as they don't handle key events properly on OSX, which makes Tab and Escape key handling difficult.

So far I've kept the current functionality, minus the ability to select from comboboxes, which I plan to compensate with suggestions from most-recently-used list in the text field.

The added functionality:

1. Incremental search for Find and Find and Replace. The text is searched as you type
2. Esc is used to close the form; the cursor is left at the current position. If you want to restore the cursor at the original position (when the form was opened), use Shift-Esc.
3. Enter in Find field does search, Enter in the Replace field, replaces one instance.
4. The status bar was moved to the search toolbar, which makes the message more visible and convenient there.

[pkulchenko]

@sladecraven, @kspes, @andrewstarks, @Fringale, @stuta, @easy82, @naturally, @pixeljetstream: I've been working on a complete search redesign and would welcome your feedback. All of you provided search related comments and suggestions in the past and I'd be interested in how the new search works for you.

There were several goals for the redesign: 1) make the UI more similar in style to the rest of the application, which has changed significantly while the search/replace dialog stayed the same, 2) preserve the existing functionality and extend/improve it in several ways, 3) make the dialog simpler and more compact, 4) add incremental search, 5) show find-in-files results in an editor tab, and 6) add preview for replace-in-files.

The last two items are still being worked on and should be available in few days, but other items are done and have been pushed to search-redesign branch. If you use ZBS cloned repo, you can do git fetch origin; get checkout search-redesign. If you prefer to not use git, you can get the latest search-redesign code in a .zip archive.

I also plan to add search/replace in open files and search/replace in selection, but these two will be done a bit later, after results-in-editor-tab is implemented.

I combined directory and mask fields into one field and you can specify one directory and multiple masks (separated by semicolon or comma). The directory is optional as the current project will be used if not specified, but will be suggested by default to show how it can be used.

The only thing that got "lost" in the redesign are the dropdowns as they have several (wxwidgets) issues/limitations associated with them (see my earlier comment in this ticket for details). I've tried to compensate in the scope field with the toolbar dropdown, but search/replace fields don't have anything to access earlier searches. I'm considering several options:

1) do nothing. 2) provide autocomplete that shows matches based on most-recently-used history. If you previously searched for foo and type f, then oo will be added and selected, so you can type over it. There are couple of issues with this as some users may not like matching (so it needs to be configurable) and it may interfere with incremental search (it will try to find foo even though you may have something else in mind). 3) use up/down arrow keys to access the history of previous search strings. Not too difficult to implement, but may be difficult to discover, so it's likely to be rarely used.

I've been using option 2 in other products and it works quite well, but I'm open to other suggestions.

Give it a try when you get a chance and let me know your thoughts. Thanks!

[kspes]

Hi Paul,

Finnaly found some time to evaluate the new branch, here are my thoughts and comments:

1) Love it! It's now way more practical to use! 2) Still get ocasional mini freezes while searching but nothing major. especially when searching for a common word in a large projects (such as "function") 3) If you quit the app while search is still active you get a lua crash 4) the filter is not remembered properly after re-launching the app. I set it to serach for .lua;.scene;.dlg, and it always defaults back to .lua for me, regardless of whether I separate it with commas or semicolons.

5) Speaking of slowdowns, the "Go to file" dialog is horribly slow for me, down to the point I can't use it. and I used it on a powerful PC as well as a laptop. I figure it's because it's searching a lot of binary file I have in my source tree, any ways to optimize the search?

Krešo

[pkulchenko]

1) Love it! It's now way more practical to use!

@kspes, thank you for the feedback; good to hear it's working well for you! I'm almost done with showing results in a separate editor tab, which should be much more convenient as well.

2) Still get ocasional mini freezes while searching but nothing major. especially when searching for a common word in a large projects (such as "function")

I'll see what can be improved, but if it's a general search (not a binary file), there is probably not much I can do without drastically changing how the search is done.

3) If you quit the app while search is still active you get a lua crash

Is it during find-in-files or the current file search? I'll check...

4) the filter is not remembered properly after re-launching the app. I set it to serach for .lua;.scene;.dlg, and it always defaults back to .lua for me, regardless of whether I separate it with commas or semicolons.

Yes, this information is not saved between sessions, but will be. I'm not sure how to show the custom filters as they are going to be in the history, but would you expect to show your filter or the default one? I can probably find the most-recently-used scope based on the the same project path and apply it (instead of the default value). Is that what you are looking for?

5) Speaking of slowdowns, the "Go to file" dialog is horribly slow for me, down to the point I can't use it. and I used it on a powerful PC as well as a laptop. I figure it's because it's searching a lot of binary file I have in my source tree, any ways to optimize the search?

Yes, I'm thinking about several improvements. (1) Read and check only the first 2k, (2) Keep track of extensions for binary files; if I find two .exe files that are binary, it probably doesn't make sense to read any exe files, so I can just filter them out (same for binary files without extensions or images and so on); and (3) Add configuration for folder/file exclusion. This would apply to everything (including showing files in the project tree and any searches); this will speed up search and go to file navigation as those files will be simply skipped.

Let me know what you think may work better for your case...

[kspes]

HI Paul,

remembering the last filter used for each project would do fine in my configuration. But I really think you're going to have to use a project/workspace file sooner or later, to store either project specific or user specific values.

as for Go to Files, I like those ideas! It's important to be able to define what a binary file is if you can't autodetect it. for example I have a proprietary image format I use than is binary, which probably won't be included if you define a list of file extensions that are binary.

also, I know some devs that use .dat files to store tex configuration info, while some use it for binary data.

Krešo

[pkulchenko]

also, I know some devs that use .dat files to store tex configuration info, while some use it for binary data.

Right, what I have in mind should still work fine for that because it will be remember for each session (I'll keep track of extensions for binary files and if the same extension is detected twice, I'll skip it in the search).

BTW, I can't reproduce the crash with quitting while searching. Is it during find-in-files? Do you simply close the application while the search is still happening?

[kspes]

yeah, I do a cmd+q (mac) while searching and it crashes.

here's a screenshot: https://www.dropbox.com/s/csdrb333yrycqfc/Screenshot%202015-03-22%2022.21.28.png?dl=0

Krešo

[pkulchenko]

Thanks; this is running the latest commit (24068634) in the search-redesign branch, right?

[pkulchenko]

@kspes, I pushed a fix that may fix this the error you are getting, but I still can't reproduce it, so would be great if you could give it a try (it may also return an error in a different place).

Also, this last batch of changes adds showing results in an editor tab and I'm interested in feedback on it. I'm working on adding indicators to the matched fragments to improve their visibility, but even in the current form it should be better than the results in the Output windows.

In terms of the replace-in-files, I'm thinking about showing the changes in a similar editor tab with the replacements already applied, so you can "save" the tab, which will apply those changes to all the files. This gives you a way to review the changes all in one place before applying them and even have "undo" if you "undo" the changes in the tab and save them again. I'm still figuring out the details, but it seems much better to me than the current "blind" replacements.

[pkulchenko]

Speaking of slowdowns, the "Go to file" dialog is horribly slow for me, down to the point I can't use it. and I used it on a powerful PC as well as a laptop. I figure it's because it's searching a lot of binary file I have in my source tree, any ways to optimize the search?

@kspes, curious, how large is the source tree in this case? Are we talking about thousands or tens of thousands files? Is the slowdown before you type the first character or while you are typing (trying to figure out if it's slow to read all the filenames or to match them as you type).

Also, I might have fixed the issue with quitting while searching, but would like you to confirm. Thanks.

[pkulchenko]

@kspes, one more question on your source tree. For the binary files only first 2k should be read, but if there are many of them, they are still read every time during the search. I'm working on changes to that logic (in Go To File) and any additional information you may provide (the overall number of files, how many of those are binary files, with what extensions) will be helpful in testing on a real use case. Thanks.

[kspes]

Hi Paul,

I'm in a deadline right now so can't give you a detailed answer, but in short, I've got tons of files in the source tree, source code intertwined with various png, jpg images and other binary data and non-source code text files. My average svn working copy in the company is about 3-5 gigabytes with thousands of files in the source tree.

---

Krešo

On Thu, Mar 26, 2015 at 7:03 PM, Paul Kulchenko notifications@github.com wrote:

@kspes https://github.com/kspes, one more question on your source tree. For the binary files only first 2k should be read, but if there are many of them, they are still read every time during the search. I'm working on changes to that logic (in Go To File) and any additional information you may provide (the overall number of files, how many of those are binary files, with what extensions) will be helpful in testing on a real use case. Thanks.

— Reply to this email directly or view it on GitHub https://github.com/pkulchenko/ZeroBraneStudio/issues/416#issuecomment-86648883 .

[LeRatierBretonnien]

Tested a few minutes ago, perfect !

[pkulchenko]

Thanks, @sladecraven. There are some more changes in the works, but it's already close to the desired state ;).

[pkulchenko]

@sladecraven, pushed a number of updates to the search functionality. Settings are now saved between restarts and auto-complete is working in find and replace fields.

4) the filter is not remembered properly after re-launching the app. I set it to serach for .lua;.scene;.dlg, and it always defaults back to .lua for me, regardless of whether I separate it with commas or semicolons.

@kspes, I made several changes to this logic, so when you switch/load the project, the scope should be set based on the last used scope for that project ("project/dir; *.ext"); it will match based on "project/dir" value, so it should work as expected in most cases.

[zorfmorf]

I just tested tested the search&replace functionality a little bit more throughly on the most recent version (checked out ~1h before this post). It feels very minimalistic (in a positive way) and fast and is a definite improvement!

Some small potential improvements:

• Toggling "Show context" should update the search results immediately
• When searching, search results are editable and saveable but this has no effect on the actual files. Maybe disable editing of search results?

I found a small issue with replacing where focus was not automatically shifted to the search results file and therefore replace was only executed in the file that was active before the search. Reproducable 100% the following way:

• Open zbstudio with a project and at least one file open
• Open the "replace in files" dialog via the menu
• click in the search field and enter a search value (in my case "update")
• click in the replace field and enter a replace value (in my case "testst")
• click the "find" button
• click the "replace button"

In my case I get 48 hits, but only the two hits in the opened file were actually replaced. It seems as if the file I was in before searching still has focus although zbstudio displays search results as opened. If I click once into the search results file before clicking on replace everything gets replaced as expected. Just a minor issue but it was confusing for a short time.

[pkulchenko]

@zorfmorf, thank you for the feedback!

Toggling "Show context" should update the search results immediately

Yes, I've been considering this after discussions on the maillist, but there is one complication: when the content of the search results is folded (which is convenient to see the list of files) and then unfolded, all the hidden context lines become visible, so I need to come up with some way to hide them back after unfolding. I have it on my list, but none of the solutions I've considered so far satisfy me.

Also, since I don't know if a user would want to look at the context, this would require always adding context lines to the results, which increases the number of lines there by 4x-5x.

Given these two things, I'm leaning to keeping it as is. If you are interested in making match lines to be more prominent in the search results, I can probably add a special marker for those, so the users can configure their background to stand out.

When searching, search results are editable and saveable but this has no effect on the actual files. Maybe disable editing of search results?

I can do that, but we also discussed "live editing" capability that would allow you to edit the search results and save them to reflect changes in the files. This would be very similar to how Replace-in-Files work, but Replace also does the replacement for you, so you can simply save the results.

This would require adding a new icon for Find-in-Files to toggle "live editing", but I agree, that when it's off, it may be better to keep the editor read-only.

It seems as if the file I was in before searching still has focus although zbstudio displays search results as opened.

Yes, the IDE keeps track of the last tab used to know where to do the search (as you can also search in Output and Console windows), but this is a bug, which I plan to fix shortly. There are some inconsistencies between results when buttons or Enter in find/replace fields are used and those should be fixed.

[pkulchenko]

search-redesign branch has been merged into master and will no longer be updated.

@zorfmorf, I fixed the last issue you noted and added read-only status to search results.

I also tried adding "live editing" for find-in-files results with a new icon, but it didn't look good in terms of the workflow, so I'm not sure I want to add it. Besides, you can always do replace-in-files with the same value to get the desired effect. I have the diff file if someone is interested to give it a try (or can push the changes to a separate branch).

Please report any issues you notice (even if it's a small stuff, like focus not going where it needs to be), especially on OSX/Linux as there were some wxwidgets incompatibilities between platforms and it took some effort to make it work as expected on Linux.

[pkulchenko]

From #444:

when I "search in all files", the result forms a new tab "Search Result" in main editing window... is there any configuration that I can go back to the original behavior (just showing the results in the output window)?

@joe528, No, there is not at the moment and having the results go to the Output window would conflict with how the replacement is done with a preview. We've discussed having an option to split the editor tabs vertically to show the results, but it hasn't been implemented yet. Sending the results to the Output window requires putting the file name on every line, which pushes the content to the right and makes it start in different positions. It also doesn't allow for showing the context. After you searched in files, you can use Find Next and Find Previous to quickly jump between matches without even leaving the search panel and doubleclick will open the file as before.

Even for simple Find in current file, there is a new Find window appears under the main editing window, and I don't know how to use it (for example, to find the next instance)...

It's the same as it was previously: you can use Enter, you can use the Find button, or you can use F3 or Shift-F3 to navigate the instances. In fact, it's a bit better as it also provides incremental search and doesn't cover the results. Some users prefer to keep it open as it doesn't take much space, which wasn't feasible with the original design.

I really like the original behavior because I really don't like new window appears in the main window. Please do let me know how to turn this new behavior off. And if there is no such option, I would strongly suggest to have one.

I originally intended for the window to be switchable between docked and floating, so users can select what they prefer, but there is a wxwidgets issue with the focus not getting to those windows when they are floating (check the earlier notes in this ticket), so it's only docked for now. Also, when the window is floating, it's problematic to implement an incremental search and the shortcuts that are working now (like Find Next and Find Previous) are not going to work.

The current state is the result of multiple trade-offs and various attempts to make it work in the most consistent and pleasant way. It also seems to integrate well visually with the rest of the application.

[joe528]

(1) I really like the original docking design. I have a 27" monitor and don't open ZBS in full screen, so I still have plenty of room for the docking search box. I don't know why it is better to make it so smaller in the main window. I could easily operate the search functions with the originally bigger dialog but now it is in a small window. The feeling is total different and the operation becomes different & difficult.

(2) For the search-all-files results shown in a new tab, it becomes very difficult for me to see all the results easily especially my main editor font is bigger. Besides, if it's in a new tab, whenever I click a result to see some code, the search result will be in the background, and I have to go back to the search result tab repeatedly. I cannot see the whole picture easily and this totally ruins my workflow.

I probably can live with (1) but I just can't live with (2). I may need to stick with v1.0 if this new design cannot be an option.

[pkulchenko]

I don't know why it is better to make it so smaller in the main window. I could easily operate the search functions with the originally bigger dialog but now it is in a small window.

@joe528, it's good feedback and I'm open to ideas on how to improve it further, however I don't plan to provide an option to use the old "floating" design. You can still search if you hit Enter in the Find field, and you can also replace once if you hit Enter in the Replace field. In fact, the font in the current search fields is larger than it used to be.

(2) For the search-all-files results shown in a new tab, it becomes very difficult for me to see all the results easily especially my main editor font is bigger.

Would you prefer to be able to set the font separately for the search tab?

Besides, if it's in a new tab, whenever I click a result to see some code, the search result will be in the background, and I have to go back to the search result tab repeatedly. I cannot see the whole picture easily and this totally ruins my workflow.

I plan to mitigate this in two ways: there will be an option to have horizontal or vertical split for the tab with search results. In addition to that, F4/Shift-F4 will jump to the next/previous result (and open the file if needed).

I probably can live with (1) but I just can't live with (2). I may need to stick with v1.0 if this new design cannot be an option.

I realize that some changes may cause strong feelings one way or the other, especially for those components that are perceived as working fine, but Search has been ripe for some improvements. As I mentioned, some of the issues caused me to make trade-offs that I didn't plan originally, but I expect the overall balance of the changes to be for the better.

[joe528]

@pkulchenko For search results in a new tab, I have tried it several times and it's just not right for me. I am not sure if it will be ok for me by using a separate font option & split window when the original design works perfectly for me and I don't like the new output (the original has only one line for each find and it gives me a way to look through things quickly and easily). I don't use advanced features in ZBS. I just edit, save, and search. The current build can totally satisfy me. Therefore, I guess it will be fine for me to stick to v1.0. It will be great if you can build a new build v1.x with important bug fixings without these new features so I can stay there with a decent build.

[pkulchenko]

@dbaileychess, @joe528, I've reconsidered my position on showing searched results at the bottom of the screen (next to the Output/Console tabs) and took another look at what it may require. The pushed changes include search.showaseditor option, that will make the search results show at the bottom when set to false; the default value is true.

I also added search.zoom, which allows to make the search results to appear smaller/larger than the current default size. For example, setting it to -2 will make them a bit smaller, but still readable enough to fit more content. The default value is 0.

It will be great if you can build a new build v1.x with important bug fixings without these new features so I can stay there with a decent build.

Joe, this is not likely to happen as it requires significant work and is not sustainable long-term. This is one of the reasons why I'm trying to collect as much comments as possible during the development (even though I may not act on them right away), as it allows me to make necessary adjustment and still maintain one main branch.

Let me know how these changes work for you and thanks again for all the feedback and suggestions.

[joe528]

I have tried it and since the search result is now in the bottom after I set the option to true. It's much better and easier for me to switch to different files based on the search result.

But I am still trying to get used to the new output format that includes some code that does not contain the keyword. This simply makes the output much longer and unorganised. Can this be an option too?

One more question, after I click cmd-F or cmd-shift-F, the search window appears in the bottom of the main editor. After I finish with the search, how to make the search window disappear? I don't like it to occupy the bottom of my editing window even though it's just a line. For example, it should just go away automatically when I click the main editor window.

[madmaxoft]

The search window goes away when you press Esc

[pkulchenko]

But I am still trying to get used to the new output format that includes some code that does not contain the keyword. This simply makes the output much longer and unorganised. Can this be an option too?

@joe528, yes, you can uncheck Show Context icon on the search toolbar and the context lines won't be shown. This setting will be saved between restarts.

One more question, after I click cmd-F or cmd-shift-F, the search window appears in the bottom of the main editor. After I finish with the search, how to make the search window disappear?

Yes, as @madmaxoft wrote, you can press Esc to dismiss the dialog (when the focus is on one of the search panel fields).

it should just go away automatically when I click the main editor window.

It's a matter of taste/preference. I got reports from some users that they keep the dialog "on" all the time as it doesn't take up much space.

[zorfmorf]

Just wanted to add that I really like the redesigned search dialog!

[pkulchenko]

Thanks @zorfmorf!

[dbaileychess]

Hi Paul,

Just tried these options and I like them. Allows me to compare the search results (with context) with my current editor, which is important to me. The zoom is great too.

Side notes

• Is there a way apply the update in the user.lua configuration without restarting ZBS?
• Could you name the System configure ('system.lua') and the User configure ('user.lua'). Seems odd they are named the same.

Take Care,

Derek

[pkulchenko]

Just tried these options and I like them. Allows me to compare the search results (with context) with my current editor, which is important to me. The zoom is great too.

Derek, sounds good; thank you for the feedback.

• Is there a way apply the update in the user.lua configuration without restarting ZBS?

No, but it's likely coming. The main issue is that some configuration changes are indeed difficult to re-apply after the IDE is already started. For example, editor settings and color themes can be applied at any time, but toolbar configuration, menu items, language, and some other things can't be easily re-applied.

I've been thinking about separating the settings into two buckets: those that can be re-applied are applied immediately when the changes are detected and those that can't be, trigger a message given an option to restart or to apply them on the next restart.

If you are thinking about applying this to search.showaseditor, then it's a good example of why it's not a straightforward thing to do. If you already have the setting as true and the search results tab is already displayed, when the setting is changed, the tab needs to be closed, so that it can be open next time in the different location. This logic is not present and it needs to be added for many other settings in a similar way.

• Could you name the System configure ('system.lua') and the User configure ('user.lua'). Seems odd they are named the same.

They are called the same on purpose (as I wanted to emphasize that it's the same configuration file, just loaded from a different location). I inherited that name from the editor that the IDE started from and if I were naming the config file today, I would pick something like settings.lua, but it's too late to change as there are too many places in notes and documentation where it's referenced.

I keep a list of would-be-nice-to-have-it-named-differently items and am waiting for the right opportunity.

[joe528]

@pkulchenko Yes, it is a preference about the search window auto-disappear. I really hope it can disappear automatically. Usually we shift between search and search in files, so it's a habit to press cmd-F or cmd-shift-F again when search is needed. There is no use for the search window to be there idling and I don't like it there to occupy the main editor window even if it's small and above all, it's a distraction.

[pkulchenko]

@joe528, I thought about it initially, but it wasn't clear when exactly to hide the window. I can auto-hide it when you select an editor, but it's possible to use the window without even giving it focus, which will hide it at the wrong time. This was one of the reasons it's designed this way now.

If you mean hiding the window when the find-in-files or replace-in-files is done, that's more straightforward.

[madmaxoft]

Any chance we could get keyboard shortcuts for the Replace dialog? In order to replace all occurrences, I have to type the text to replace and then use the mouse to click the button, I'd much rather press a shortcut (most IDEs use Alt+A for that particular action).

[pkulchenko]

Any chance we could get keyboard shortcuts for the Replace dialog?

Yes, I do plan several extensions for the shortcuts. For example, Shift-Enter to search/replace back, Alt-F3 to select all instances, Alt-Enter to find/replace all instances in a separate tab (similar to how find-in-files works, but for the current file).

I'm not sure about Alt+A, but Ctrl-Enter may be a good option. I think the question is, should it do the replacement when the focus is on any field or only when it's on the replacement field? If there is a shortcut for the search panel, I think it shouldn't matter what field the focus is on.

[madmaxoft]

Those shortcuts, do they work while the focus is in the editor? Because it would be nice to have consistent shortcuts that did work while in the editor, too.

[pkulchenko]

Those shortcuts, do they work while the focus is in the editor? Because it would be nice to have consistent shortcuts that did work while in the editor, too.

By working "in the editor", do you mean when the search dialog is already on the screen? Then, yes, it would be convenient, although it means that Ctrl-Enter is not an option.

Having a global shortcut would allow replacement to work when the search dialog is not even on the screen. I know some editors allow to use something like Ctrl-E to set 'find' text and Shift-Ctrl-E to set 'replace' text, so you can then replace all instances (probably using something like Shift-Alt-F3). This is probably too advanced for most users, but may be useful in some scenarios.

[pkulchenko]

@joe528, I added search.autohide option, which you can set to true to hide the search panel after find-in-files operations.

[pkulchenko]

@kspes, I've made several improvements to file traversal that should improve performance in command bar and file searches. I've also added exclusion lists and binary file lists (#455) that you can extent to skip the files or folders you don't need searched. Give it a try and let me know if it's any better...