[question] Un-ignore flag?

[wavexx]

Coming from ripgrep, I'm wondering if there is a way to quickly reset the "ignore" rules with a command flag? (same as ripgrep's -u flag, for example).

I'm using a couple of "ignore-files" rules in my .ugrep config, which are helpful to exclude non-interesting things in a project. Occasionally though I want to search for all files and drop any previous ignore rules, but still preserve my other options in .ugrep.

ripgrep offers the "-u/--unignore" flag, which when used once resets the ignore list, repeated also searches for hidden files, exactly for this sort of usage scenario.

Is there something similar in ugrep?

[genivia-inc]

Thanks for asking. You can undo the --ignore-files options with --no-ignore-files, when specified after.

Most long options permit a --no- to disable.

More suggestions:

• just use the ugrep --pretty command that doesn't read a .ugrep config file, instead of trying to undo (many) options that are set in a .ugrep file
• If you've specified ignore-files in .ugrep config file and want to override it in specific directories to search from, then ug --no-ignore-files --save-config will save a .ugrep locally without the ignore-files. This local .ugrep will be used when executing ug from that directory as the PWD.
• Or perhaps create an alias to add to your .bashrc to run ug with --no-ignore-files, like alias ugall 'ug --no-ignore-files'.

[genivia-inc]

I've added a sentence "Long options may start with '--no-' to disable, when applicable" when executing ug --help OPTION as a reminder.

I agree that typing --no-ignore-files is a bit long. By comparison, enabling searching hidden files is just -.. Perhaps there is room for a short non-letter option like -: to override all .ugrep specified-options all at once. If there is sufficient enthusiasm for this then I'll be happy to add it. It will help in cases where you don't want to figure out what was set in a .ugrep file.

What do you think?

Or perhaps add -: or -@ as a short option for --no-ignore-files. For example, ug -.: will search hidden and ignored files. I don't like the rg use of -uuu as a level, because it does not offer any choice, whereas with -. and -: you have a choice what you want. Binary files are searched by default, like GNU grep. This can be turned off with ignore-binary in a .ugrep file. To search binary files, use -W or -X which shows results in hex dumps or use --no-ignore-binary. Lots of flexibility here.

I think it is a good idea to also add --no-config to not load the default .ugrep configuration file.

[wavexx]

The extra clarification in the manual is nice.

What was surprising to me, is that the --include/exclude doesn't seem to affect --ignore-files, maybe that should be clarified as well. I kind of expected patterns/globs to have a single map internally.

What I like about rg' "-u" option is that very often when used with --ignore-files (and other exclude rules), you're deliberating ignoring sizable chunks of a project because there's usually stuff you don't want to see most of the time. But sometimes, you search for something and you expect results and you suspect you're hiding it due to some exclusion, which you don't know. In some project it might be a hidden configuration file (.github to name one which would require -.), some generated makefile (via .gitignore..), maybe an exclude glob in .ugrep. So which one do I toggle to be sure I'm not ignoring anything? rg repeating -u progressively unignoring stuff does answer this pretty well. Which stuff is unignored first is debatable (depends on what you're doing), but I know that after 3 I'm searching everything.

I'm mentioning this because I was using the ack-grep/silversearcher before, and due to this I frequently resorted to plain "grep" when I was in doubt some of my exclusions could ignore stuff I was expecting.

It's also true I could just add a shell alias to unexclude all options, but appending -u to the current line is very ergonomic..

[genivia-inc]

I exactly understand that when we are in doubt we want to search all files, for example after nothing is returned as a match, but we expected something nonetheless or we want to make sure nothing matched. Searching all files will give this assurance that nothing was found.

How about:

    --all-files, -@
            Search all files except hidden: cancel previous file and directory
            search restrictions and cancel --ignore-binary and --ignore-files
            when specified.  Restrictions specified after this option, i.e. to
            the right, are still applied.  For example, -@I searches all
            non-binary files and -@. searches all files including hidden files.
            Note that hidden files and directories are never searched, unless
            option -. or --hidden is specified.

This new option offers flexibility. To really search everything, use -@. (--all-files --hidden). If you want to search all files except binary files and hidden files, then -@I (-@ -I) will do the trick, because we ignore binary files again after -@. Likewise, -@I. searches all non-binary files, including hidden files. Writing this up makes it sound complicated, but it's only one extra option -@ to remember that cancels restrictions. It's more flexible than -u, -uu, -uuu which reveal nothing about what's actually unrestricted.

I deliberately picked @ ("at") which is hopefully easier to remember as a short option associated with --all-files.

Because we have a short option -@, we can also include a new ALT-@ command in the TUI to search all not hidden files. This cancels restrictions until ALT-@ is pressed again the switch back to restricted searching.

[wavexx]

I personally like how this would work. Maybe we want to see if somebody else has some extra opinion on the behavior?

[genivia-inc]

Ping @leahneukirchen @pikumar @stephentalley if any of you is interested in following this discussion and perhaps comment on a new -@ (--all-files) option as described above.

[leahneukirchen]

Sounds useful this way!

[genivia-inc]

About -@: as much as I would like to use a single letter short option for this, there are no single letters unassigned that we can use.

Also, in my opinion, using a special character like -@ should be OK because it suggest this is a special option i.e. with behavior that is a bit different than the usual way options work. In this case with -@, all file selections before ("to the left" on the command line) are ignored, while all file selections after ("to the right" on the command line) are respected. This is not typical for (short) options.

[wavexx]

In this case with -@, all file selections before ("to the left" on the command line) are ignored, while all file selections after ("to the right" on the command line) are respected. This is not typical for (short) options.

I would consider this typical? I didn't really see this difference between short/long options "in the wild".

Generally the rightmost argument always overrides a previous one (incidentally, this is what allows shell aliases to work ok, as you can override previous behavior).

[genivia-inc]

Generally the rightmost argument always overrides a previous one (incidentally, this is what allows shell aliases to work ok, as you can override previous behavior).

True, but surely not all options behave that way. Most options are orthogonal, complimentary, like -n, -b, and -k, -o. Some options like -G, -F, -E and -P are conflicting in certain combinations (and -q, -l, (ell) -c are too, but stay silent). Some options can be repeated, "pile on" as it were, like --include and --exclude. And some options switch modes, like -I, -a, -W and -X that change --binary-files=TYPE so that they do not conflict and can change the way binary files are processed in this case. This latter form overrides the previous mode.

The -@ option is like a kill switch for all previous options that affect file and directory selections.

[genivia-inc]

Working on finalizing version 5 to include this improvement. Version 5 has a regex engine revision to increase search speed, which takes time to verify and test.

[genivia-inc]

I will name the new option --all with short option -@. We like --all better, because it removes restrictions from searching files and directories (not just files that --all-files perhaps would suggest).

The new option will be available with version 5 to be released very soon.

[genivia-inc]

Implemented.