Tag list and Search fails with filenames using brackets: `[` and `]`

[realshovanshah]

I'm a regular user but here's what happened today.

Background

1. I created a new note as usual but this time, for some reason, the index didn't render properly in the console output. The index in this case was 1142.

   Eg: Instead of Added: [1142] new_note.md, it just showed Added: [] new_note.md.

2. I ran nb (without the colors too) to check if everything was only but again, here's what I see.

   

3. Other operations like edit or show seems to be working fine. And the .index file also has the note on the correct index.

   

The Killer Problem

I dismissed it as some parsing or rendering issue and went about continuing what I was working on. But all of a sudden the tag listing and search commands such as nb #a, nb q <query> --tags a or even nb q <query> started misbehaving.

In fact, the same tag list command that I had tried a couple of commands earlier returned way shorter results. And even searching for words (tags) that I've check exists, isn't showing up anymore. Not sure if there's a correlation but this started after index bug.

Note: I also now notice a delay in the time that it takes for the footer to render when running nb to show up. It's apparently stuck running the cat command.

Here are some things that I've tried:

• Both reconciling and rebuilding the index.
• Create a different local notebook with all my notes.
• I also didn't find any similar issues apart from https://github.com/xwmx/nb/issues/154 and https://github.com/xwmx/nb/issues/178.

I'll try a couple of more things, but I want to make sure it isn't something related to the broken rendering I mentioned above. Does nb generate any log file? I'd be helpful to see if there's a way to see if any underlying commands that failed or maybe threw some error when parsing text.

This is a really weird one. Unfortunately, I haven't really maneuvered files/text directly since I started using nb, so I'm a bit handicapped right now; but I guess now is the chance...

[realshovanshah]

Man did I just get a "programming rush".

Looks like the problem was using [ ] in the file name; there seems to be some problem parsing (at least the.index file) when square brackets are used. Is this a known issue?

Anyway, in the meantime I can at least use a work around now. I guess I won't be learning find anytime soon.

[xwmx]

@realshovanshah Thanks for the information. This should be resolved in nb version 7.7.0. Please let me know if you still encounter any issues with this after upgrading.

[realshovanshah]

Thanks for the swift fix @xwmx. Adding notes with square brackets in the filename/title seems to be fixed now.

But there seem to be more issues when doing things like removing the notes with [ / ], or searching for texts with square bracket: nb q '['. Moreover, running nb also broke a couple of times when trying to convert my notes to use this fix.

Perhaps, it's best to do a thorough review of most features' compatibility with this character.

One more thing:

Does nb generate any log file? I'd be helpful to see if there's a way to see if any underlying commands that failed or maybe threw some error when parsing text.

[xwmx]

Thanks for the additional information.

The brackets are being interpreted as part of a regular expression pattern when passed to sed and maybe other tools and, therefore, need to be escaped with a leading backslash within the nb program code. Other regular expression metacharacters like ( and ) are apparently interpreted as normal characters and do not require escaping according to tests. Since this issue affected all cases where the id is retrieved using the file name, it could be the underlying cause of other errors.

Technically the search query behavior is intended since it’s providing regular expression search with the underlying grep-style tool. The errors I encountered in search while resolving this were related to the id retrieval.

There is no logging. Error output from underlying tools is usually hidden in nb by piping standard error to /dev/null, e.g., example-command 2>/dev/null. 2>/dev/null could be developed into a function that optionally enables the printing of error output from underlying tools.

[realshovanshah]

Can confirm that escaping works: nb "\[" and nb q "\[". Should the square brackets be behaving similar to parenthesis ( ) ?

Not sure why other operations like nb remove are still breaking tho; the commit should have fixed it no? It looks like grep has started throwing an error now tho.

grep: brackets ([ ]) not balanced

As for extracting logs rather than piping them to /dev/null, it'd no doubt be a good investment (plus the fact that there's already some demand for it from what I've seen). The users of nb are presumably more technical, so it'd also help them debug when blocking issues arise; especially those that use this tool as their daily driver.