What does the warning on `--append` about "not sorted" mean? Does append actually work?

[jberryman]

I don't know anything about the internals of ctags format. Why should I care that the tags file is no longer sorted? And if it is necessary that it be sorted then --append needs to be fixed, right?

So this is either a documentation or implementation bug report.

[jberryman]

Reading the exuberant ctags manual it looks like tags being sorted was a legacy requirement of the format, and _TAG_FILE_SORTED can be used to indicate to programs consuming tags whether the file is sorted (so that they can do a binary search through the file, say).

From vim manual

                                                        tag-binary-search
Vim uses binary searching in the tags file to find the desired tag quickly
(when enabled at compile time +tag_binary).  But this only works if the
tags file was sorted on ASCII byte value.  Therefore, if no match was found,
another try is done with a linear search.  If you only want the linear search,
reset the 'tagbsearch' option.  Or better: Sort the tags file!

Note that the binary searching is disabled when not looking for a tag with a
specific name.  This happens when ignoring case and when a regular expression
is used that doesn't start with a fixed string.  Tag searching can be a lot
slower then.  The former can be avoided by case-fold sorting the tags file.
See 'tagbsearch' for details.

It's not clear whether vim observes TAG_FILE_SORTED or just assumes sorted and falls back to linear search as described above.

I'm also not sure what e.g. vim-tagbar or other clients do...

So:

• --append should at the very least set _TAG_FILE_SORTED=0
• --append doesn't currently do anything reasonable, at least with respect to vim; a quick hack might be to prepend the new symbols to the file (and mark as unsorted), but this would cause tag files to grow very large
• doing this right by parsing and replacing entries and resorting is probably the way to go

[jhenahan]

I'm looking into this and #68 over the weekend, so I'll see if I can make some sense of this. I've never even used --append (I just regenerate tags), so I've never really given it a look. Even the ctags FAQ seem to suggest that it's gross. I'll think on this.

[jberryman]

Yeah I'm sort of giving up on this. I'm mostly interested in this in order to use gutentags, but they actually delete tag files from a file that changes before doing --append, apparently because the operation is flaky in some ctags. So I guess the end result there should be that the file is no longer sorted, and (hopefully) vim will find the tag via linear search fallback. But at least there won't be duplicates (meaning tags will randomly be wrong depending on where they fall in the alphabet).

[jhenahan]

Cool. I won't burn too many cycles on matching weird --append edge cases, then, but I'll at least try to get a grasp on what it's doing and make sure it's sensible. I'll go play with ctags and see if it warns about ordering when appending. If it doesn't squawk about it, there's probably no reason we need to, so maybe we can drop the warning altogether.

[jberryman]

Actually now that I'm testing, vim doesn't seem to find tags that are just appended (and out of alphabetical order), again this is with gutentags so maybe that's causing some of the issue. It seems to have erased the meta line about sorting, although I'm not sure why that would make a difference.

Cutting and pasting to re-alphabetize the tags file manually caused jumping to work.