[Feature request] Hierarchical tags

[ghost]

Like a Evernote:

---

Operating system

• [x] Windows
• [x] macOS
• [x] Linux
• [x] Android
• [x] iOS

Application

• [x] Desktop
• [x] Mobile
• [x] Terminal

[darkphase]

I couldn't understand exactly what you mean with "you apparently have a solution how separate tags in a hierarchy are really separate entities and when searching for a tag, you will get all children returned" sentence.

All I'm trying to say that rendering that way would be enough... no need to introduce hierarchy between tags... Tagging methodology may model very complex graph relations which may not be modelled by any hierarchy.

If tags that include separator character as kept as whole and not as multiple entities (except search) It would be fairly easy to implement such feature. Just by rendering them as if they were trees.

I'm not familiar with the codebase of Joplin of course. Maybe I'm wrong about easy implementation. I'm just proposing a feature that already implemented in other note taking apps.

Searching is whole another issue. I don't know if we should discuss it here but I think in that particular case Tags may have broken by seperator and indexed that way.

[tessus]

Just by rendering them as if they were trees.

This is my point. Just rendering but not having the algorithms in the back will do exactly nothing.

So, please explain what you think is the solution.

All hierarchies are trees. I just don't understand what you are trying to do. I just think you might not know what this entire subject entails.

[darkphase]

This is my point. Just rendering but not having the algorithms in the back will do exactly nothing.

I'm not familiar with backend functions.. Can you tell me what algorithms you're referring to? Don't get me wrong just asking to learn.. Maybe I'm proposing (architecturally) a very improper thing here...

If you mean how I get the related notes regarding clicked item on the left most tree:

If I would attach a tag to a note something like

Books / Programming / Languages / Python

4 entries may have inserted relating that note into DB.

• Books (HID:1)
• Books / Programming (HID:2)
• Books / Programming / Languages (HID:3)
• Books / Programming / Languages / Python (HID:4)

Books (HID:1)
 |------ Programming (HID:2)
                |-------- Languages (HID:3)
                              |----------- Python (HID:4)

When I click "Books" all notes have HID:1 relation would be listed right side. etc. etc... Maybe, field holds relation with the note in the relation table need to be created with UNIQUE keyword. I'm not sure.

What is your opinion @tessus? Do you think that would work?

Regards.

[tessus]

I'm not familiar with backend functions.. Can you tell me what algorithms you're referring to?

I meant the backend meta data and structure that has to be maintained. While a database should usually be in 3NF and make use of foreign keys and constraints, this is not done because of the sync algorithm.

This means that the application has to make sure that the data inserted and handled in the database is consistent.

Your idea of adding these tags to a table with n:m relation is basically correct and pretty much what the existing PR does. Your initial comment rather suggested to put all 4 tags in one column and record, which would basically do nothing but allow you to show these 4 tags in the UI, but that's all. This would not solve anything to make hierarchical tags work.

Either way, as long as nobody is revisiting the PR (which was subsequently reverted) and making the necessary adjustments, nothing is going to happen in that regard.

[darkphase]

Your initial comment rather suggested to put all 4 tags in one column and record, which would basically do nothing but allow you to show these 4 tags in the UI, but that's all. This would not solve anything to make hierarchical tags work.

I didn't propose such thing... I never talked about database structure... At least the intention wasn't that... I'm sorry if I conveyed the idea badly.

I meant the backend meta data and structure that has to be maintained. While a database should usually be in 3NF and make use of foreign keys and constraints, this is not done because of the sync algorithm.

3NF or not, I don't think this is directly relevant to the issue... It's a general thing.

Your idea of adding these tags to a table with n:m relation is basically correct.

I implemented that in many projects with sqlite. Not just n:m relations, It makes all the difference, if tags are stored in a way I tried to explain above.

I cannot access the implementation about this, but I suspected that it was trying to construct parent-child relations between tags after splitting them via "/". So is the performance issues while querying deep hierarchies... ( just suspicion of course don't get offended please. If I'm wrong please correct me )

Either way, as long as nobody is revisiting the PR (which was subsequently reverted) and making the necessary adjustments, nothing is going to happen in that regard.

You should say that nobody has time to implement it at the very beginning =)

[grm9762]

As someone who only keeps one notebook, but hundreds of tags, the switch from evernote to joplin is unfortunately impossible.

I have an idea:

Would it be possible to add at least some kind of "headings" to the tags-overview? I mean only a cosmetic update, i.e. these headings are not tags, but should only help to better organize the real tags.

Hope for your support, that would already bring a huge relief.

[simonsan]

Either way, as long as nobody is revisiting the PR (which was subsequently reverted) and making the necessary adjustments, nothing is going to happen in that regard.

As this PR is locked (for whatever reason), was there effort made to profile the changes? What was the/or a potential bottle neck?

@vsimkus @tessus

[vsimkus]

The feature worked well but it introduced significant performance issues to users with thousands of tags. I planned to come back and fix it but life got to me and I never found time to revisit this. I hope someone else will take it up and bring this feature to Joplin.

As far as I remember the main bottleneck was due to note counting for each tag. It is not straightforward as you may think: you need to check for duplicate notes down in a hierarchy in order to count the number of notes for any upper-level tag otherwise you may count the same note multiple times. Though I think the initial version of this PR did not have such issue; it was introduced during the discussion as there were some changes requested by the maintainers---specifically I was asked not to use SQL views for maintaining the counts (although they were significantly more efficient), since they wanted to move away from any SQL-dialect specific code. That being said I think it should be possible to make it work efficiently with some clever tree algorithms. There is some more discussion in https://github.com/laurent22/joplin/pull/3559, where I've outlined some of the considerations that must be made and possible resolutions to this issue, which I hope may be helpful to anyone who takes it up.

[simonsan]

As far as I remember the main bottleneck was due to note counting for each tag. It is not straightforward as you may think: you need to check for duplicate notes down in a hierarchy in order to count the number of notes for any upper-level tag otherwise you may count the same note multiple times.

What I wonder as you say that, why would you need to count notes at all, doesn't each note have a UUID or maybe even better a ULID that can be used to identify each note? That being said I have not looked deeply into the code base for now, but I would be interested, because maybe it would make sense to introduce some more basic changes to how the data is handled first to then implement some more efficient algorithm/methods for this issue here?

As in: the upper-level tag or in general each tag should be made aware of it's child-tags and which notes (nodes) it contains. Moving from tag to tag (retagging) could then be handled by a tag processor that is handing over the UUID/ULID the the other entity.

[kimcosmos]

Darkphase, you are asking if the nesting tags remained joined for faster indexing? Minimising n:m :m joint foreign key messy lookups. The same reason file paths are stored entire. It would slightly increase the tag length but SQLite does not have any string type sizes. So another DB might be more efficient if tag heirarchy depth was capped at 255 bytes. TINYTEXT type allowing 85 to 255 chars in UTF-8 is not in SQLite

[tessus]

specifically I was asked not to use SQL views for maintaining the counts (although they were significantly more efficient)

Maybe this is something that Laurent would reconsider, if it makes the access way more performant.

[darkphase]

@kimcosmos oh at last some action here... You may use whatever DB you want use of course... It was just a simple suggestion... But why keep real tag hierarchies.. They are just tags... which we can display like hierarchies...

[BeatLink]

Perhaps for note count purposes?

[darkphase]

@BeatLink

I think, it should be easier the way I proposed =) And to calculate sum over the (seemingly) upper branches

sql SELECT count(*) FROM table WHERE tag LIKE 'Books / %' simple...

And this way you can put some notes on the branches (in my opinion as it should be) not being have to put all of them on leaves...

[BeatLink]

huh okay. sounds good then! I'd be excited to see this feature

[ericvaandering]

I will send $100 to anyone or the group of people that solves this. This has been the sticking point in migration away from Evernote for every app I have looked at: no way to organize my ~1000 tags.

[penguin9000]

I'm a big fan of the way Hydrus handles tags for its image database. Each image, in addition to having normal tags, can have hierarchical tags added. For example, if I set up "painting" to be a child tag of "art". Whenever I add "painting" to an image, it automatically also adds "art", saving me the trouble of adding both each time I want to tag an image. When "painting" is removed, it also removes "art".

My favorite part of all is how easy it is to create a search for some logical AND/OR/NOT combination of tags. I like @miciasto's idea for dynamic notebooks, which would be equivalent to a search page in Hydrus. I would find it very useful to have a to-do system centered around notebooks that contain notes matching specific criteria.

By default, tags are displayed in order of how frequently they occur. Right-clicking a tag opens a context menu that allows the user to quickly add to the current search instead of needing to type in the search bar and modify some kind of formatted search string. When a search is performed, the tag list updates to display only those tags applied to the images in the current search.

I would love it if some of these tagging features came to Joplin. I'm sharing this because I hope that the Joplin devs might be able to take some inspiration from Hydrus's code to help implement this feature.

[tsilvs]

In addition to @ericvaandering suggestion, I am ready to pay 100 eur for this feature.

[lvankampenhout]

I'm adding in another € 100 for the person (or group) that brings this feature alive

[ul]

+$100 to the bounty from me.

[popovoleksandr]

Dear @laurent22 and amazing Joplin team! Any updates on this feature? I feel a lot of Joplin users are waiting for this and I'm wondering what is required to put it into the scope for one of the future releases.

[Auzout]

Does there any other software features nest tags? I only found evernote does it well. I need a better alternative.

[ivan-liadniou-epam]

@Auzout, Obsidian has that.

[WillRieske]

Does there any other software features nest tags?

@Auzout https://github.com/standardnotes/app

[grm9762]

@Auzout, Obsidian has that.

@ivan-liadniou-epam Thanks for the tip with Obsidian :) I have already imported all my notes from Evernote to Obsidian.

Unfortunately, my tag structure is not correct in Obsidian. Only the tags that are linked to notes are available. All tags that are without notes and only serve as tag headings are not in Obsidian.

Have you successfully managed to import your tags?

[Auzout]

This plugin enables obisidian nested tags function. https://github.com/pjeby/tag-wrangler