feature/subfolder

[ferdinandyb]

This PR is trying to add creating zettels in subfolders.

Currently work in progress.

[ferdinandyb]

Hey @michal-h21 , I started the WIP PR so there's a place to discuss things. See my comment on the code above. The original happy path was for vimwiki#path#current_wiki_file() == s:vimwiki_dir.path which I don't think triggered many times, so the default behaviour was creating in the current directory. My other PR actually is I think also bad, just to be sure ...

[michal-h21]

Do we want to actually support (1)? I.e. support not placing it in the one single folder that is the zettelkasten? Imho, that is not a useful thing.

I think this means that the current directory is registered as Vimwiki directory. Every Vimwiki directory can act as Zettelkasten, so I think it is correct.

[ferdinandyb]

I'll need testcases anyway so, with the following setup

 let g:vimwiki_list = [{"path": "~/wiki1"},{"path":"~/wiki2"}]
 let g:zettel_options = [{},{'rel_path': 'zettel/']

the expected behaviour would be this:

current working directory -> ZettelNew creation path

• ~/wiki1/randomdir -> ~/wiki1/randomdir
• ~/notwikidir -> ~/wiki2/zettel
• ~/wiki2/subdir -> ~/wiki2/subdir
• ~/wiki2 -> ~/wiki2

Is my understanding correct? If yes, I'd probably also add an option to have ~/wiki2/subdir -> ~/wiki2/zettel.

[michal-h21]

~/notwikidir -> ~/wiki2/zettel

I think the default wiki is always the first one, unless you use something like :ZettelSetActiveWiki

~/wiki2/subdir -> ~/wiki2/subdir
~/wiki2 -> ~/wiki2

Shouldn't be ~/wiki2/subdir and ~/wiki2/ -> ~/wiki2/zettel if you have rel_path': 'zettel/' in the configuration for the second wiki? I thought this was the original intent :)

[ferdinandyb]

~/notwikidir -> ~/wiki2/zettel

I think the default wiki is always the first one, unless you use something like :ZettelSetActiveWiki

I read 2) as meaning this, and since wiki is the first one with non-empty zettel options, that's the one that should be triggered.

~/wiki2/subdir -> ~/wiki2/subdir
~/wiki2 -> ~/wiki2

Shouldn't be ~/wiki2/subdir and ~/wiki2/ -> ~/wiki2/zettel if you have rel_path': 'zettel/' in the configuration for the second wiki? I thought this was the original intent :)

Yes, at least that is what I want, but that is not what I read from 1), hence my confusion and why I wanted to have some examples to make it more clear. I might suggest some doc updates as well I think.

So do we agree on this?:

• ~/wiki1/randomdir -> ~/wiki1/randomdir
• ~/notwikidir -> ~/wiki2/zettel
• ~/wiki2/subdir -> ~/wiki2/zettel
• ~/wiki2 -> ~/wiki2/zettel

Obviously, rel_path should default to just saving to the wiki root as previously.

If we agree, I'll change up my config to add a dummy wiki so I can also test for multiple cases and start fleshing this out.

[michal-h21]

    ~/notwikidir -> ~/wiki2/zettel

I think the default wiki is always the first one, unless you use something like :ZettelSetActiveWiki

I read 2) as meaning this, and since wiki is the first one with non-empty zettel options, that's the one that should be triggered.

Zettel options are not necessary, I would follow Vimwiki in this regards, and select the current wiki.

    ~/wiki1/randomdir -> ~/wiki1/randomdir
    ~/notwikidir -> ~/wiki2/zettel
    ~/wiki2/subdir -> ~/wiki2/zettel
    ~/wiki2 -> ~/wiki2/zettel

except for ~/notwikidir -> ~/wiki2/zettel we agree.

[ferdinandyb]

So basically, drop rule 2) from the current docs:

The newly created wiki-file will be saved in: (1) the current directory if it is in the g:vimwiki_list paths (2) the directory-path of the first g:vimwiki_list with non-empty g:zettel_options (3) the directory-path given by the first g:vimwiki_list entry (4) vimwiki's default directory

It does make sense to let vimwiki decide on which wiki to use.

Just editing in for reference:

• ~/wiki1/randomdir -> ~/wiki1/randomdir
• ~/notwikidir -> ~/wiki1
• ~/wiki2/subdir -> ~/wiki2/zettel
• ~/wiki2 -> ~/wiki2/zettel

[michal-h21]

Yes, that point in docs is obviously wrong :(

[ferdinandyb]

I took a deepdive into vimwiki and looked at how diary is implemented. I also found that you actually had logic for 2) in reference_dir_idx which is indeed not how vimwiki works. Diary works that it uses the buffer to figure out the wiki and if you're not in a vimwiki buffer it falls back to the default wiki. On the other hand, the creation commands do take counts where you can specify the wiki. Adding counts to say ZettelNew should probably be a different PR as this one might get a bit complicated already.

If you think that we should roll like vimwiki usually does I suggest doing the same: i.e. let vimwiki figure out where we are, and mimic how diary works. That means the path would always be given by something like this and I would purge stuff like this.

[ferdinandyb]

Oh yeah, the implication would also be that ~/wiki1/randomdir -> ~/wiki1 as we'd just take the default rel_path which is the root of the wiki.

[ferdinandyb]

Hey @michal-h21,

with the last two commits things actually work for creating new zettels. I haven't yet tested creating them with the other methods yet.

But I've been discovering some things about the code, which I wanted to ask you about. First of all:

function! zettel#vimwiki#initialize_wiki_number() set the wiki to the first one that actually has any zettel options. A sane decision I think, but it was at odds with a totally separate logic in create() that also tried to figure out where we're at. With setting in initial wiki we're basically never outside one.

The other strange thing I see is what I think is duplicate logic trying to get the current wiki number. So there's also function! s:reference_dir_idx() which does a somewhat similar logic as the initialze wiki, yet it's not always used, in a lot of place vimwiki is directly queried with let wiki_nr = vimwiki#vars#get_bufferlocal('wiki_nr') type calls. I did away with some of the calls to reference_dir_idx, but I'm not sure if I see this right, or how you see it.

If we'd want this to work like diary, then both the initalize wiki and the reference dir thing needs to go. Let me know what you think!

[ferdinandyb]

I just noticed you can make PR-s drafts :)

[ferdinandyb]

Found another interesting piece: https://github.com/michal-h21/vim-zettel/blob/067362349ab701ae945285a4797918ed7a649575/autoload/zettel/vimwiki.vim#L776, which references the diary?

• [ ] this needs checking: https://github.com/ferdinandyb/vim-zettel/blob/f1e065741cd046e0d158ba9a99e03181fc64018a/autoload/zettel/vimwiki.vim#L910

Otherwise I think I cleared up all the path issues. Haven't tested the ones I did today. Also see my remarks from yesterday.

[azthec]

There is something strange happening with front_matter tags, on master ZettelNew generates the following correctly:

---
title:
date: 2023-09-06 16:06
type: note
tags:
---
...

On this branch it does work, but I get an error and the template is generated as follows:

type: note
tags:
---
title:
date: 2023-09-06 16:09
---
...

The error log is:

setting  wiki number: 0
new zettel: 230906g
wikiexists1
Error detected while processing function zettel#vimwiki#zettel_new[23]..zettel#vimwiki#save_wiki_page:
line    2:
E121: Undefined variable: idx
E116: Invalid arguments for function vimwiki#vars#get_wikilocal
line    4:
E121: Undefined variable: newfile
line    5:
E121: Undefined variable: newfile
Error detected while processing function zettel#vimwiki#zettel_new[24]..zettel#vimwiki#find_header_end:
line    1:
E484: Cannot open file 0
line    7:
E1098: String, List or Blob required
"zettel/230906g.md" [New] 7L, 56B written
"zettel/230906g.md" 13L, 84B written

My only configuration change is adding to my configs.lua: rel_path = "zettel"

And thanks for working on this feature

[michal-h21]

@ferdinandyb Sorry, I cannot follow right now, I got a bug report tsunami on my other projects, so I am a bit overwhelmed right now. I hope that it will get better next week.

Regarding the diary handling, I never used that feature, so I am a bit surprised that it is there. It seems to originate from a pull request, where I didn't notice that at that moment: https://github.com/michal-h21/vim-zettel/commit/b75d0bed4eac4dd39a13b2131e3d23ac612c9947

[ferdinandyb]

@michal-h21 No worries :)

[ferdinandyb]

There is something strange happening with front_matter tags, on master ZettelNew generates the following correctly:

---
title:
date: 2023-09-06 16:06
type: note
tags:
---
...

On this branch it does work, but I get an error and the template is generated as follows:

type: note
tags:
---
title:
date: 2023-09-06 16:09
---
...

The error log is:

setting  wiki number: 0
new zettel: 230906g
wikiexists1
Error detected while processing function zettel#vimwiki#zettel_new[23]..zettel#vimwiki#save_wiki_page:
line    2:
E121: Undefined variable: idx
E116: Invalid arguments for function vimwiki#vars#get_wikilocal
line    4:
E121: Undefined variable: newfile
line    5:
E121: Undefined variable: newfile
Error detected while processing function zettel#vimwiki#zettel_new[24]..zettel#vimwiki#find_header_end:
line    1:
E484: Cannot open file 0
line    7:
E1098: String, List or Blob required
"zettel/230906g.md" [New] 7L, 56B written
"zettel/230906g.md" 13L, 84B written

This is what you get for only partially testing what you did :) I fixed the idx error and also made a cosmetic tweak to s:zettel_path()

My only configuration change is adding to my configs.lua: rel_path = "zettel"

Try with zettel/!

And thanks for working on this feature

nm thanks for testing :)

[azthec]

I've tested some more commands since your latest commits, I've also updated my path to include the / suffix:

ZettelBacklinks

Works correctly

ZettelGenerateLinks

It seems that internally we still use vimwiki / the vimwiki paths to generate links, so it does not only generate the links to the notes subfolder but to all links in the wiki.

ZettelGenerateTags

Perhaps for a similar reason, when I run VimWikiGenerateTags it includes all in the wiki (as expected). I can't test if this is functional as I always get metadata corrupted both on master and on this feature branch when I run the actual ZettelGenerateTags command

ZettelInbox

Generates unrelated vimwiki notes as unlinked

ZettelNew

Works as expected now!

ZettelOpen & ZettelSearch

It's searching through the entire vimwiki, and not only the zettel subfolder

[ferdinandyb]

Thanks, that's pretty comprehensive :)

On Thu Sep 07, 2023 at 10:13, Rui Balau wrote:

ZettelGenerateLinks

It seems that internally we still use vimwiki / the vimwiki paths to generate links, so it does not only generate the links to the notes subfolder but to all links in the wiki.

ZettelGenerateTags

Perhaps for a similar reason, when I run VimWikiGenerateTags it includes all in the wiki (as expected). I can't test if this is functional as I always get metadata corrupted both on master and on this feature branch when I run the actual ZettelGenerateTags command

Hmm, I might skip fixing this if it is already not working on master. Maybe a separate issue could be opened about this.

ZettelInbox

Generates unrelated vimwiki notes as unlinked

I think I don't understand this. Oh, I do, so this is probably also a pathing issue?

ZettelOpen & ZettelSearch

It's searching through the entire vimwiki, and not only the zettel subfolder

And also a pathing issue. Cool, I'll try to look into these today. Probably need to go through all the calls to vimwiki#something#something and see if they need to be constrained to zettels only.

-- +36305425054 bence.ferdinandy.com

[ferdinandyb]

ZettelOpen should be fixed now.

[ferdinandyb]

I think the Inbox is definitely still not ok, and the appending of a,b,c on Zettels created within the minute should also be tested. I think it works, but somethings a bit fishy and I got to run now :)

[ferdinandyb]

I don't know why ZettelOpen doesn't respect dir ...

[ferdinandyb]

@michal-h21 If you have time I think it is almost done. Currently the only error I know of is that ZettelSearch and ZettelOpen get confused if you're vimwiki is in git, since then you set the working dir for fzf but ag seems to run from the root of the git repo. I think we'd have to disable the vcs option by default or something. I'll also probably send a patch for a defautl search with ripgrep and maybe one with ugrep.

@azthec Could you do another round of testing? Thanks!

[ferdinandyb]

Ok, so I did a little testing about the search and git and it's the vim-router plugin messing with me :)

[ferdinandyb]

Managed to fix that as well (and remove some more unnecessary code).

[ferdinandyb]

@michal-h21 @azthec I think this is now really done, let me know if you find any issues!

[azthec]

All of the following seem to work correctly: ZettelBackLinks, ZettelInbox, ZettelNew, ZettelOpen, ZettelSearch, ZettelCapture, ZettelTitleSelected and ZettelYankName.

ZettelInsertNote

Also seems to be a pathing issue, I missed this one before, it searches on the root directory rather than the zettel subdirectory.

Error detected while processing function 349[30]..<SNR>80_callback[25]..function 349[30]..<SNR>80_callback:
line   23:
Vim(let):E484: Cannot open file /Users/xxx/Documents/Wiki/230919.md

ZettelGenerateLinks & ZettelGenerateTags

I am fine if it still renders vim-wiki links I would leave to @michal-h21 to decide.

[ferdinandyb]

InsertNote should be fixed. I'm looking at GenerateLinks, it should be a two line fix, but I can't get it working somehow ....

[ferdinandyb]

@azthec GenerateLink is also fixed I hope

[azthec]

Everything seems to be working properly now, thanks!

[ferdinandyb]

@michal-h21 I think this is done, except for GenerateTags, but @azthec said those didn't work anyway, so I think that should be dealt with separately. Next up is adding something similar to bibtexcite.vim to handle literature notes separately :)

[michal-h21]

@ferdinandyb I finally got to this issue. Sorry for the delay. I've found one issue – when you create a new Zettel, you cannot get back to the previous position using the backspace key. It seems to be caused by this change in zettel#vimwiki#create function: https://github.com/michal-h21/vim-zettel/blob/afd82b46e39ff149846d38dfe0d1740a13566892/autoload/zettel/vimwiki.vim#L561

The third argument to vimwiki#base#open_link, if used, should be filename that will be put as the previous location in the history. Currently, it uses the wiki directory, so you will get the directory listing when you try to get back to the previous note.

Originally, we used this code:

if s:vimwiki_dir.path =~ vimwiki#path#current_wiki_file()
    " this call should support Vimwiki navigation commands
    call vimwiki#base#open_link(':e ', format, s:vimwiki_dir.path)
  else
    " this happens when we run :ZettelNew outside of wiki. we need to pass
    " the full path, so the file will be saved in the correct directory.
    " Vimwiki navigation commands will not work in the new note.
    call vimwiki#base#open_link(':e ',  "./". format, )
endif

I've found that this code wasn't correct too, as the detection if we are inside a wiki didn't work. This one seems to work:

  if match(vimwiki#path#current_wiki_file(), zettel#vimwiki#path(a:wiki_nr)) == 0
    " let vimwiki to open the wiki file. this is necessary
    " to support the vimwiki navigation commands.
    call vimwiki#base#open_link(':e ',  "./". format, )
  else
    " this happens when we run :ZettelNew outside of wiki. we need to pass
    " the full path, so the file will be saved in the correct directory.
    " Vimwiki navigation commands will not work in the new note.
    let newfilename = zettel#vimwiki#path(a:wiki_nr) . format . vimwiki#vars#get_wikilocal('ext', a:wiki_nr)
    execute(':e ' . newfilename)
  endif

Note that we need to use the normal :e command to open the file if we use :ZettelNew outside of wiki. The first branch is called mainly when you create a new note from visually selected date using the z letter. The filename will need some additional handling if you want to support subfolders.

[ferdinandyb]

Thanks for checking! So if we have a wiki file open we should use the open_link with only two arguments and if we are not in a wiki file we should just :edit the file so the wiki history is not messed up? Is that the purpose of the second branch? I'll do a fix tomorrow probably.

[michal-h21]

I've used :edit because open_link seems to be confused when you pass a full path to it - it wants to create that full path as subdirectories in the wiki root, so it is not useful :)

[ferdinandyb]

I've pushed a fix for the backlinks, but I realized that when you create a link with call zettel#vimwiki#zettel_new_selected() it will only create the correct link now if you call it in the zettel folder, every other time it will be bad (the zettel will be created in the folder while the link will point to the current folder). I'll need to fix this as well I think ...

[ferdinandyb]

Well, you did mention it will be a lot of work :D

[michal-h21]

@ferdinandyb yes, unfortunately, we accumulated too many features over the years, so almost any change can break something :(

[ferdinandyb]

@azthec Would you mind making a comprehensive test again? Just to be on the safe side :) @michal-h21 you might want to look at the last commit including the message to make sure you're ok with the behaviour.

[michal-h21]

@ferdinandyb I am not sure about this, wouldn't be the relative path better? Absolute paths can be broken in wikis that are shared over several machines.

[ferdinandyb]

It's only "absolute" within vimwiki, which means it's relative to the vimwiki root, just not relative to the file the link is in. It should be safe shared across machines.

[azthec]

I've been using this branch for a while, it works pretty well, generating links, tags, unlinked notes etc.

Can't replicate the issue where you cannot get back to the previous position using the backspace key, it works now.

👍

[michal-h21]

@ferdinandyb ah, it is good then. It seems to work for me too.

[michal-h21]

@ferdinandyb so can I merge now, or do you plan more updates?

[ferdinandyb]

@michal-h21 I was just going to come and ask if you want me to do anything else :D So unless do you wish for a change it can be merged, thanks!

[michal-h21]

No, all seems OK, so I merged it. Thank you for all the work!

[ferdinandyb]

Cool, great, thanks!