search

protesilaos / denote

Simple notes for Emacs with an efficient file-naming scheme
https://protesilaos.com/emacs/denote
GNU General Public License v3.0
506 stars 54 forks source link

denote-rename-file and associated commands (denote-rename-file-keywords) replace formatting of the title with sluggified title #439

Open hapst3r opened 6 days ago

hapst3r commented 6 days ago

Hej prot,

as a new phenomenon, I noticed that when I rename a filebuffer I am currently working on by adding keywords with denote-rename-file-keywords, my formatting gets overwritten with the sluggified title. I tried searching if this is intended behaviour (functions/variables, release notes) but couldn't find anything related.

This is what my front matter is supposed to look like:

#+title:      AS Gaza, Rudner: Gaza ist in uns, wir sind in Gaza
#+date:       [2024-09-17 Di 09:50]
#+filetags:   :innenpolitik:
#+identifier: 20240917T095025

This is what my front matter looks like after adding a keyword using denote-rename-file-keyword:

#+title:      as-gaza-rudner-gaza-ist-in-uns-wir-sind-in-gaza
#+date:       [2024-09-17 Di 09:50]
#+filetags:   :innenpolitik:krieg:
#+identifier: 20240917T095025

Since I use denote-rename-buffer-mode, I find this sluggification undesirable, because it takes away from the readability afforded by automatic buffer renaming.

Have a good day :)

jeanphilippegg commented 5 days ago

This is not desirable.

What version of Denote are you using? I cannot reproduce this on my end.

Can you give the steps you are doing to get this behavior? For example, here is one of my tests:

1- Call denote to create a note with title "A note". 2- Save the note. 3- With the note still open, call denote-rename-file-keywords and add a keyword.

Also, can you evaluate this while inside the problematic note (using M-:) and share the output?

(denote-retrieve-front-matter-title-value buffer-file-name 'org)
protesilaos commented 5 days ago

From: Jean-Philippe Gagné Guay @.***> Date: Tue, 17 Sep 2024 17:22:50 -0700

This is not desirable.

What version of Denote are you using? I cannot reproduce this on my end.

I cannot reproduce it on my end either.

-- Protesilaos Stavrou https://protesilaos.com

LukasCBossert commented 4 days ago

I had this issue as well, but with the latest update I cannot reproduce it either.

hapst3r commented 4 days ago

This is not desirable.

What version of Denote are you using? I cannot reproduce this on my end.

Can you give the steps you are doing to get this behavior? For example, here is one of my tests:

1- Call denote to create a note with title "A note". 2- Save the note. 3- With the note still open, call denote-rename-file-keywords and add a keyword.

Also, can you evaluate this while inside the problematic note (using M-:) and share the output?

(denote-retrieve-front-matter-title-value buffer-file-name 'org)

I made a new note with title Test. Changing the title to Test a filename: de sluggify and renaming the file using denote-rename-file-using-front-matter, the title remained the same(!).

However, as soon as I add a keyword to the note using denote-rename-file-keywords, or rename the file with denote-rename-file, the title is sluggified.

The value of (denote-retrieve-front-matter-title-value buffer-file-name 'org) is Test a filename: de sluggify, the correct title.

EDIT: Okay. After gradually re-building my configuration, I found that with (find-file-visit-truename t), this phenomenon occurs regularly. I am able to reproduce in emacs -Q following these steps: 1) emacs -Q 2) M-x package-initialize 3) M-x package-list-packages 4) install denote 5) (setq find-file-visit-truename t) 6) create note 7) denote-rename-file

Again, it is my impression that this change in behaviour is recent, as in, appearing at some point in the last two weeks.

jeanphilippegg commented 3 days ago

Thank you! I can reproduce the issue if I have these files:

~/Documents    <-----  symlink to some_other_directory/
~/some_other_directory/notes/

Here are the steps:

1. Run "emacs -Q".
2. Install denote
3. Create a new note with "M-x denote" with title "A Title".
4. Save and CLOSE the new note.
5. Open the note again.
6. Run "denote-rename-file". The suggested title prompt is "a-title".

Do you have symbolic links involved as well?

It seems like the issue is when checking that the file is in the current directory.

@protesilaos

I am not sure what is the best way to handle this. Maybe fix denote-filename-is-note-p and denote--dir-in-denote-directory-p to use file-truename to do the comparison?

Making (denote-directory) return the file truename may not be a good idea as I think we display it in some of the prompts and the symlink probably looks better.

hapst3r commented 3 days ago

Thank you! I can reproduce the issue if I have these files:

~/Documents    <-----  symlink to some_other_directory/
~/some_other_directory/notes/

Here are the steps:

1. Run "emacs -Q".
2. Install denote
3. Create a new note with "M-x denote" with title "A Title".
4. Save and CLOSE the new note.
5. Open the note again.
6. Run "denote-rename-file". The suggested title prompt is "a-title".

Do you have symbolic links involved as well?

It seems like the issue is when checking that the file is in the current directory.

Yes I do. It has become second nature to me which is why I didn't mention it :facepalm:

I also have set (denote-kill-buffers 'on-creation), but in my testing this didn't make a difference with regards go the problem at hand. I might be wrong, though

protesilaos commented 3 days ago

[... 27 lines elided]

Thank you!

@protesilaos

I am not sure what is the best way to handle this. Maybe fix denote-filename-is-note-p and denote--dir-in-denote-directory-p to use file-truename to do the comparison?

We do use 'file-truename' to get the subdirectories. I think we have not had any issues with this approach.

Making (denote-directory) return the file truename may not be a good idea as I think we display it in some of the prompts and the symlink probably looks better.

If it is only about the prompt, then we can work around that. But let's first do the other functions and keep this for last.

-- Protesilaos Stavrou https://protesilaos.com

protesilaos commented 3 days ago

From: hapst3r @.***> Date: Thu, 19 Sep 2024 22:59:34 -0700

[... 19 lines elided]

Do you have symbolic links involved as well?

It seems like the issue is when checking that the file is in the current directory.

Yes I do. It has become second nature to me which is why I didn't mention it :facepalm:

Can you tell me more about your workflow? I am curious what the use-case is. This can help anticipate future bugs.

-- Protesilaos Stavrou https://protesilaos.com

LukasCBossert commented 2 days ago

ok, I can confirm this issue: it happens when I am in dired and add keywords, then the titles are all "messed up"

jeanphilippegg commented 2 days ago

ok, I can confirm this issue: it happens when I am in dired and add keywords, then the titles are all "messed up"

You also have symbolic links?

hapst3r commented 2 days ago

[... elided lines] Can you tell me more about your workflow? I am curious what the use-case is. This can help anticipate future bugs. … -- Protesilaos Stavrou https://protesilaos.com

Yeah sure. Essentially I have one folder that is supposed to store all state data (config, music, savegames, notes, ...). This folder is synchronized using git and a personal cloud to facilitate backups and re-deployment on new machines. On a new machine, using stow, I redeploy all data to the correct locations.

If I want my notes at ~/notes/denote/, I have them stored at ~/.site-data/notes/denote.

Since denote notes are very long by nature (mine are, anyway), I wanted the short version to be displayed (omitting the .site-data component). Now I can't remember I set find-file-truename because of that or if there was another reason, to be honest...

protesilaos commented 2 days ago

Thank you @hapst3r for the explanation!

I just merged the changes done by @jeanphilippegg to account for symlinks. Please give it a try.

hapst3r commented 2 days ago

Protesilaos Stavrou @.***> writes:

Thank you @hapst3r for the explanation!

I just merged the changes done by @jeanphilippegg to account for symlinks. Please give it a try.

I've just tested it on an existing and a new note, both with (find-file-visit-truename nil) and (find-file-visit-truename t). It works :-)

LukasCBossert commented 2 days ago

ok, I can confirm this issue: it happens when I am in dired and add keywords, then the titles are all "messed up"

You also have symbolic links?

yes, my emacs/denote-folder is also within a path with symbolic links.

LukasCBossert commented 2 days ago

I just tested with the latest update and in my use-case (being in dired and adding/removing keywords I noticed two things: 1) title still got slugged but keywords in #+filetags: have been updated properly (cf. #441 ) 2) Keyword was added to filename and title in file stays the same (not getting slugged), but the keywords in #+filetags: are not updated.

jeanphilippegg commented 1 day ago

I just tested with the latest update and in my use-case (being in dired and adding/removing keywords I noticed two things:

1. title still got slugged but keywords in `#+filetags:` have been updated properly (cf. [Write `#+filetags: XXX` to file when calling `denote-dired-rename-marked-files-add-keywords` #441](https://github.com/protesilaos/denote/issues/441) )

2. Keyword was added to filename and title in file stays the same (not getting slugged), but the keywords in `#+filetags:` are not updated.

Can you share the details of your tests? These elements would be useful:

jeanphilippegg commented 1 day ago

This issue is tricky. I have reverted my last pull request. Though, it fixed your specific case, it introduces other breakage.

Let's say denote-directory is ~/Documents/notes and you have the following files:

~/Documents/notes/some-dir-link/   <---- symbolic link to ~/some-dir/
~/some-dir/

When visiting ~/some-dir/, how can we know that we are actually visiting a directory that is linked inside denote-directory? The only way is to scan denote-directory for a directory that links to ~/some-dir/.

This is technically doable. However, this scanning of denote-directory is a costly operation. We probably don't want to introduce this in denote-filename-is-note-p, as it is used in many places and it intended to be a quick check.

So I think we will just have to include this warning somewhere:

If your `denote-directory` or its subdirectories involve symbolic links, make
sure that you access them using those links (and that
`find-file-visit-truename` is nil). Otherwise, Denote features will not work.

This is not ideal, but if you visit "~/some-dir/some-note.org" (directly or with find-file-visit-truename t), we cannot know that this file is in denote-directory without the costly operation of scanning the whole denote-directory.

LukasCBossert commented 1 day ago

I just tested with the latest update and in my use-case (being in dired and adding/removing keywords I noticed two things:

1. title still got slugged but keywords in `#+filetags:` have been updated properly (cf. [Write `#+filetags: XXX` to file when calling `denote-dired-rename-marked-files-add-keywords` #441](https://github.com/protesilaos/denote/issues/441) )

2. Keyword was added to filename and title in file stays the same (not getting slugged), but the keywords in `#+filetags:` are not updated.

Can you share the details of your tests? These elements would be useful:

* Directory structure

* Commands that you used and where you execute them (using the symlink or through the actual directory)

* Value of `find-file-visit-truename`

Directory Structure

 ~/@mycene/00-09_system -> /Users/lukascbossert/@cloud/nextcloud/00-09_system

and within 00-09_system

00-09_system
├── 01_emacs
 │   ├── 01.00_system
 │   ├── 01.01_attachments
 │   ├── 01.02_bib
 │   ├── 01.03_img
 │   ├── 01.04_private
 │   ├── 01.05_pim <----- my denote-directory
 │   ├── 01.06_
 │   ├── 01.06_denote
 │   ├── 01.07_texografie-emaix
 │   ├── config.el
 │   ├── custom.el
 │   ├── init.el
 │   └── packages.el

commands I used

I am in dired in the folder 01.05_pim and I run M-x denote-dired-rename-marked-files-add-keywords

value of variable

find-file-visit-truename is nil

jeanphilippegg commented 1 day ago

I cannot reproduce the issue with a similar structure.

LukasCBossert commented 1 day ago

I cannot reproduce the issue with a similar structure.

* From the place you execute `M-x denote-dired-rename-marked-files-add-keywords`, can you share the output of `default-directory` (using `M-:`)

M-: default-directory says: "~/@mycene/00-09_system/01_emacs/01.05_pim/" which is a symbolic place, it is actually
/Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs/01.05_pim

* If you open one of the notes and execute `buffer-file-name` (using `M-:`), what is the output?

it says: "/Users/lukascbossert/@mycene/00-09_system/01_emacs/01.05_pim/20211128T153953--research-data-management-rdm__concept.org" which is then the "correct/normal" path.

* What is the behavior when you run the keywords command above? Title sluggified? Keywords modified in front matter?

I ran denote-dired-rename-marked-files-add-keywords and the keyword has been added to the filename and to #+filetags:; however the title got sluggified :(

* What is the value of `denote-directory`?

"/Users/lukascbossert/.config/doom/01.05_pim/"

jeanphilippegg commented 1 day ago

Ok, thank you! I will try experimenting with this. One other question:

M-: default-directory says: "~/@mycene/00-09_system/01_emacs/01.05_pim/" which is a symbolic place, it is actually /Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs/01.05_pim

...

* What is the value of `denote-directory`?

"/Users/lukascbossert/.config/doom/01.05_pim/"

If I understand correctly, the real file is /Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs/01.05_pim/. Is your denote-directory ("/Users/lukascbossert/.config/doom/01.05_pim/") another symbolic link?

LukasCBossert commented 1 day ago

Ok, thank you! I will try experimenting with this. One other question:

M-: default-directory says: "~/@mycene/00-09_system/01_emacs/01.05_pim/" which is a symbolic place, it is actually /Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs/01.05_pim ...

* What is the value of `denote-directory`?

"/Users/lukascbossert/.config/doom/01.05_pim/"

If I understand correctly, the real file is /Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs/01.05_pim/. Is your denote-directory ("/Users/lukascbossert/.config/doom/01.05_pim/") another symbolic link?

"/Users/lukascbossert/.config/doom/01.05_pim/" is not directly a symbolic link. But there might be two symbolic links in action: 1) "/Users/lukascbossert/.config/doom -> /Users/lukascbossert/@cloud/nextcloud/00-09_system/01_emacs 2) /Users/lukascbossert/@mycene/00-09_system -> /Users/lukascbossert/@cloud/nextcloud/00-09_system

In 00-09_system I have 01_emacs which is "populated" with the content from .config/doom (see above)

Edit: @jeanphilippegg Not sure if this is important: This is how I define my denote-directory

  (setq denote-directory
        (expand-file-name (concat doom-user-dir "01.05_pim/")))
jeanphilippegg commented 1 day ago

Thank you! I could reproduce the issue, but not with the latest code. Are you sure you have the latest code?

Can you evaluate this and try again:

(defun denote-filename-is-note-p (filename)
  (and (string-prefix-p (file-truename (denote-directory))
                        (file-truename (expand-file-name filename)))
       (denote-file-has-identifier-p filename)
       (denote-file-has-supported-extension-p filename)))

If you want to do a quick test, you can just open your problematic note and evalute (denote-retrieve-title-or-filename buffer-file-name 'org). We want it to be not sluggified.

LukasCBossert commented 1 day ago

@jeanphilippegg With the snippet everythings seems to work and I didnt see any issue of sluggified titles anymore.

(denote-retrieve-title-or-filename buffer-file-name 'org) gives the proper title.

jeanphilippegg commented 1 day ago

Ok, thank you! This makes the situation clear.

The latest code creates other issues and should be reverted though. We need to think of a way to handle symlinks.

With your setup, you could set denote-directory to the directory that you are going to use to access your notes, that is "~/@mycene/00-09_system/01_emacs/01.05_pim/". This way, both the latest code and the previous code should work.

LukasCBossert commented 1 day ago

btw, I am running denote with Build: HEAD -> master, origin/master, origin/HEAD c1b04ff 2024-09-21 00:06:04 +0000

my denote-directy is set to (expand-file-name (concat doom-user-dir "01.05_pim/")) which expands with M-: to "/Users/lukascbossert/.config/doom/01.05_pim/" which is the same as "~/@mycene/00-09_system/01_emacs/01.05_pim/" because of the symlink from ~/.config/doom/--> ~/@mycene/00-09_system/01_emacs.

Do you suggest to set explicitly (setq denote-directoy "~/@mycene/00-09_system/01_emacs/01.05_pim/") ? Or is it fine as it is?

jeanphilippegg commented 1 day ago

btw, I am running denote with Build: HEAD -> master, origin/master, origin/HEAD c1b04ff 2024-09-21 00:06:04 +0000

I cannot find this commit in Denote repo. Maybe this is a Doom repo.

Do you suggest to set explicitly (setq denote-directoy "~/@mycene/00-09_system/01_emacs/01.05_pim/") ? Or is it fine as it is?

Yes, I suggest changing it. It is logically the same thing, but that is what Denote does not handle well as the moment. For now, I think it would be better if you set denote-directory to the directory that you will actually use to handle your notes ((setq denote-directory "~/@mycene/00-09_system/01_emacs/01.05_pim/")).

protesilaos commented 1 day ago

From: Jean-Philippe Gagné Guay @.***> Date: Sat, 21 Sep 2024 06:14:43 -0700

[... 16 lines elided]

This is technically doable. However, this scanning of denote-directory is a costly operation. We probably don't want to introduce this in denote-filename-is-note-p, as it is used in many places and it intended to be a quick check.

So I think we will just have to include this warning somewhere:

If your `denote-directory` or its subdirectories involve symbolic links, make
sure that you access them using those links (and that
`find-file-visit-truename` is nil). Otherwise, Denote features will not work.

This is not ideal, but if you visit "~/some-dir/some-note.org" (directly or with find-file-visit-truename t), we cannot know that this file is in denote-directory without the costly operation of scanning the whole denote-directory.

How about we introduce a user option which will do this costly operation? It will be an opt-in feature, so that not everyone has to incur this performance penalty.

-- Protesilaos Stavrou https://protesilaos.com

protesilaos commented 1 day ago

From: Jean-Philippe Gagné Guay @.***> Date: Sat, 21 Sep 2024 14:30:23 -0700

Ok, thank you! This makes the situation clear.

The latest code creates other issues and should be reverted though. We need to think of a way to handle symlinks.

Fine, I will do it now.

-- Protesilaos Stavrou https://protesilaos.com

jeanphilippegg commented 6 hours ago

How about we introduce a user option which will do this costly operation? It will be an opt-in feature, so that not everyone has to incur this performance penalty.

It could be a good solution. And also warn users of its cost.

protesilaos commented 5 hours ago

From: Jean-Philippe Gagné Guay @.***> Date: Sun, 22 Sep 2024 20:42:47 -0700

How about we introduce a user option which will do this costly operation? It will be an opt-in feature, so that not everyone has to incur this performance penalty.

It could be a good solution. And also warn users of its cost.

Do you want to do it, or shall I give it a try?

-- Protesilaos Stavrou https://protesilaos.com