a thought on filetypes

[basaran]

Hi Prot,

I recently came across your youtube channel through other intermediary hops (iosevka-comfy) and saw the package denote and wanted to ask you if you see any value integrating source code files as notes? You know at times someone might just want to collect various bits of information on stack sites for reviewing later (and usually never does because it gets lost in a combined org or markdown file).

All languages have a comment structure and the adhoc nature of denote might be useful for quickly aggregating and later searching. I understand there is a fine line between this and code snippets but snippets are usually something you use frequently and not for research.

I know we can use org code blocks too, but having the notes in their native format can be useful too when you do something on the shell and help possibly use the note in another editor. Just wanted to ask what you think, thank you for your efforts. I will use denote in the next few weeks.

[protesilaos]

Hello @basaran!

wanted to ask you if you see any value integrating source code files as notes?

It is an interesting concept. With the right keywords and/or with subdirectories, it can work.

All languages have a comment structure and the adhoc nature of denote might be useful for quickly aggregating and later searching.

In terms of implementation, this would use something like the denote-type command.

We would provide a prompt to select the major mode. This seems to return all installed programming modes, though I did not test it thoroughly:

(cl-loop for sym the symbols of obarray
         when (and (functionp sym)
                   (provided-mode-derived-p sym 'prog-mode))
         collect sym)

The tricky part, then, is how to arrange for all the front matter to be included. We could simply refrain from adding it? Or we can use the denote-text-front-matter and simply call comment-region after we insert it---Emacs will take care to use the appropriate comment of the given major-mode.

Also, what happens with the linking facility? In principle, the buttonisation we use for non-Org files should already work, but it needs to be tested and any issues be consider further.

Overall, I like the idea. Though I need to consider how it would be fleshed out, ideally without impacting what we already have and without adding too much complexity on top.

At any rate, there is work being done by @jeanphilippegg on streamlining the commands we provide, so I would wait for that first.

[basaran]

Hi there)

simply call comment-region

I thought of the same thing, let emacs deal with it :)

Sourcenote (I just made this up) files could be written to js-mode or lisp-mode and so on. That way, it can be expanded later with modifying the list function. My lisp is very new but I will try and send commits to the repository as well.

[basaran]

Also, what happens with the linking facility?

I don't think linking is necessary for this kind of note (at least at this stage). Idea is to be able to store bits of random code in a streamlined way, and help search it through tags.

For instance, you have a piece of javascript code that you want to test (fetching and writing files), so you store this snippet through denote, quickly issue the command to open the file path in a terminal and start nodemon on it to do things on it. I hope it makes sense.

I think these source code files should be stored in their own directories.

[protesilaos]

simply call comment-region

I thought of the same thing, let emacs deal with it :)

Good! This is efficient. I am also doing it with a command I have to produce a scratch buffer for a given mode.

Sourcenote (I just made this up) files could be written to js-mode or lisp-mode and so on. That way, it can be expanded later with modifying the list function. My lisp is very new but I will try and send commits to the repository as well.

Yes, that is the idea. We first prompt for the major-mode and we then need some way to get the filetype extension for it. The rest is easy as we already have everything in place.

I don't think linking is necessary for this kind of note (at least at this stage). Idea is to be able to store bits of random code in a streamlined way, and help search it through tags.

I get the point, though I think we need to consider it. If we provide a feature, it is better to make it a first-class citizen. Besides, I think we do not need to do anything more than we already have.

I think these source code files should be stored in their own directories.

You mean we should arrange for Denote to have a preference for this? I think it is better to let the user decide. For example, I would prefer to add a snippet keyword (or something like that) and keep everything in one directory. This is consistent with how we handle things in general.

[basaran]

You mean we should arrange for Denote to have a preference for this?

Yes, I think it should be based on user choice. The idea behind the directory is for easier scaffolding. I do javascript mostly, so I will give the example based on that. But it applies to anything really.

You find a snippet that uses some package that you install via NPM. You grab that and denote it. Later you get into that folder (buffer path) and npm init and install the dependencies and do a bunch of things on it.

It should be possible to filter out the other files in the note directory, as the written note would have a specific format to it in its filename. What do you think?

[basaran]

we then need some way to get the filetype extension for it

auto-mode-alist seem to have the extensions by the way.

[protesilaos]

`auto-mode-alist' seem to have the extensions by the way

It has regular expressions, which means that we need to process them further. It will definitely give us problems. This, for example:

("\\.[ckz]?sh\\'\\|\\.shar\\'\\|/\\.z?profile\\'" . sh-mode)

Also, I noticed that it returns what would be a false positive for us. For example, I have this in my dotemacs:

(add-to-list 'auto-mode-alist '("PKGBUILD" . sh-mode))

If I evaluate the following:

(rassq 'sh-mode auto-mode-alist)

It returns just the cons cell with the PKGBUILD even though it is not the only one and certainly not what we would need.

Perhaps, then, the solution is to let the user specify a string for denote-file-type which will signify the file type extension. Then we can search if that string exists in auto-mode-alist. If it does, we insert the front matter and comment is as we described earlier. Otherwise we do not insert any front matter.

What do you think?

[protesilaos]

The idea behind the directory is for easier scaffolding. I do javascript mostly, so I will give the example based on that. But it applies to anything really. [...]

For this to work, the command would have to prompt for a directory and also create it if it does not exist. Currently, Denote does not create subdirectories. We need to think if this is something we should support. Otherwise what you describe would have to be a user-level code that builds on a generic function that we will provide.

[basaran]

I agree, now that I think more through, denote's fundamental design should remain intact.

Directory -> Bunch of Notes

Since the auto-mode-alist works the other way around, a denote-filetypes-alist can be used for user defined file types to choose from as a luxury.

For the moment, I think for my use case, I can use org type and create a command that will:

1. Get the buffer name, grab the note name
2. Create a directory at a desired location (probably the same notes folder with the same name.
3. Tangle the source code block via org-babel-tangle. It takes arguments.
4. Open a terminal somewhere (emacs, tmux etc) to drop into.

Perhaps it would be better this way, because you might eventually end up adding notes to the snippet later anyway. Org will allow more flexibility for such purpose. There is also detangle to put things back into the note file.

For everything else like you said, I suppose we should have some hooks.

[protesilaos]

Since the auto-mode-alist works the other way around, a denote-filetypes-alist can be used for user defined file types to choose from as a luxury.

Yes, this is an option. I think the eventual goal is to support more file types, as we really only need to make fairly minor tweaks to the code once we get the file type extension right. Currently there is refactoring working going on, as I noted before, but I will definitely work on this afterwards.

For the moment, I think for my use case, I can use org type and create a command that will: [...]

Very well! This would be a good use of Org's feature set.

For everything else like you said, I suppose we should have some hooks.

Indeed. We have not introduced hooks yet as it is still not clear what we need exactly. I prefer to add them at the right moment, so that we avoid any back-and-forth with the underlying code. Especially as hooks are user-facing and we do not want to break things willy nilly.

[basaran]

If we had an after-denote-created-hook, I could insert an src_block to it depending on some tags or keywords (we should have a command to give user the tags on the document).

Perhaps this speicial source code case could be streamlined into denote as any number of points below.

1. prog-mode org type
2. user can then choose from the list of types (emacs-lisp)
3. denote will insert a global property org property
4. #+PROPERTY header-args:emacs-lisp :mkdirp yes :tangle denoteFolder/denoteFilename/denoteFilename.Extension

Of course, all of this could also be accomplished on the fly with a hook.

[9viz]

[ I only gave this a very light skim. ]

Yes, that is the idea. We first prompt for the major-mode and we then need some way to get the filetype extension for it. The rest is easy as we already have everything in place.

If you know the major-mode, then you also know how to write a comment then instead of messing with the extension, why not just write the elisp equivalent of

;; -*- mode: emacs-lisp; -*-

at the top of the file? When you visit that file, Emacs should do TRT. I doubt there's a toolkit out there that would be pedantic about the file extension. Or the sh-equivalent would be

# -*- mode: sh; -*-

This is similar to what add-file-local-variable-prop-line does so there's an example already.

Also,

For instance, you have a piece of javascript code that you want to test (fetching and writing files), so you store this snippet through denote, quickly issue the command to open the file path in a terminal and start nodemon on it to do things on it. I hope it makes sense. [emphasis mine]

Sounds like a shell-command-on-note command to me? The difference between this command and just opening the dired-buffer would be that the former could match against the filename which is probably more convenient.

[protesilaos]

If you know the major-mode, then you also know how to write a comment then instead of messing with the extension, why not just write the elisp equivalent of [...]

That is an option. Personally, I do not like such variables because (i) they do not work outside Emacs as far as I can tell and (ii) you lose the chance to filter by file type extension in Dired.

---

At any rate, I am not sure we should implement this feature after all. It would complicate things for us.

[9viz]

If you know the major-mode, then you also know how to write a comment then instead of messing with the extension, why not just write the elisp equivalent of [...]

That is an option. Personally, I do not like such variables because (i) they do not work outside Emacs as far as I can tell and (ii) you lose the chance to filter by file type extension in Dired.

You mean you use something other than Emacs?  ;-) I kid but (ii) is a real pain.  I wish a "metadata" glob was a standard in Unix i.e., ls :type="image/" would list all image files.

At any rate, I am not sure we should implement this feature after all. It would complicate things for us.

Indeed.  IMO, one can just execute the region as a shell-command and call it a day for simple and straightforward snippets.  I have the following bound to M-|,

(defun vz/execute-region-as-shell-command (beg end)   "Execute the region from BEG to END as shell command. If the region is not active, then the current line is executed instead.  For convenience, \"[%$#] \" is stripped from the beginning of the string, if present.  If the command begins with

, then it executed as root using the doas TRAMP method.

The command is actually passed to shell-command+'."   (interactive "r")   (unless (use-region-p)     (setq beg (line-beginning-position)           end (line-end-position)))   (let* ((string (string-trim-left (buffer-substring beg end)))          (doasp (string-prefix-p "#" string))          (string (string-trim-left string "\\\(?:[%$#][[:space:]]\)?")))     (shell-command+ (concat (and doasp "doas ") string) (point) (point))))

[ shell-command+ handles the doas stuff for me.  ]

[protesilaos]

You mean you use something other than Emacs? ;-)

Haha! I only use Emacs. This is just me thinking about the longer-term consequences.

I wish a "metadata" glob was a standard in Unix

Yes, that would be nice.

Indeed. IMO, one can just execute the region as a shell-command and call it a day for simple and straightforward snippets. I have the following bound to M-|

Thanks for sharing! I have not had a scenario where I needed such a feature, as I only code in Elisp (with the occasional shell script). Maybe others reading this will put it to good use.

[protesilaos]

Hello again, folks! I am closing this issue as such a feature would greatly complicate the code base of Denote. Thanks for your time!

[basaran]

Hi, in case somebody needs it, this is the command I started using for easy scaffolding. As a convenience, it copies the path to clipboard, so you can quickly switch to a terminal to do stuff in that folder, and take notes in the note file. Whatever that note file might be.

(defun mb/yank-project-from-note ()
"creates a folder using buffer-file-name and copies it to
clipboard."
  (interactive)
  (setq projectFolder (file-name-sans-extension buffer-file-name))

  (let ((projectile-dir (concat projectFolder))
        (select-enable-primary t))
    (make-directory projectile-dir :parents)
    (kill-new projectile-dir)))

I will probably modify this later to insert the meta data to the front matter, about the folder.