Is it necessary to have fnl/dotfiles?

[kkharji]

Hey @Olical, I wondee if it possible to have different structure of files in fnl/. for example have all second level dir under dotfiles be under lua.

Thanks

[Olical]

It's possible to change Aniseed to do this, the reason I designed it this way was for simplicity and lack of decisions required.

So here's the entire source of the dotfiles module:

(module aniseed.dotfiles
  {require {nvim aniseed.nvim
            compile aniseed.compile}})

(def- config-dir (nvim.fn.stdpath :config))
(compile.add-path (.. config-dir "/?.fnl"))
(compile.glob "**/*.fnl" (.. config-dir "/fnl") (.. config-dir "/lua"))
(require :dotfiles.init)

So the entire fnl directory is compiled, or should be. It's just dotfiles.init that's important. Maybe this is all the info you need really?

[kkharji]

Thank a lot, it is. I believe it would be better if aniseed compile all fnl dir default. This way it feels more like lua dir, or how it would be used normally by other configuring with lua.

Nonetheless, this just my opinion, as I like to have my own structure. If I want to group configuration under dotfiles names I should be able to do that, otherwise an fnl/init.fnl, (like nvim/init.vim & lua/init.lua) should be the entry point. Idk perhaps it would be better to have fnl dir complied by default.

Additionally, I want to ask if aniseed recompile everything when a a single file get changed?

Thanks @Olical

[Olical]

The problem with a shallow tree like lua/init.lua is that it's a global module search path, any other file/module called init will collide with this, hence the enforced prefix and why all Aniseed plugins rename their dependencies to put their name in front of each module. (require :init) could mean the one in your dotfiles or the one in some other Lua plugin (which is bad on their part). You need to reach some level of specificity with Lua module naming or you'll collide with everything I'm afraid.

The entire fnl dir is compiled, so you can put other directories in there, just the entry point is dotfiles.init by convention, but that's all. Every other part of your code can be in ../dotfiles.

And no, there's no auto recompile yet, I recommend Conjure for all interactive work with Fennel/Aniseed. If you would like to recompile on save you'd need to hook up an autocmd that ran a similar glob call to what I showed above but then you have the issue of finding every changed Lua file and ensuring they're reloaded from disk. This is hard and easy to get wrong which would introduce very subtle but bad bugs, which is why I didn't attempt it.

If you meant does everything get recompiled when you re-open Neovim, that's a no too. Aniseed compiles each file on an individual basis and compares the modified time of the .fnl to the .lua, so it'll only recompile what changed. If you need to recompile everything you can delete your lua dir which is what I do when I sync my dotfiles across machines to ensure nothing desyncs in weird ways. (like deleting a .fnl will not delete the .lua which isn't normally a problem but can be in rare cases)

[kkharji]

I understand why you went with dotfiles namespace. But usually lua/init.lua doesn't get referenced anywhere, only in init.vim, in which it get referred to by full path (i.e. luaf lua/init.lua).

The entire fnl dir is compiled, so you can put other directories in there, just the entry point is dotfiles.init by convention, but that's all. Every other part of your code can be in ../dotfiles

Nice, that just what I need. How about having aniseed/init.fnl or aniseed.fnl as entry point, much better than dotfiles 😬.

If you meant does everything get recompiled when you re-open Neovim, that's a no too.

Oh that great, sorry I doubt that anissed only compiles changed files. Nonetheless, I'm not sure if the current way of checking is efficient (not that I have a better way) due to a wired refresh of ui, less the split of second when re-opening neovim, I believe it might be something todo with checking mechanism.

[Olical]

So are you mostly concerned with dotfiles.* being the chosen prefix? I'm definitely open to that although it'd have to be backwards compatible which should be okay (for a while anyway until the old way can be phased out with a warning etc).

Maybe it could do with some benchmarking (I really want to add a time macro like Clojure's) but here's the code that compiles a file and checks if it should even do it.

(defn file [src dest opts]
  "Compile the source file into the destination file if the source file was
  modified more recently. Will create any required ancestor directories for the
  destination file to exist."
  (when (or (and (a.table? opts) (. opts :force))
            (> (nvim.fn.getftime src) (nvim.fn.getftime dest)))
    (let [code (a.slurp src)]
      (match (str code {:filename src})
        (false err) (nvim.err_writeln err)
        (true result) (do
                        (-> dest fs.basename fs.mkdirp)
                        (a.spit dest result))))))

Maybe glob could be altered so it filters out whole directories based on getftime :thinking: not sure if the directory modified time will be in line with the modified time of the newest file though.

(defn glob [src-expr src-dir dest-dir opts]
  "Match all files against the src-expr under the src-dir then compile them
  into the dest-dir as Lua."
  (let [src-dir-len (a.inc (string.len src-dir))
        src-paths (->> (nvim.fn.globpath src-dir src-expr true true)
                       (a.map (fn [path]
                                (string.sub path src-dir-len))))]
    (each [_ path (ipairs src-paths)]
      (when (or (a.get opts :include-macros-suffix?)
                (not (string.match path "macros.fnl$")))
        (file (.. src-dir path)
              (string.gsub
                (.. dest-dir path)
                ".fnl$" ".lua")
              opts)))))

But as you can see, there's not much code here, it shouldn't take long to run (especially in LuaJIT), so the slowness may lie elsewhere. Only benchmarking will tell I guess.

[Olical]

What do you think would be a better name for this concept, instead of dotfiles? Like aniseed.editor-config? I called it dotfiles because they reside under dot prefixed files and "dotfiles" is a common term for config you carry with you somehow.

[kkharji]

@Olical After some thoughts, I come to the conclusion that perhaps there is a more simpler, less sophisticated strategy that doesn’t involve checking the file system and compere stuff, nor compiles during initialization. This strategy involves to autocmd, and two functions

• On a nvim/fnl/* BufPostWrite event, a function get called that append the file path to a table of files to be recompiled.
• On vim exist event, another function get called to read this table, delete duplicates, and create a background process to recompile the table’s files.

One drawback to this method is that changes won’t be reflected on new vim instance unless the current instance get closed. One solution to this is to provide the user with a command to call the exit event functions manually. A better one is perhaps to make the table persistent through saving/appending it to a lua file that returns the table when required; so that on new instance, if that file is readable — which wouldn’t be the case if the other instance is closed — then our exit event function is then called.

[kkharji]

What do you think would be a better name for this concept, instead of dotfiles? Like aniseed.editor-config? I called it dotfiles because they reside under dot prefixed files and "dotfiles" is a common term for config you carry with you somehow.

🤔 .... aniseed.env sounds right, after all dotfiles are what influence and shape our environment. But again I wish if you reconsider having init.fnl as the entry point 🙏

Maybe it could do with some benchmarking (I really want to add a time macro like Clojure's) but here's the code that compiles a file and checks if it should even do it.

Definitely, it is also extremely beneficial to compare a lua based and fnl based vim config. I could be your test subject.

[Olical]

I'm not sure if I'd consider that simpler or less sophisticated, I feel like that'll involve more mechanisms and branching conditions. My metric for simplicity is the amount of "if"s and "but"s I have to insert while explaining how it works :sweat_smile:

through saving/appending it to a lua file that returns the table when required

I'd like to avoid introducing anything stateful other than a single flow of derived state that can be reproduced in a "dumb" way, as soon as we're modifying files and building up something over time I'll end up losing a fair bit of sleep worrying about it.

I think the first step that we should take here is adding robust benchmarking tools to Aniseed and then benchmarking the various assumed pain points for your config as well as mine. Maybe we'll see interesting differences between our setups!

I'll consider renaming the config module too and putting a warning in the current one. Still not 100% on the name but env is a contender, and I still don't want to go with anything as short as init because I don't think require("init") is unique enough, I fear that it'll collide with something eventually. I could however make this configurable which would probably let you name it whatever you want while allowing me to set it to a default I feel comfortable with.

[kkharji]

I think the first step that we should take here is adding robust benchmarking tools to Aniseed and then benchmarking the various assumed pain points for your config as well as mine. Maybe we'll see interesting differences between our setups!

Looking forward.

I'm not sure if I'd consider that simpler or less sophisticated, I feel like that'll involve more mechanisms and branching conditions. My metric for simplicity is the amount of "if"s and "but"s I have to insert while explaining how it works sweat_smile

Sucks that there isn't any workaround the flash screen on initializing vim, which I'm kinda sure is because work done on VimEnter by aniseed or perhaps the compiling part being at startup. So perhaps either make files compile on save or find a way to NOT do anything remotely related to fnl -> lua VimEnter. Nonetheless I can open another issue for that when we start testing and benchmarking.

I'll consider renaming the config module too and putting a warning in the current one. Still not 100% on the name but env is a contender, and I still don't want to go with anything as short as init because I don't think require("init") is unique enough, I fear that it'll collide with something eventually. I could however make this configurable which would probably let you name it whatever you want while allowing me to set it to a default I feel comfortable with.

Oh even better @Olical !! just let us worry about the naming aspect :D >> heads up I'm going with init :P.

Thanks a lot looking forward.

[Olical]

Hmm you're seeing a flash when you open Neovim? I'm not :thinking: maybe something else is at play... is it a flash of text / an error or just a blank screen for a split second while Neovim opens?

And I'm going to leave this open as a reminder and something for me to close as I get around to addressing it :smiley:

[kkharji]

Oh sorry about that. Yes it is a flash of text. it fairly quick. And during developing with fennel I used to noticed it a lot. However, I think this should be skipped until we have benchmarking mechanism.

[Olical]

Yeah, I've never seen that text, that's not normal, if you could capture it with an https://asciinema.org/ or something that'd be great, it's probably a brief flash of an error?

[Olical]

The develop branch contains a deprecation warning for aniseed.dotfiles + help about aniseed.env and the new code. Let me know what you think :slightly_smiling_face:

[kkharji]

Sure thing, I'm going to start migrating some stuff and test stuff out starting tom, thanks a lot @Olical

[Olical]

I've added a timing macro and it takes 0.5ms to check if anything in my (fairly extensive) Fennel config needs compiling and then 21ms to load all of my config (ignoring regular viml plugins).

Which proves almost all of my startup time which is in the 100-300ms range is my viml plugins. Fun!

[Olical]

Closing since I've addressed all of this on develop and I'm going to release it. You can also base your Fennel source in your root Neovim config directory now, so you can remove the /fnl/ prefix if you really want to because someone asked for that.

[kkharji]

The new changes brought a lot of flexibility 💪🏼. I’m still working with making my lua config more migratable and thus later comparable to fnl for testing the performance difference with time macro you’ve created . Thanks

[kkharji]

@Olical I like the idea of this a lot

lua require('aniseed.env').init({ module = 'dotfiles.init' })

Could we use aniseed.env.init to conditionally load modules?

For example:

au! Colorscheme * lua require('aniseed.env').init({ module = 'global.theme' })

or

au! Filetype lua require('aniseed.env').init({ module = 'ft.lua' })

Thanks

[kkharji]

I've added a timing macro and it takes 0.5ms to check if anything in my (fairly extensive) Fennel config needs compiling and then 21ms to load all of my config (ignoring regular viml plugins).

Hmm 🤔 that interesting, on my current fairly ludicrous setup when I do time nvim -c I get from 0.14 to 0.17. it bugs me a bit that I use to have 0.07 before. Anyways, I’ll try the macro soon and report results

[Olical]

Could we use aniseed.env.init to conditionally load modules?

Yes, but every time you call init it'll check if it should compile the fennel again (which is sub millisecond fast, but still). You could just use a normal lua require like require('global.theme') under an if statement or autocmd etc. That'll work fine, you can rely on Aniseed to compile everything then conditionally require modules you need.