option to bring exported functions to `Main` scope when loading a module via snail?

[orialb]

Hi, I don't know if this fits the scope of this package, but would it be possible to (optionally) bring exported functions to the Main scope in the julia-snail REPL when a module is loaded via snail? i.e. similar to loading the module with using instead of include.

When working on a package I find it convenient to have all the exported functions available in the main scope. I could of course just do using MyModule in the julia-snail REPL, but then I would not get code completion of unexported functions when working inside the module files (I would be able only to get completion for MyModule.myfunc, but when working inside the module I don't need the MyModule prefix).

I don't know how hard it is to have such a functionality within julia-snail, or whether it's a good idea at all.

On a side note, I would love to hear what is the package development workflow that you envision/recommend when working with julia-snail? It might be that I'm just used to a Revise.jl based workflow and don't understand how to properly use julia-snail during package development.

[gcv]

The problem with namespaces you're describing is inherent to the native Julia REPL. In SLIME and Cider, you can switch the current module (or namespace) in the REPL, and this would definitely be a feature in a (future) Elisp Julia REPL implementation.

I think the code to copy things between modules would essentially iterate over all names in the source module and assign them to the same names in the target module. A basic implementation should be straightforward. You can probably add a function which does this to ~/.julia/config/startup.jl and just call it from the Julia REPL. There is no Snail-specific code required (though a Snail wrapper for calling it from a key binding is easy to write).

You can also make a short alias to your package name in Main, so it's easier to type:

julia> import MyPackage
julia> m = MyPackage
julia> m.my_function()

I don't use Revise myself, but I don't see why Snail wouldn't mostly work with it if you like it. Just start Revise in Snail's REPL buffer. You would have to use julia-snail-send-buffer to prime the cache of included modules, but that's about it. You can still use the REPL, and I think xref and completion would work.

In general, I don't see why package development would be any different from other Julia development. Could you give more detail about what you're missing?

I'm also curious about what you mean here:

but then I would not get code completion of unexported functions when working inside the module files (I would be able only to get completion for MyModule.myfunc, but when working inside the module I don't need the MyModule prefix).

Can you please give me a detailed example? You mean you don't get completion of unexported functions of MyModule when working inside MyModule itself? That sounds like a bug.

[orialb]

Thanks for the detailed reply. Sorry for not explaining myself more clearly, let me try with a concrete example. Let me preface by saying that it is a relatively minor thing I'm asking about here, and I could definitely live without this feature at the moment.

Assume I have a module, defined in a file mymodule.jl :

module MyModule

export foo

function foo(x)
...
end

function bar(x)
...
end

include("another_file.jl")
end

I have several options to load the module code in the REPL:

1. What julia-snail does when julia-snail-buffer-file is called is basically include("mymodule.jl") (if I understand correctly). In this case foo and bar would be available in the REPL as MyModule.foo, MyModule.bar.
2. Another option to load the module in the REPL is to do using MyModule (assuming MyModule is properly added to the current active environment). In this case foo would be available in the Main scope (i.e one could call foo(x) in the REPL). bar would not be in the Main scope, so one still has to call it with MyModule.bar.

Basically what I'm asking is whether it is possible/makes sense to optionally get behavior similar to scenario 2 above when loading a module via a julia-snail command.

Can you please give me a detailed example? You mean you don't get completion of unexported functions of MyModule when working inside MyModule itself? That sounds like a bug.

I don't think there is a bug here. What I meant is the following scenario:

1. I load the module in the REPL via using MyModule instead of using julia-snail-send-buffer-file.
2. When working inside another_file.jl, snail would not be aware that it is part of the module MyModule, it will think it's part of the Main scope.
3. In that case code completion for bar would not work inside another_file.jl (because snail is unaware that another_file.jl is inside MyModule).

In general, I don't see why package development would be any different from other Julia development. Could you give more detail about what you're missing?

Actually thinking about it a bit more I don't think I'm missing anything crucial. I take that question back.

Hope it helps to clarify what I meant.

[gcv]

Ah, okay, I think I understand.

What about trying both using MyModule and julia-snail-send-buffer-file? I think that'll work, you'll get both Snail completion and the effects of using in the REPL.

I didn't try it, though, so I'm not sure what happens when you update the code. You'll have to run julia-snail-send-buffer-file for code changes to get picked up, but I don't know if it'll update all using references in Main (the REPL module). It might work for existing functions, but not pick up newly-defined ones. Not sure. This might be a good use case for Revise, since — if I understand it correctly — you'll only have to use julia-snail-send-buffer-file once per file to prime Snail's module caches, but Revise will take care of updating the code in the running Julia image.

[orialb]

It seems that it doesn't work so well to do both using and julia-snail-send-buffer-file (i.e. includeing the file).

If I first do julia-snail-send-buffer-file and then try using the module I get an error that the module is already defined in Main.

If I first do using MyModule and then julia-snail-send-buffer-file , first of all I get WARNING: replacing module MyModule, then:

1. without Revise: The exported functions (foo) are still available in the Main scope, but if I change foo and run julia-snail-send-top-level-form the changes are not propagated to the Main scope (i.e. calling foo() will use the older definition, calling MyModule.foo() will use the new definition)
2. with Revise: After I do a change to foo and try to call it I get a MethodError: no method matching foo(), so it seems that the fact that the module was loaded with an include after using is interfering with Revise.

I am trying to write a function similar to julia-snail-send-buffer-file but with

(julia-snail--send-to-server
        :Main
        (format "using %s" module)
...

instead of the code at l822. I couldn't get it working yet, but I'll keep trying. Not sure why it is not working because running JuliaSnail.eval_in_module([:Main],:(using MyModule)) in the REPL seems to work.

[orialb]

Ok, the following function (which is just a small modifiction to julia-send-buffer-file) seems to be able to load the module defined in the buffer with using. It might not work with nested modules. Updating an exported function with julia-send-top-level-form seems to also work. It also seems to work with Revise, as long as julia-send-top-level-form is not called (i.e. julia-send-top-level-form breaks Revise code tracking somehow). This is not really an issue because if one uses Revise there is no need to manually update function definitions.

(defun julia-snail-load-buffer-module ()
  (interactive)
  (let* ((jsrb-save julia-snail-repl-buffer) ; save for callback context
         (filename (expand-file-name buffer-file-name))
         (module (julia-snail--module-at-point))
         (includes (julia-snail-parser-includes (current-buffer))))
    (unless (equal module '("Main"))
        (julia-snail--send-to-server
         '("Main")
         (format "using %s" (car module))
       ; rest of the function identical to julia-snail-send-buffer-file

Would having such a function/functionality as part of the package make sense?

[gcv]

To clarify: the intent of the function you propose is to add a Snail command which executes using MyModule?

[orialb]

Yes, a command that loads the module with using and in addition loads everything that is needed into the julia-snail cache such that it can recognize all other included files as being part of MyModule to provide code completion and xref functionality in the correct scope.

[gcv]

I'm not entirely comfortable with a standard Snail command which changes namespaces other than the ones in the file currently being edited (i.e., in this case, it'll change Main even though you're editing MyModule).

But, I think that a command which updates the module caches without julia-snail-send-buffer-file makes sense. Take a look at a potential implementation on this branch: https://github.com/gcv/julia-snail/tree/refresh-module-cache

You have to manually run using MyModules to use it and get the effect you want, but I think it addresses a more general problem, that of module caches depending on julia-snail-send-buffer-file.

What do you think?

[orialb]

Thanks for working on this. If having a command that does using doesn't fits Snail's philosophy, then a refresh-module-cache feature seems to indeed be the best solution.
I could then easily write a small private utility function that will just call julia-snail-refresh-module-cache + using MyModule.

[gcv]

Cool.

I changed the name of the function to julia-snail-update-module-cache, changed its default keybinding (to one that actually works and doesn't conflict with anything), and added documentation. The change should be in MELPA now, but you might have to restart Emacs after updating julia-snail to pick up the keymap change. Let me know what you think.

[orialb]

looks good, thanks a lot!

I didn't see any keybinding for this new command after restarting emacs, but I am using Spacemacs so maybe something conflicts there. I am anyway defining my own custom keybindings so if the keybinding works for you we can close the issue