virtualenv support

[gaborbernat]

Someone who knows and understands elvish should add virtual environment activation script support - per https://github.com/pypa/virtualenv/issues/1775.

[zzamboni]

I know Elvish well, but I don't code Python regularly so I don't fully understand how virtualenv works. @gaborbernat is there documentation somewhere about how to write a new activator? And how to test if it's working correctly? I looked a bit over the existing activators at https://github.com/pypa/virtualenv/tree/master/src/virtualenv/activation and it doesn't seem too complicated.

[iwoloschin]

I've written an elvish module that provides virtualenv support. you make a virtualenv like normal, but instead of sourcing an activate file somewhere inside of the virtual env, it uses an elvish module to alter the environmental variables.

https://github.com/iwoloschin/elvish-packages/blob/master/python.elv

I'm not sure if you could flip this around to the normal virtualenv activation method of sourcing a script, does elvish support that? I think this is why I wound up making a module in the first place.

[iwoloschin]

Ok, with a couple of quick, half caffeinated tests I think you could do this, but it'd require using elvish's -source command. I'm not sure if using -source for something like this is recommended, any thoughts @xiaq or @zzamboni?

[zzamboni]

@iwoloschin right - I remembered seeing something about virtualenv somewhere, and it was your module indeed which I had seen. I agree that using -source is not recommended, since it's an "unsupported" function (that's why the name starts with -).

I think the Elvish way of doing this would be to specify something like the following as the commands to run (instead of the default source bin/activate - I assume the specifics of activation might already differ slightly between shells):

epm:install &silent-if-installed https://github.com/iwoloschin/elvish-packages
use github.com/iwoloschin/elvish-packages/virtualenv
virtualenv:activate

If/once the module gets merged into upstream virtualenv, the URLs would change accordingly.

@gaborbernat @xiaq @iwoloschin thoughts?

[iwoloschin]

I think the big difference here is that most (all?) other shells have some ability to source a file and make changes to the running scope. Elvish does have this ability as well, but it is "unsupported" and it isn't entirely clear to me why it is unsupported as it's been available for as long as I can remember (for instance, when I first asked about this in #582!).

In any case, a short term change might be adding some instructions to elv.sh that explain how to use a python virtual environment, basically what you posted above with a few changes to make it handle more than just my use case of "all virtualenvs in a single directory". That documentation would probably go somewhere under Learn?

Long term I do think we should get a better understanding of why -source is unsupported.

In the meantime, I'm not sure if we could easily upstream my module. In bash (and other shells) the activate tool is called from within the project, source ~/.virtualenvs/project/bin/activate. In elvish, without -source we have a "global" activation tool, virtualenv:activate project. I feel like this doesn't really fit into how virtualenv works, @gaborbernat any comment here?

[gaborbernat]

virtualenv (and similar venv) does not really require source ability. Here's the contract that's expected to be supported; the ability to (usually by having some activator script inside the virtual environment the user can call):

• shim the executable within the virtual environments script and binary folder in front of the PATH executables, (e.g. post the activation typing python should refer to the pyhton executable within the virtual environment, not to the executable it resolved before),
• alter the prompt of the shell to display the virtual environment prompt value (post-activation),
• deactivate both the prompt and the shimming mechanism above by typing in the deactivate command.

I don't know elvish enough to tell how it's best to support these, but an elvish virtual environment activator needs to provide these contracts.

[rdw20170120]

I need this capability, to customize my Elvish shell to configure my PATH, to activate a Python virtual environment, to configure other tools, etc. If Elvish intentionally lacks a source command or equivalent, then how is such shell session configuration supposed to be achieved.

Please note that my (and many other's) use case involves putting all the relevant shell configuration into a code repository as part of a project. Therefore, it MUST NOT involve modifying the user's configuration (in XDG directories and files).

Can this be done in Elvish, or must I look for another shell?

[krader1961]

@rdw20170120, It wasn't necessary to spam six issues with your comment. A single relevant issue, like this one, is sufficient to get feedback. :smile:

Let me start by saying I use Conda. I wrote my own, trivial, con and coff Elvish functions to activate and deactivate a Conda environment. This is not hard to do and does not require anything like the POSIX source command.

Can this be done in Elvish, or must I look for another shell?

It definitely can be done in Elvish and can be implemented multiple ways without a source command. The solutions differ in how idiomatic they are in an Elvish context. The closest in spirit to using source is to do something like eval (slurp </path/to/setup-script). More idiomatic is to use /path/to/setup-script; where /path/to/setup-script is actually the file /path/to/setup-script.elv. Both solutions will want to use the edit:add-var command to insert the newly defined interactive commands (i.e., functions) into the interactive REPL namespace. The eval based solution should only be used if you have to depend on running a command that output Elvish source rather than having a static setup-script.elv file.

[krader1961]

Also, @rdw20170120, I am perplexed by this statement you made:

Therefore, it MUST NOT involve modifying the user's configuration (in XDG directories and files).

Unless you expect the user to run the virtualenv initialization commands interactively they have to, by definition, modify at least the ~/.config/elvish/rc.elv script which is part of the user's Elvish configuration (which, with recent changes, is XDG compliant). Why is that acceptable in light of your stated constraint? I'm guessing you mean that it should not be necessary to add files to a directory such as ~/.config/elvish/lib but modifying the primary interactive configuration script (i.e., ~/.config/elvish/rc.elv) is acceptable. But if that is true you can also simply document that the user should copy/paste a sequence of lines into their rc.elv file that is no more complicated that a single use or eval statement.

[zzamboni]

Let me start by saying I use Conda. I wrote my own, trivial, con and coff Elvish functions to activate and deactivate a Conda environment. This is not hard to do and does not require anything like the POSIX source command.

I also wrote full Elvish support for conda, although it has not been merged yet: https://github.com/conda/conda/pull/10731

[rdw20170120]

@krader1961, I am certainly picking up on Elvish's built-in assumption that configuration MUST be done within ~/.config/elvish/rc.elv. This is very unfortunate, and probably makes Elvish unusable for me and many others. It is quite fair to ask for explanation, so please allow me to provide that.

I work (among other things) primarily as a DevSecOps architect and engineer. My goal is secure and effective site reliability engineering as Google defines it. This constrains my possible solutions in many ways. For one, I must assume that my solutions/projects MUST coexist with others.

Given that, I routinely build projects (my own and for clients) that consist of significant shell scripting in a source-controlled repository. It is essential that the solution be self-contained in that repository. It MUST NOT require capturing the user's personal account configuration (dot files) into the repository--for so many practical as well as security concerns. Attempts to automatically modify the user's personal account configuration tend to likewise be unacceptable, and tend to be difficult to do reliably even if they might be acceptable.

The classic (counter)example is the behavior of many tools to append to the user's ~/.profile script in order to do some essential modification to PATH, etc. Many and various ways have been attempted over the years to accomplish deployment of a tool or solution with such configuration modifications, and they deliver a universally bad experience. Solving this challenge is a crucial part of my primary project.

I am selecting a set of tools to provide a good SRE/DevOps automation experience, packaged into a monorepo that can be easily deployed and used. Therefore, my project's design constraints include that it MUST NOT automatically modify the user's personal account configuration, nor must it require any significant (error-prone) manual reconfiguration by the user.

I have been easily able to accomplish this with Bash, Fish, and Xonsh using the source command. Unfortunately, Bash is a nightmare to use in general, and so I have finally abandoned it. Now I seek a suitable replacement. I tried Fish over the last several months, but was likewise forced to abandon it. Fish is a huge improvement, and I love the syntax, but the always-on autosuggestions, its many bugs, and the unfriendly maintainers force me to find an alternative. I also made a quick foray with Xonsh, but its performance is prohibitively slow. I have been researching many other shells, and Elvish has risen as my next best contender.

My use case depends upon a simple approach to safely coexist with a user's personal configuration and other projects that they may be using/developing. ALL of my solution's configuration MUST be contained within the repository, and it is ONLY active when the user chooses it to be, within a single (or multiple) shell session(s). The workflow is this.

1. User opens a new terminal using their login shell (presumably and preferrably Bash).
2. Jump to the root directory of the cloned copy of my project: cd ~/.../ROOT_OF_REPO.
3. Invoke the target shell for the relevant generation of my project:
4. In that past, that was Bash. So just activate the project: source activate.bash.
5. For my previous generation g0, it was Fish: fish. Activate with source activate.fish.
6. For my next generation g1, I attempted Xonsh: xonsh. Activate with source activate.xsh
7. Now, I am trying Elvish: elvish. Activate with use ./activate.
8. Work...edit code, compile, run tests, deploy, etc.
9. When finished working in this session, exit.

I have spent over a week avidly researching EVERYTHING I could learn about Elvish, attempting to kickstart a new generation of my solution. I love most all that I see about Elvish. However, I got stuck. Or, rather, nothing that I tried would "stick". I wrote a bunch of Elvish script, but my changes to the environment did not affect the current shell environment.

Late last night I was finally able to piece together enough about use to figure out that ONLY use ./activate would work to properly invoke my activation script. To be frank, I HATE that syntax and semantics. I would recommend use ./activate.elv at least. I believe that source activate.elv would be best. Nonetheless, that got me far enough to actually change an environment variable and have it "stick" in my current shell session.

In the process, I learned that I apparently have only two options for where to put my "modules". Either I put my modules in ~/.config/elvish/lib, which violates my design constraints, or I put them in my project repository (where they belong) but I have to use explicit relative path references in every use statement. This leaves me feeling that using Elvish would be prohibitively fragile, ugly, and labor-intensive.

I love that Fish allows me to put my functions wherever I want, and then just reference the directories on the fish_function_path. My solution already has two such directories in my generation g0, one for functions intended for interactive use and one for functions that are generally useful (noninteractive).

I acknowledge that Elvish is young. I acknowledge that it takes an IMMENSE amount of work to create such beauty and THEN make it easily presentable. I am happy to help with that. To start, I think that Elvish desperately needs tutorials focused on real use cases rather than just language features. Almost no one would ever put as much effort into using Elvish as I have over the past few weeks; they would just exercise some good sense and give up. :^)

But, as it is, I am feeling that the current assumptions and vision for Elvish excludes my use cases. I encountered that with Fish, Xonsh, and many other shells as well. I am trying to find the perfect shell for truly engineered (SRE) automation. It has already occurred to me that I just might have to create it. I think that Elvish could be on the right track.

Thanks for your time, @krader1961. I read a little about your background, and I think you probably comprehend my perspective. I am not trying to be insulting to anyone, especially not Elvish and your hard work. I am just trying to provide open and honest feedback about my struggles, to answer your questions and to hope that you can help answer mine.

[rdw20170120]

Oh, to address the other comment from @krader1961: I was not spamming. I looked through dozens and dozens of issues in my week-long search for answers about how to use Elvish. I commented on what I found to be the minimal set of issues that I was directly experiencing. Each issue is about a different aspect, or at least another significant nuance, of my experience being stuck with Elvish. I was not being a noob trying to get attention.

[zzamboni]

@rdw20170120 as you discovered, you can easily achieve #7 by storing your code in an activate.elv file in your repository, and then running use ./activate from within the directory. I'm not sure I understand why you find this syntax unacceptable - I find it no worse than source activate.fish. Can you clarify?

Note that you could also easily implement a fish-like "functions path" with Elvish, with a module that loads all the files found in a particular directory. This is how my alias module works (in a sense, the alias directory is an implementation of Fish's functions directory), so you could use that as an inspiration - look at the -load-alias and init functions, which do most of the work.

If you write a module for this, it would be a welcome contribution to the community!

[rdw20170120]

@zzamboni, you are quite right to ask for clarification about why I do not like the use ./activate syntax. The reasoning is simple: it appears magical instead of easily discoverable.

The documentation says that this usage of use requires a relative path starting with ./ or ../. That is fine. So then I expect to write use ./activate.elv because THAT is the actual relative path. But ./activate is NOT a relative path; it seems magical because the file suffix is being assumed. It takes away from me the right to choose my file suffix, or to have no suffix at all (though I personally like the .elv suffix). More importantly, this is an undocumented nuance that tripped me up for about a week.

I actually resolved this issue for myself by hunting down your "dot-elvish" project, Diego. I read through your rc.elv configuration script, line-by-line, until I understood what it was doing. For each use statement, I was eventually able to guess that the .elv file suffix was assumed by tracing the connection manually to the actual module files being loaded (from several other GitHub repositories). It simply should NOT be necessary for a user to go to such lengths to figure out how Elvish works, especially for such a fundamental feature as modules.

Also, use does not seem to work with an absolute path, which makes no sense to me. One of my project's fundamental design criteria is to AVOID relative paths: evil lies down that way. I go to a lot of trouble to make ALL my paths absolute, and to provide user feedback consistently in such absolute terms. What is magical about relative paths, such that they should work with use but absolute paths cannot? It is my understanding that the modules are cached by their absolute paths anyway, right?

I am very concerned about how use seems to work with GitHub repositories as well. I have not tested it yet, but I eventually hope to do so. What I see so far makes me wonder whether use will work with a private GitHub repository that I access via a custom SSH configuration in ~/.ssh/config.

As a quick aside, could we please have Elvish cache the loaded modules for a short-but-reasonable timeout rather than forever, like Fish? I love that feature of Fish, because it allows me to work on a module (function), then test it after a few seconds when the old definition has been dropped from the cache. I suggest a timeout of about five seconds. As it is, I simply exit my Elvish shell, then start a new one with elvish. That clears the cache rather well. I do this anyway with bigger changes to ensure that my testing is valid. :)

I much prefer how Python or even Fish handle source lookups. Python uses an explicit module search path with a very-well-documented algorithm for how a module name is matched up to the actual file. Likewise, Fish uses an explicit function search path and a well-documented algorithm for how a function name is matched up to the actual file. In contrast, this lookup behavior of Elvish seems unnecessarily obtuse and magical. Now, I am all for magical--in the right context. But this kind of magic in my shell is not welcome.

I cannot help but wonder how many potential Elvish users we have lost because they got stuck on this and similar issues, just as I am witnessing many Elvish users here that are lost to Fish because of its misbehaviors. I am not trying to be insulting by any means--I am just trying to think out loud about the user experience for our potential audience. Modules are a crucial feature of Elvish that can be made much more approachable to the newbie and the expert alike.

Yes, Diego, you are right to point out that this can be added. I have noted your alias module, and I will dig into it. It seems to me though that this functionality deserves (eventually) to be a well-designed and well-integrated part of Elvish. Maybe I am destined to be the person that adds it, and maybe I just volunteered. :) As I said before, I fully acknowledge that Elvish is young and there is much work to do. This is particularly true because Elvish is addressing the whole realm of scripting from a very different perspective. We are on a very UN-beaten path, perhaps even on holy ground.

A related issue to this is that it seems difficult to discover what parts of Elvish are suitable for such tinkering, and which are not. I see mention repeatedly in the documentation and in these issues of "private" implementation functionality, and I see mention of the fact that many things are still not discoverable through exposed APIs (e.g., list of namespaces, etc.). I also see many mentions of the fact that the documentation and the implementation are not in sync. Maybe that is another thing that I can help with, since I tend to be rather rear-end-retentive about such things. :)

I realize that I wrote a lot here, and that it probably belongs in several distinct issues. I am happy to go create those separate issues, as appropriate. Maybe all of this needs to be moved to some kind of "approachability" issue. I think that Elvish is WAY too cool to let it be missed because of approachability.

[zzamboni]

I am very concerned about how use seems to work with GitHub repositories as well. I have not tested it yet, but I eventually hope to do so. What I see so far makes me wonder whether use will work with a private GitHub repository that I access via a custom SSH configuration in ~/.ssh/config.

use does not work with GitHub repositories. What you have seen is a naming convention used by epm, see https://elv.sh/ref/epm.html.

epm can install modules from arbitrary locations, but it knows out-of-the-box how to fetch modules from GitHub, GitLab and BitBucket. It then installs them under the corresponding path, including the host, so they must be loaded like that, e.g.:

epm:install github.com/zzamboni/elvish-modules
use github.com/zzamboni/elvish-modules/util

(I'll reply to some of your other points in the morning)

[krader1961]

@rdw20170120, You wrote

The classic (counter)example is the behavior of many tools to append to the user's ~/.profile script in order to do some essential modification to PATH, etc.

and

I have been easily able to accomplish this with Bash, Fish, and Xonsh using the source command.

I agree that installing a tool should not automatically modify my shell's interactive configuration scripts. I hate tools which do so and avoid them if possible. What you seem to be saying is that you expect users of your tool to interactively execute a source command. As I wrote in my previous comment that is certainly possible but seems to me to be suboptimal. Perhaps you mean that it is up to the user to add the relevant source command (or Elvish equivalent) to their shell's interactive config file. I am confused since you seem to be making mutually exclusive statements regarding how you solve this conundrum with other shells and how you expect it to work in the Elvish shell. It seems to me trivial to document adding the relevant use statement to the Elvish ~/.config/elvish/rc.elv file in your tool's documentation.

[gaborbernat]

you are quite right to ask for clarification about why I do not like the use ./activate syntax. The reasoning is simple: it appears magical instead of easily discoverable.

I'm the maintainer/author of the virtualenv project. Let me say that I think the entire concept of activation/deactivation is error-prone and magical, especially for new users. I personally would remove it, and force people to invoke commands via a relative (or absolute) path... But alas is too widely used to do that. If you're looking for less magic, just don't use an activation script. In the spirit of explicit is better than implicit: an explicit path is better than an implicitly resolved one.

[rdw20170120]

@zzamboni, thanks for clarifying the interaction and conventions of use and epm:install. I missed that completely, but I had not reached the point of diving into it yet. However, I have been left with the impression that epm is not really a usable and "released" tool. Is it still experimental?

@krader1961, yes, I expect a user of my tool to invoke a quick series of commands in order to use my "tool" or project, with effect ONLY in that specific shell session. Actually, my "tool" starts as simply a set of conventions for how to resolve this exact issue: how do I use multiple source repositories, projects, and tools in such a way that maintains proper isolation, encourages good practices, gives reliable behavior, and scales? My starting convention is that no part of my repository, project, or "tool" requires making any significant changes to the user's account and/or machine outside the cloned repository. So I have no intention of asking my users to make any changes to their dot files. Specific to Elvish, I constrain my design to require no changes within ~/.config/elvish (or any other part of the XDG filesystem conventions). They may opt to do so on their own, for their convenience, but I will never require it. My "tool" may require installation of Elvish, but I will not constrain their use of it beyond my activation script and its conventions.

Instead, my typical user would clone my repository, or clone their own repository into which they have injected my "tool". For a work session, they would open a Bash shell, change directory to the root of that repository clone, invoke elvish, invoke use ./activate, then perform whatever tasks they may have for this work session using the full power of my "tool" and their project. When they are done, they simply exit the shell.

@gaborbernat, I certainly do not mean any of this to negatively reflect on your virtualenv support for Elvish. I have not used it yet, and frankly I probably never will. We'll see. I likewise HATE the typical activation scripts that I have seen over the four decades of my career. Such scripts tend to be awful, error-prone, do too little, or do too much. When I use tools that have such activation scripts, I read through them, identify the relevant steps for my situation, then incorporate those steps into my own project scripts. I never use such an activation script directly. My own scripts do exactly and only what I need them to do: exactly how, when, and where I need them to do it. I do exhaustive error-checking, so my scripts fail-fast if the conditions are not right.

My "tool" is focused on providing that capability to EVERYONE. I have written a set of scripts, conventions, and supporting functionality to make it easy for any user to do the same with any project. So my activation script is actually their activation script. I provide a kickstart, a strategy, that solves most of the issues that routinely make us hate activation scripts. I also provide functionality and guidance for the user to make it their own: specific to them, their project, and their tools. In other words, my "tool" is a framework for building custom activation for any project, incorporating any scripts, tools, and applications the user deems appropriate. It is designed to be readable and accessible to everyone, presuming some programming knowledge. Hence, minimal magic...

My ultimate intent is to provide a reliable context for enterprise-scale reliable development and infrastructure automation. Therefore, it must coexist peacefully with other projects, tools, and configuration that the user may also be pursuing.

I do not think that the awfulness of activation scripts reflects upon their authors. I think this is an inherent consequence of the challenge of writing an activation script that fits (almost) everyone in (almost) every situation. For the record, I am glad that people write such activation scripts for their tools, because the scripts serve as a crucial complement to whatever documentation the author may have provided. I prefer to go through the documentation, then peruse their script as I write my own. I also think that such activation scripts are an essential way for tool maintainers to stay in touch with the experiences of their users. The simple fact is that most users are going to be too lazy to do anything but use the provided activation script.

I hope that helps provide more context for my use case. I have considered the possibility of eliminating activation or making it transparent/automatic, but I likewise strongly prefer explicit versus implicit. I have a couple of major strategies still to mix in, but my approach has been very successful for me over the last many years and numerous clients. We will see how it plays out.

[krader1961]

@zzamboni, thanks for clarifying the interaction and conventions of use and epm:install. I missed that completely, but I had not reached the point of diving into it yet. However, I have been left with the impression that epm is not really a usable and "released" tool. Is it still experimental?

It is not experimental. It has been a core part of Elvish since it was merged as a builtin module four years ago (see commit f4878bbfb7). It has certainly undergone many revisions since then but it's not going away and is an integral part of Elvish.

So I have no intention of asking my users to make any changes to their dot files. Specific to Elvish, I constrain my design to require no changes within ~/.config/elvish (or any other part of the XDG filesystem conventions). They may opt to do so on their own, for their convenience, but I will never require it.

That's fine as I noted in my reply to your comment in an issue about the desirability of a source command. A user can add to their Elvish interactive config scripts any commands you expect the user to run manually. Illustrating why I felt you should have focused your questions in one or two issues, or possibly opening a new issue since you are really asking for advice about using Elvish that spans a lot of ground. In particular your most recent comment suggests you are trying to write a meta-import tool that works with multiple shells.

[zzamboni]

@rdw20170120 commenting on a few of your points throughout your thread. My apologies, but I feel that you are commenting on a wide variety of issues so it's hard for me to distinguish whether you are asking for help on how to do things, commenting on the design of Elvish, or just discussing philosophically. You may want to open separate issues for specific topics, or jump into the channel for asking for help on specific things.

Anyway, a few things:

• epm is not experimental, it's been with Elvish for a long time.

• You said:

  But ./activate is NOT a relative path; it seems magical because the file suffix is being assumed.

  Most import/use/require modules assume the file suffix. E.g. in Python you use import module, not import module.py.

• You said:

  could we please have Elvish cache the loaded modules for a short-but-reasonable timeout rather than forever, like Fish?

  I believe this has been discussed, there may even be an issue about it. @xiaq might be able to provide a specific pointer. This behavior is documented though: https://elv.sh/ref/language.html#re-importing. What I do is simply open a new shell when I want to test changes. exec elvish also works.

• You said:

  In contrast, this lookup behavior of Elvish seems unnecessarily obtuse and magical.

  The lookup mechanism is documented: https://elv.sh/ref/command.html#module-search-directories. It does seem however that the documentation at https://elv.sh/ref/language.html#user-defined-modules should be updated to reflect the new XDG implementation.

I'm happy that you are exploring Elvish! I think you'll find the developer (@xiaq) and the community are welcoming of constructive feedback. I don't have as much time these days due to my job, but I'm happy to help as much as I can.

[rdw20170120]

@krader1961 has rightly suggested that I open a new issue for my various questions.

[rdw20170120]

As I read literally everything I could find about Elvish, I found no significant discussion or presentation of EPM. I still have not. What few mentions I did find seemed to be pre-release.

[rdw20170120]

@zzamboni: Yes, there is some mention that Elvish will search for modules, but I feel that it could be so much better. I have read literally everything I can find on Elvish, and I never encountered a specific mention of .elv as an actual convention. It is certainly not prominent in the brief discussion of module search.

Note that Python's module search functionality is based on a module packaging and naming convention that is specifically distinct from relative paths. In Python, one does not ever (for example) import ./a/b/c/d/module.py or even import ./a/b/c/d/module. Instead, one will import .a.b.c.d.module. The syntax is intentionally different, and the mapping between them is well documented.

I found the Elvish use ./module syntax to be misleading because it is a hybrid. It is not really a relative path, but it is also not a distinct module/package naming convention. Whatever it is or may become, I suggest that it would benefit from additional discussion and actual real-world examples.

As for cacheing modules, I stand by my request that it be time-limited. Cacheing is a crucial tool to improve performance and scalability, but it can also destroy performance and scalability. Cacheing all modules for all time will, pure-and-simple, make Elvish unusable for enterprise-class solutions because it will become a problem as numerous modules begin consuming more and more RAM and CPU during a day-long or months-long shell session. Forcing the user to abandon their shell session is simply unacceptable.

I am realizing that there probably already exist separate issues for these specific concerns, so I will try to move the discussion there.

[krader1961]

@rdw20170120

As I read literally everything I could find about Elvish, I found no significant discussion or presentation of EPM. I still have not. What few mentions I did find seemed to be pre-release.

Have you seen the official documentation? See https://elv.sh/ref/epm.html.

Note that Python's module search functionality....

That is not relevant since Elvish is not Python.

I found the Elvish use ./module syntax to be misleading because it is a hybrid. It is not really a relative path....

It absolutely is a relative path in the normal sense of the term. Also, ../module works. Could this be better documented? Probably, since it is rare that documentation of a topic or feature can't be improved.

As for cacheing modules, I stand by my request that it be time-limited.

No. Please, god, no. I do not want my shell instance to randomly change its behavior because a timer expired and caused the shell to reload an external module. I say that as a user of Fish for several years who did appreciate that feature when experimenting by editing files in my personal ~/.config/fish/functions directory. I now view that "feature" to be a misfeature. Regardless, given the design of Elvish it isn't practical to implement that behavior except in a manner that would be even more confusing than it is in Fish.

[xiaq]

Not going to respond to every issue discussed here - the correct mechanism for implementing virtualenv support would be asking the user to eval a file, not use a module.

Modules are mostly a code reuse mechanism and its side effects are intentionally only executed once.

Also, getting back to the original question of adding virtualenv support: since Elvish's language is not considered stable now, it'd make the most sense for Elvish to bundle an implementation for the activate script. This would only be feasible if the setup needed for virtualenv doesn't vary between versions, though.

[rdw20170120]

By the way, I finally stumbled upon the documentation for the .elv naming convention for modules. So it took me somewhere around two-to-three weeks of intensive study and attempted use to uncover than convention. I am likewise finding that to be the case with many other nuances of Elvish syntax and usage. Right now, I am stumped about how to reliably convert between lists and strings in non-trivial cases.

[zzamboni]

By the way, I finally stumbled upon the documentation for the .elv naming convention for modules. So it took me somewhere around two-to-three weeks of intensive study and attempted use to uncover than convention.

@rdw20170120 I'm not sure to which documentation you are referring, but the use of .elv is literally everywhere in the Elvish documentation, so it's pretty easy to pick up.

E.g., literally the first thing in the "User-defined modules" section:

You can define your own modules in Elvish by putting them under ~/.elvish/lib and giving them a .elv extension

Also in the Quick Tour:

You can define your own modules by putting .elv files in ~/.elvish/lib.

And basically every example in the documentation of an Elvish file, they all end in .elv. I'm not sure how this could have been more apparent.

Right now, I am stumped about how to reliably convert between lists and strings in non-trivial cases.

Can you give an example of what you are trying to do? Some example conversions to maybe get you started (BTW these should work on arbitrarily complex lists):

> l = [foo bar baz]
> to-string $l
# Convert using Elvish syntax
▶ '[foo bar baz]'
> eval "put "(to-string $l)
▶ [foo bar baz]
# Conversion using JSON
> to-json [$l]
["foo","bar","baz"]
> to-json [$l] | from-json
▶ [foo bar baz]

Also, as I've said before, I think these usage questions are much better resolved in the chat than in the issues.

[rdw20170120]

@zzamboni, I responded in a new issue #1406.

[tesujimath]

bash-env-elvish may help. I initially wrote a bash-env plugin for Nushell, then ported it for Elvish.

This is a Bash script bash-env-elvish and Elvish modules bash-env and virtualenv for:

• importing Bash environment into Elvish
• extracting Bash style shell variables from source files like /etc/os-release
• activating/deactivating Python virtualenv

Source files may be arbitrarily complex Bash, including conditionals, etc.

I'm pretty new to Elvish, so there may be a nicer way to integrate this. Suggestions welcome.

(Cross posted in #205, since relevant to both)