Not actually calling `fdfind`

[watsonb]

Greetings,

I love the concept of this plugin and desperately want to get it working (I think it worked once!). It seems to me that when I run VenvSelect or use my keybind <leader>vs, I see an empty Telescope pop up. If I turn on VenvSelect debugging, I no longer see the Telescope pop up. Regardless of debugging status, when looking at tools like top and htop, I never see fdfind being run as a result of using VenvSelect.

Here is my current plugin config:

return {
  "linux-cultist/venv-selector.nvim",
  opts = {
    auto_refresh = true,
    fd_binary_name = "fdfind",
    name = "venvtest",
    dap_enabled = true,
    path = "~/workspace",
    parents = 1,
    enable_debug_output = true,
    search_venv_managers = false,
    search_workspace = false,
    search = true
  }
}

I have a test Python venv created as follows: python3 -m venv ~/venvtest

I'm working on a project in the following path: /home/ben/workspace/kiewit/ansible/playbooks/ap_linux_instance

Here is the VenvSelect debugging when running the VenvSelect command:

VenvSelect: 
{
  anaconda = {},
  anaconda_base_path = "",
  anaconda_envs_path = "/home/ben/.conda/envs",
  auto_refresh = true,
  cache_dir = "/home/ben/.cache/venv-selector/",
  cache_file = "/home/ben/.cache/venv-selector/venvs.json",
  changed_venv_hooks = { <function 1>, <function 2>, <function 3> },
  dap_enabled = true,
  enable_debug_output = true,
  fd_binary_name = "fdfind",
  hatch_path = "~/.local/share/hatch/env/virtual",
  name = "venvtest",
  notify_user_on_activate = true,
  parents = 1,
  path = "~/workspace",
  pipenv_path = "~/.local/share/virtualenvs",
  poetry_path = "~/.cache/pypoetry/virtualenvs",
  pyenv_path = "~/.pyenv/versions",
  search = true,
  search_venv_managers = false,
  search_workspace = false,
  venvwrapper_path = "~/.virtualenvs"
}
VenvSelect: Setting fd binary to 'fdfind' since it was found on system and requested by user instead of fd.
VenvSelect: Using path from settings path: ~/workspace
VenvSelect: Looking for parent venvs in '~' using the following parameters:
VenvSelect: 
{ "--absolute-path", "--color", "never", "-E", "/proc", "-HItd", "^venvtest$", "~" }
VenvSelect: There are 0 results to show.

This all happens very fast (almost too fast IMHO), I see no fdfind activity in top/htop.

However, if manually run fdfind as follows:

→ time fdfind --absolute-path --color "never" -E "/proc" -HItd "(^venvtest)" ~
/home/ben/venvtest

real    0m20.835s
user    0m9.728s
sys     0m17.971s

I get the result I expect and it does take some time (I also see several parallel fdfind processes in top/htop).

Any idea what I'm doing wrong here? I'm using neovim v0.9.2 using a customized/tuned LazyVim.org configuration, running on Ubuntu 22.04 with fd 8.3.1 (installed as fdfind).

[linux-cultist]

Hi and thanks for a very detailed post!

I will have a look this weekend but my first thought is that maybe your path is not being resolved. I see that VenvSelect still puts the path as ~/dir in the debug output and I was expecting it to resolve that to /home/youruser/dir.

Try putting the full path without the tilde there and see if it makes a difference, but otherwise I will try your settings on my machine this weekend. We will figure this out. :)

[linux-cultist]

And like you say, it could be that it's not running fdfind also, maybe some bug related to using fdfind instead of fd. I will have a look. :) The debug output doesn't mention what fd it's actually trying to run, so I will add that too...

[watsonb]

I appreciate your help. I'm trying to avoid using absolute paths where possible, hence the tilde, to keep the config generic and working on several machines. Also, I tend to lump all of my venvs directly in my home directory (probably a long overdue bad practice on my part) and not nested in a commonly named sub-folder.

I'll experiment with the absolute path and with nesting my venvs in a more formal file-system structure.

[watsonb]

Well we have some progress to report. Simply changing the path from ~/workspace to /home/ben/workspace seems to have fixed it. I definitely see fdfind being run in the background and the telescope list eventually populates with items.

Perhaps my use-case is different/strange compared to other virtual environment managers out there (I personally never got into them). I grew up using virtualenv <path>. When Python shipped a built-in venv module, I switched to python -m venv <path>. However, my <path> has historically been ~/venv_<context>, where <context> is some naming clue/convention to remind me what's inside that particular venv and what it is used for.

Yes, this does leave my home directory littered with several venv_* folders, but it has worked for me for many years. Anyhow, I could probably find a "more standard" place to make these, but this will likely lead me to another issue. Given I name my environments fairly consistently venv_<context>, would it be possible for VenvSelect to search a folder for all folders starting with a common prefix? I notice your plugin feeds a regex to fd as follows "(^venv$ | ^env$)". Perhaps removing the trailing dollar sign ($) would allow for more flexibility (e.g. "starts-with") rather than me having to explicitly list all of the specific venv_<context> things to search for.

I know I'm adding another issue to this issue, so we can relax the regex talk for another time. As for the original tilde issue, is there any way to support this? I'd rather not have to configure the plugin depending on the system I'm using at the time.

On the command-line, this works:

fdfind --absolute-path --color "never" -E "/proc" -HItd "(^venv_)" ~

And so does this

fdfind --absolute-path --color "never" -E "/proc" -HItd "(^venv_)" ~/

Note the trailing slash on the tilde in the second example.

And this works also

fdfind --absolute-path --color "never" -E "/proc" -HItd "(^venv_)" $HOME

Thanks again for writing this awesome plugin and taking the time to look into my issue.

[linux-cultist]

Hi,

Ok great, then it should be an easy fix. I assumed neovim would resolve ~ but it doesn't do that automatically. There is a function for it, so I will just add that to the code and problem will be solved.

And yes, I've been thinking about letting the user specify the regexp for environment names... And make the plugin more flexible. Right now it's just matching the exact name as you noticed, because I originally wanted to keep it simple for users so they don't have to understand regexp syntax. But we could add another option for power users so they could experiment more with stuff like that, as long as they understand that the plugin will show them lots of weird results if they get it wrong. :)

I think you are probably getting bad performance now though since the plugin has to search your entire home directory. If you would put your virtual envs in a new directory and set the path to that one (with parents set to 0), you will get instant results.

Or if they are in workspace folder already, then set parents to 0 so the plugin doesn't go up into your home directory.

[linux-cultist]

I pushed a fix for this so if you update the plugin now, you can set your path with ~ as we expect. :)

The rest of the stuff you wrote about regexp, please create another issue for it and label it Feature Request. :+1:

[watsonb]

Thanks for your rapid response and fix! Keep up the good work.