Run helm-fiind (or recursive helm-find-file) on multiple directories simultaneously

[vspinu]

I would like to see all the files and directories recursively from a fixed pre-defined list of directories.

Any easy solution for this? Thanks.

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

I would like to see all the files and directories recursively from a fixed pre-defined list of directories.

Any easy solution for this? Thanks.

AFAIK nothing like this exists (would be too slow unless the directories are small). Otherwise on one directory you can use helm-browse-project.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[priyadarshan]

Could perhaps emacs-async be of some use?

[thierryvolpiatto]

Priyadarshan notifications@github.com writes:

Could perhaps emacs-async be of some use?

No, what could be done though is use locate with "--database" option:

locate -d foo.db:bar.db:baz.db myfile

will search myfile in foo, bar and baz assuming you created a locate db file in each directories.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[priyadarshan]

No, what could be done though is use locate with "--database" option:

Ah, very smart! Thank you for the tip.

[vspinu]

Find command can accept multiple directories as input.

The main use case is to navigate between common dayly projects and documents. In my case it's about 20 directories with 10-100 files in each. I think whatever solution should work just fine.

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

Find command can accept multiple directories as input.

No, find is too slow, files are not indexed.

The main use case is to navigate between common dayly projects and documents. In my case it's about 20 directories with 10-100 files in each. I think whatever solution should work just fine.

So for this, what you are asking for is not the good approach IMO. You can use helm-find-files+bookmarks+helm-browse-project.

Here how to do:

1) Navigate in each of your projects with helm-find-files', and in each project useC-x r m', this will create a bookmark for each of your project without quitting helm-find-files.

2) Once this work is done, start helm-find-files and hit `C-x r b' to find one of your projects and hit RET.

3) Once in helm-find-files, hit `C-x C-d', you will have a buffer source showing the open buffers of this project and a source displaying the list of files in this project.

NOTE: For 3) you need some dependencies: helm-ls-git for your git controlled projects, helm-ls-hg for the hg ones and helm-ls-svn for the svn ones.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[thierryvolpiatto]

Thierry Volpiatto thierry.volpiatto@gmail.com writes:

Here how to do:

1) Navigate in each of your projects with helm-find-files', and in each project useC-x r m', this will create a bookmark for each of your project without quitting helm-find-files.

2) Once this work is done, start helm-find-files and hit `C-x r b' to find one of your projects and hit RET.

3) Once in helm-find-files, hit `C-x C-d', you will have a buffer source showing the open buffers of this project and a source displaying the list of files in this project.

NOTE: For 3) you need some dependencies: helm-ls-git for your git controlled projects, helm-ls-hg for the hg ones and helm-ls-svn for the svn ones.

Also forget to mention that in 3) you can also start directly to bookmarks with M-x helm-filtered-bookmarks.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[vspinu]

On Tue, Sep 01 2015 08:55, Thierry Volpiatto wrote:

2) Once this work is done, start helm-find-files and hit `C-x r b' to find one of your projects and hit RET.

This is nice but still a bit too chainy, and it's not recursive. I am also not a user of helm-find-files (probably the only thing that I cannot get used to in helm).

I think I will have a look into the locate alternative as you proposed earlier.

3) Once in helm-find-files, hit `C-x C-d', you will have a buffer source showing the open buffers of this project and a source displaying the list of files in this project.

NOTE: For 3) you need some dependencies: helm-ls-git for your git controlled projects, helm-ls-hg for the hg ones and helm-ls-svn for the svn ones.

Good to know. This is similar to projectile-find-file I guess.

[thierryvolpiatto]

I think I will have a look into the locate alternative as you proposed earlier.

I will add something like below soon:

(defun helm-ff-locate-from-multi-db (_candidate)
  (let ((dbs (helm-marked-candidates)))
    (helm-locate-with-db dbs)))

(setq helm-find-files-actions (helm-append-at-nth
                               helm-find-files-actions
                               (helm-make-actions
                                "Locate files from dbs"
                                'helm-ff-locate-from-multi-db)
                               4))

I will have to check if marked files are really db files.

[thierryvolpiatto]

This seems better to add:

(defvar helm-locate-project-list nil
  "A list of directories, your projects.")

(defun helm-locate-find-dbs-in-projects ()
  (let* ((pfn (lambda (candidate directory)
                (unless (= (shell-command
                            (format helm-locate-create-db-command
                                    candidate
                                    directory))
                           0)
                  (error "Failed to create locatedb file `%s'" candidate)))))
    (cl-loop for p in helm-locate-project-list
             for db = (expand-file-name
                       helm-ff-locate-db-filename
                       (file-name-as-directory p))
             if (file-exists-p db)
             collect db
             else do (funcall pfn db p)
             and collect db)))

(defun helm-locate-find-files-in-projects ()
  "Find files with locate in `helm-locate-project-list'."
  (interactive)
  (let ((dbs (helm-locate-find-dbs-in-projects)))
    (when dbs
      (helm-locate-with-db dbs))))

Probably it is this I will add, as using it from marked files is not so useful.

[priyadarshan]

I find this addition tremendously useful. Leveraging the "slave OS" locate db feels really good from the engineering pov, too.

[thierryvolpiatto]

Priyadarshan notifications@github.com writes:

I find this addition tremendously useful. Leveraging the host OS locate db feels really good from the engineering pov, too..

Will add soon after reviewing it (not sure it is working well as I quickly wrote it this morning).

Thanks.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[vspinu]

On Tue, Sep 01 2015 23:47, Thierry Volpiatto wrote:

This seems better to add:

(defvar helm-locate-project-list nil "A list of directories, your projects.")

This is awesome. Thanks!

Would it be difficult to add an updatedb action on all those data bases?

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

On Tue, Sep 01 2015 23:47, Thierry Volpiatto wrote:

This seems better to add:

(defvar helm-locate-project-list nil "A list of directories, your projects.")

This is awesome. Thanks!

Would it be difficult to add an updatedb action on all those data bases?

It is already done, read further the code, the release version will allow refreshing dbs with C-u.

Thierry

https://emacs-helm.github.io/helm/

[vspinu]

@thierryvolpiatto New, allow browsing a list projects recursively (#1148).

Thanks. I am trying this out. Unfortunately it's not that useful as I expected due to still huge number of candidates.

I see two potential improvements that might alleviate the problem One, having separate source for each directory in helm-locate-project-list. The other, having a separate helm-locate-projects-command config to allow for inclusion of -A option for pre-narrowing on the file types.

[vspinu]

nclusion of -A option for pre-narrowing on the file types.

-A is not particularly useful as it doesn't allow for regexp (linux).

I cannot make -r option work either because helm inserts a trailing space after a pattern.

For instance

(setq helm-locate-command "locate %s -e -r %s\\(cpp\\|el\\)")

produces :locate -i -e -r xxx \(cpp\|el\) (note the space after xxx).

Besides that, the helm-locate-command cannot be bound dynamically:

(defun my-helm-projects-find-files (update)
  (interactive "P")
  (let ((helm-locate-command "locate %s -e -r %s\\(cpp\\|el\\)"))
    (helm-projects-find-files update)))

would still use the global helm-locate-command.

Any ideas of how to pre-narrow the candidates? updatedb has a couple of options to exclude a list of directories, but that's really awkward.

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

Any ideas of how to pre-narrow the candidates? updatedb has a couple of options to exclude a list of directories, but that's really awkward.

See /etc/updatedb.conf You have a manpage for this also. The line to uncomment is: PRUNENAMES=".git .bzr .hg .svn"

Thierry

https://emacs-helm.github.io/helm/

[vspinu]

On Sat, Sep 05 2015 03:51, Thierry Volpiatto wrote: You have a manpage for this also. The line to uncomment is: PRUNENAMES=".git .bzr .hg .svn"

Yes. I know. This is what I was referring to as "awkward". That works for directories only but I want to exclude all files except of handful of file types matching a patterns. I was hopping for an inclusion option rather than an exclusion.

Would it be possible to fix the "trailing space" in helm-locate-command format expansion?

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

On Sat, Sep 05 2015 03:51, Thierry Volpiatto wrote: You have a manpage for this also. The line to uncomment is: PRUNENAMES=".git .bzr .hg .svn"

Yes. I know. This is what I was referring to as "awkward". That works for directories only but I want to exclude all files except of handful of file types matching a patterns. I was hopping for an inclusion option rather than an exclusion.

Would it be possible to fix the "trailing space" in helm-locate-command format expansion?

No,

(setq helm-locate-command "locate %s -e -r %s\\(cpp\\|el\\)")

is non-sense if you read carefully the source code, what you want to do is reimplement all helm locate and the way it works here.

So this wont work let-bounded or not.

Thierry

https://emacs-helm.github.io/helm/ Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[vspinu]

(setq helm-locate-command "locate %s -e -r %s(cpp|el)") is non-sense if you read carefully the source code, what you want to do is

It almost works except for that silly space.

 (setq helm-locate-command "locate %s -e -r %s.*\\(cpp\\|el\\)$")

I can see that with the input "xxx" the actual regexp passed to locale is `"xxx .*(cpp|el)$". This would have worked just fine for my need if helm would not have inserted that space behind "xxx". I don't think this is nonesense at all. And unless you have other ideas, this is the closest solution to the working system so far.

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

(setq helm-locate-command "locate %s -e -r %s\(cpp\|el\)") is non-sense if you read carefully the source code, what you want to do is

It almost works except for that silly space.

(setq helm-locate-command "locate %s -e -r %s.*(cpp|el)$")

I can see that with the input "xxx" the actual regexp passed to locale is `"xxx .*(cpp|el)$". This would have worked just fine for my need if helm would not have inserted that space behind "xxx". I don't think this is nonesense at all. And unless you have other ideas, this is the closest solution to the working system so far.

No it is not a solution, it is just a hack. So I will not go in this direction.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[vspinu]

On Mon, Sep 07 2015 05:20, Thierry Volpiatto wrote:

No it is not a solution, it is just a hack.

Sounds like you have some ideas in that regard. I will hold my breath then:)

[michael-heerdegen]

The locate implementation I use does allow to use -A together with -r. "mlocate" - implements locate just like GNU locate, but with some more features. With this, what you want @vspinu seems to be trivial.

[vspinu]

The mlocate and locate seems to be the same thing on Ubuntu (mlocate = locate + updatedb):

~$ locate --version
mlocate 0.26
Copyright (C) 2007 Red Hat, Inc. All rights reserved.
This software is distributed under the GPL v.2.

But the -r options cannot be used with -A.

I fixed it by piping the locate into grep. The following works:

(setq helm-projects-locate-command "locate %s -e -r \"\\.\\(cpp\\|el\\|R\\|cl\\|sh\\|java\\|org\\|txt\\|md\\)$\" | grep -i %s")

(defun my-helm-projects-find-files (update)
  (interactive "P")
  (let ((helm-locate-command helm-projects-locate-command))
    (helm-projects-find-files update)))

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

But the -r options cannot be used with -A.

Here the recommended command for helm locate:

"locate %s -e -A --regex %s"

I fixed it by piping the locate into grep. The following works:

(setq helm-projects-locate-command "locate %s -e -r \".(cpp|el|R|cl|sh|java|org|txt|md)$\" | grep %s")

You are defeating multi regexp matching and fuzzy matching (if enabled).

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[michael-heerdegen]

Vitalie Spinu notifications@github.com writes:

The mlocate and locate seems to be the same thing on Ubuntu (mlocate = locate + updatedb):

No. mlocate and GNU locate are different implementations with different features. They provide binaries with the same names however.

But the -r options cannot be used with -A.

Who says that? Works perfectly here, mlocate 0.26.

Michael.

[thierryvolpiatto]

Michael Heerdegen notifications@github.com writes:

Vitalie Spinu notifications@github.com writes:

The mlocate and locate seems to be the same thing on Ubuntu (mlocate = locate + updatedb):

No. mlocate and GNU locate are different implementations with different features. They provide binaries with the same names however.

But the -r options cannot be used with -A.

Who says that? Works perfectly here, mlocate 0.26.

Same here.

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997

[vspinu]

Who says that? Works perfectly here, mlocate 0.26.

~$ locate -i -e -A -r cpp tmp
locate: non-option arguments are not allowed with --regexp
~$ locate -i -e -A cpp -r tmp
locate: non-option arguments are not allowed with --regexp

Here the recommended command for helm locate: "locate %s -e -A --regex %s"

That was it!

Now I have:

(setq helm-projects-locate-command "locate %s -e -A --regex \"\\.(c|h|cpp|java|xml|el|R|cl|sh|org|txt|md)$\" %s")
(defun my-helm-projects-find-files (update)
  (interactive "P")
  (let ((helm-locate-command helm-projects-locate-command))
    (helm-projects-find-files update)))

Works great except for the helm-resume, but I can live with that.

You are defeating multi regexp matching and fuzzy matching (if enabled).

If -rexex is recommended, why do you set helm-locate-command to "locate %s -e -r %s" on linux? -r doesn't allow for multi regexp matching. (Also, the doc of helm-locate-command is incorrect, it says that the command is "locate %s -e -A %s" on Gnu/linux).

[michael-heerdegen]

Vitalie Spinu notifications@github.com writes:

If -rexex is recommended, why do you set helm-locate-command to "locate %s -e -r %s" on linux? -r doesn't allow for multi regexp matching.

To make it possible what the doc states: "Note: you can add locate options after entering pattern.". You can (must) specify another -r for multi regexp matching.

Michael.

[thierryvolpiatto]

Vitalie Spinu notifications@github.com writes:

If -rexex is recommended, why do you set helm-locate-command to "locate %s -e -r %s" on linux?

Because there are still people using previous versions of locate that don't support these arguments.

(e.g ubuntu 12.04 is providing an old version of locate)

Thierry Get my Gnupg key: gpg --keyserver pgp.mit.edu --recv-keys 59F29997