Initialize the ASDF source registry with all projects in ~/dev/lisp/, excluding specific project directories

[vindarel]

Initialize the ASDF source registry with all projects in ~/dev/lisp/, excluding specific project directories (asdf/ and old/).

• In the source registry, :EXCLUDE overrides any previous exclusion. Really not the right name for what it does. Also see :ALSO-EXCLUDE.
• :EXCLUDE and :ALSO-EXCLUDE take "EXCLUSION-PATTERNS". Which are not patterns. They must be directory names, or they will never ever match anything.
• Due to how it's implemented, if you ignore "foo", you will ignore all "foo" subdirectories in any directory previous added. Just give me path globs!
• Finally, :EXCLUDE and :ALSO-EXCLUDE do not seem to do anything, COMPUTE-SOURCE-REGISTRY is always called with the default exclusion list even though PROCESS-SOURCE-REGISTRY processed the :EXCLUDE form.

source: https://gist.github.com/galdor/059db07a20748f3b86247387d2dcf8eb && https://framapiaf.org/@galdor@emacs.ch/112542427965071262

(let* ((initialize-source-registry
         (intern "INITIALIZE-SOURCE-REGISTRY" (find-package "ASDF")))
       (project-root-directory
         (merge-pathnames #p"dev/lisp/" (user-homedir-pathname)))
       (blacklisted-directories
         (mapcar (lambda (subpath)
                   (merge-pathnames subpath project-root-directory))
                 '(#p"asdf/"
                   #p"old/")))
       (project-directories
         (delete-if (lambda (path)
                      (member path blacklisted-directories :test #'equal))
                    (directory (make-pathname :defaults project-root-directory
                                              :type :wild :name :wild)))))
  (asdf:clear-source-registry)
  (funcall initialize-source-registry
           `(:source-registry
             :ignore-inherited-configuration
             ,@(mapcar (lambda (path)
                         `(:tree ,path))
                       project-directories))))

[svetlyak40wt]

Finally, :EXCLUDE and :ALSO-EXCLUDE do not seem to do anything, COMPUTE-SOURCE-REGISTRY is always called with the default exclusion list even though PROCESS-SOURCE-REGISTRY processed the :EXCLUDE form.

This is incorrect. The order of rules does matter. The chapter "Search Algorithm" says:

Exclude statements are not propagated to further included or inherited configuration files or expressions; instead the defaults are reset around every configuration statement to the default defaults from asdf::default-source-registry-exclusions.

For example, if we have two files ~/lisp/foo/foo.asd and ~/lisp/bar/bar.asd with a content like this:

(defsystem "foo")

and

(defsystem "bar")

Then we could create such registry configuration:

(:source-registry
  :inherit-configuration
  (:also-exclude "bar")
  (:tree "~/lisp/"))

But a version where :also-exclude goes after the :tree like this:

(:source-registry
  :inherit-configuration
  (:tree "~/lisp/")
  (:also-exclude "bar"))

will not work, because these directives are processed sequentially by this code.

[svetlyak40wt]

Also, I've found pathname-match-p and a wild pathname could be used for having path globs.

Here is a small example:

CL-USER> (pathname-match-p "~/lisp/blah/minor"
                           (uiop:wilden "~/lisp/"))
T

CL-USER> (pathname-match-p "/tmp/lisp/blah/minor"
                           (uiop:wilden "~/lisp/"))
NIL

It would be cool to incorporate ability to add wild pathnames into the ASDF's exclusion lists and to match them agains a full pathname instead of the last directory component!