Various dependency failures on Windows 10

[bigos]

When I wipe out quicklisp and install everything again and try to install (ql:quickload :qtools) I get errors.

For example Component "iterate" not found Condition of type ASDF/FIND-COMPONENT:MISSING-COMPONENT

used sbcl 1.4.2

[Shinmera]

This is a new bug that was opened up by ASDF 3.3.1. I currently have no idea why it happens or what to do about it.

[bigos]

If I load the 2 dependencies separately, namely zlib and iterate, everything else seems to work.

[fare]

My experience is that when you load quicklisp, it can find previously-downloaded systems if you call asdf:load-system, but will only download them if you use ql;quickload. I once tried to tell Xach how he could make it work better, but he was not interested.

Since asdf itself knows nothing about quicklisp, when it loads defsystem-depens-on dependencies, it uses asdf:load-system and not ql:quickload. Could that explain the issue?

[Shinmera]

defsystem-depends-on is not the problem. Quicklisp scans system definitions statically before deployment. When a system is quickloaded, it automatically downloads the whole transitive dependency tree according to its own internal database.

[fare]

I looked in more detail at the .asd file. Wow, it's trying to live patch the system from commonqt, and poking deep in non-supported ASDF internals.

You know, will things like that are acceptable short term stop gap, you've been doing it since 2015, and not one time did you shoot an email to asdf-devel to discuss how to improve the issue. No wonder it breaks eventually.

[fare]

Interestingly, my tests with cl-test-grid didn't detect this breakage, probably because the qt requirements were already breaking the test platform.

[fare]

While I can imagine a lot of better workarounds to live patch qt.asd in a more robust way, the real solution is to properly teach asdf and cffi about C libraries as dependencies. cffi-toolchain is a good start for that, but only a start.

[Shinmera]

I'll fully take the blame for not writing to asdf-devel. Communicating to other people about things I might need is definitely not among my strong suits.

As for why it didn't show on cl-test-grid; the breakage only happens if the dependencies of the qt system are not already in ASDF's registry. In that case qt-libs will ask Quicklisp to download the system, and later on load it. I imagine that on your test grid you already had the systems downloaded, yeah?

I'm not sure how teaching ASDF/CFFI about the library dependencies is a solution here, could you elaborate on that? Even if we, say, change the qt system to depend on its .so as a file rather than trying to generate it, how would qt-libs be able to hook into that to say that it will provide that file without directly introducing a dependency from qt to qt-libs?

[Shinmera]

The cleanest solution to all of this would be to have CommonQt depend on qt-libs. I'll see what Stas says about this idea.

[Shinmera]

Related: Shinmera/qt-libs@d8dfd9de8f233b81ced55a861988e0118297f640 Shinmera/commonqt@a7e80bbefbf32134c2af33666dc4b20cca745c58 commonqt/commonqt#29

[Shinmera]

This is now merged and rolled out.