elbe initvm v14.9.3 fails due to unavailable dbsfed.xsd URL

[hnez]

Hi,

I am currently in the process of getting my feet wet with ELBE and can not yet claim to have gathered a deep understanding of its inner working, so some of the Issues and Pull Request that I am about to open may be due to user error. Feel free to close them accordingly. I just want to start reporting things I've come across while the memory is still fresh.

I've noticed that the most recent ELBE release v14.9.3 fails to set up the initvm on my system.

I've set up the ELBE python project via venv and pip on my Arch Linux system:

$ git show HEAD
commit 02c730b498579a70f4c3220f61526386bb366b50 (HEAD, tag: v14.9.3, origin/releases/v14.9.3)
Author: Benedikt Spranger <b.spranger@linutronix.de>
Date:   Thu Oct 26 15:31:14 2023 +0200

    changelog: Bump elbe version to 14.9.3

    Generate the changelog via gbp.

    Signed-off-by: Benedikt Spranger <b.spranger@linutronix.de>
    Reviewed-by: Volker Haspel <volker.haspel@linutronix.de>
$ python3 -m venv --system-site-packages ~/.venvs/elbe-experiments
$ python3 -m pip install .
$ cd ../elbe-initvms/

And have noticed that elbe initvm --devel create fails due to https://www.linutronix.de/projects/Elbe/dbsfed.xsd no longer/not being available:

$ elbe initvm --devel create
elbe preprocess failed.
Traceback (most recent call last):
  File "/home/leonard/.venvs/elbe-experiments/bin/elbe", line 56, in <module>
    cmdmod.run_command(sys.argv[2:])
  File "/home/leonard/.venvs/elbe-experiments/lib/python3.11/site-packages/elbepack/commands/preprocess.py", line 46, in run_command
    xmlpreprocess(args[0], opt.output, variants, opt.proxy, opt.gzip)
  File "/home/leonard/.venvs/elbe-experiments/lib/python3.11/site-packages/elbepack/xmlpreprocess.py", line 315, in xmlpreprocess
    schema_tree = etree.parse(schema_file)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/lxml/etree.pyx", line 3547, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1952, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1978, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1881, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1200, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 670, in lxml.etree._raiseParseError
OSError: Error reading file 'https://www.linutronix.de/projects/Elbe/dbsfed.xsd': failed to load external entity "https://www.linutronix.de/projects/Elbe/dbsfed.xsd"

'elbe init' Failed
Giving up

I think on master this is fixed by b33923fe, which removes the URL from elbepack/xmlpreprocess.py, but I think it would be good if there was a solution for the released version as well.

While ripgrepping around the code I've also noticed that many example xmls contain the xsi:schemaLocation="https://www.linutronix.de/projects/Elbe dbsfed.xsd"> (e.g. with a space before the filename). That looks a bit strange.

[t-8ch]

Hi and welcome to elbe!

elbe in its current form is not intended to be run from a virtual environment. The python package does not carry all necessary files (in this case the schema files). For now either use a Debian package or run ./elbe from the source directory. (I am currently working on aligning elbe more to normal python conventions, which also includes correct packaging)

Even if this was fixed you would run into other issues of elbe with new versions of its dependencies that are available in ArchLinux. These are fixed in current master, as I use ArchLinux myself to develop elbe. In general, master should be stable, if not I'm happy for bug reports.

xsi:schemaLocation="https://www.linutronix.de/projects/Elbe dbsfed.xsd"

This is correct, the URL is only for identification. It is not actually used to download the schema. (It could be, but right now it is not)

[hnez]

Hi and welcome to elbe!

Thanks! Apart from the usual friction caused by not reading the manual or ignoring what it tells me to do (like not using the Debian packages and using venv + pip) I got something up and running quite quickly. Thumbs up for that. I quite like the approach to preventing host tool leakage by just using a virtual machine to do the majority of the work in (even though it is a bit of a sledgehammer method), because host tool leakage has caused by some grief in the past when using e.g. Yocto.

elbe in its current form is not intended to be run from a virtual environment. The python package does not carry all necessary files (in this case the schema files). For now either use a Debian package or run ./elbe from the source directory. (I am currently working on aligning elbe more to normal python conventions, which also includes correct packaging)

Even if this was fixed you would run into other issues of elbe with new versions of its dependencies that are available in ArchLinux. These are fixed in current master, as I use ArchLinux myself to develop elbe. In general, master should be stable, if not I'm happy for bug reports.

I did indeed need to tweak the setup.py to point it to the new location of the schema files to get master running in a venv. I can live with a bit of breakage, so I think I'll continue in a venv and open pull request if issues appear.

xsi:schemaLocation="https://www.linutronix.de/projects/Elbe dbsfed.xsd"

This is correct, the URL is only for identification. It is not actually used to download the schema. (It could be, but right now it is not)

Ah, so the URL not being available is actually expected behavior? In that case feel free to close this issue.

[t-8ch]

Ah, so the URL not being available is actually expected behavior?

Yes.

Ok, then I'll close this. (And I'll send a patch to fix the schema location in virtualenv, that needs to happen sooner or later anyways)