Building XED: unable to rename xed and mbuild directories

[mwkrentel]

There is an issue beginning with v2023.04.16 where building XED with xed and mbuild clones, the xed directory must be named exactly xed, not xed-old or xed-new or anything else.

How to reproduce:

Pick a directory with a fairly short path from /. In this case, I'll use /tmp/krentel/work.

Inside work, clone mbuild and xed, except rename xed to something else like repo.

So, the contents of /tmp/krentel/work are:

$ ls -l
total 8K
drwxr-xr-x  7 krentel users 4096 May 18 10:49 mbuild
drwxr-xr-x 14 krentel users 4096 May 18 10:50 repo

The repo directory is the clone of xed, just renamed as repo.

Check the path from work to / to make sure there are no adjacent directories named xed, so no /xed, /tmp/xed, /tmp/krentel/xed, etc. That's the reason for picking a short path to /.

Now, cd to repo and run mfile.py.

$ cd repo
$ ./mfile.py
...
R: 0 P: 0 C: 115 E: 0 / 8 secs 
    BUILT: /tmp/krentel/work/repo/obj/xed3-phash-lu-vv0.o
R: 1 P: 0 C: 115 E: 0 / 8 secs 
[REMOVING] obj/libxed.a
R: 0 P: 0 C: 116 E: 0 / 8 secs 
    BUILT: /tmp/krentel/work/repo/obj/libxed.a
Could not find /xed file, looking upwards
ABORT: XED ERROR: mfile.py could not find scripts directory

It dies trying to find the scripts directory of the subdirectory xed. It works if repo is named specifically xed, but nothing else.

But the strange thing is that it works if I create an empty directory xed as a sibling to repo and build inside repo.

$ ls -l
total 12K
drwxr-xr-x  7 krentel users 4096 May 18 10:49 mbuild
drwxr-xr-x 16 krentel users 4096 May 18 10:56 repo
drwxr-xr-x  2 krentel users 4096 May 18 11:02 xed

$ ls -l xed
total 0K

This suggests that something is looking for the scripts directory as a subdir of xed but then not using it, at least not for this part of the build.

Note: it also fails in the same way if mbuild is named something else. mbuild can be in a different location (not sideways to xed) by setting PYTHONPATH, but it still has to be called mbuild. For example, if I rename mbuild as foo and set PYTHONPATH,

R: 0 P: 0 C: 115 E: 0 / 7 secs 
    BUILT: /tmp/krentel/work/repo/obj/xed3-phash-lu-vv0.o
R: 1 P: 0 C: 115 E: 0 / 7 secs 
[REMOVING] obj/libxed.a
R: 0 P: 0 C: 116 E: 0 / 7 secs 
    BUILT: /tmp/krentel/work/repo/obj/libxed.a
Could not find /mbuild file, looking upwards
ABORT: XED ERROR: mfile.py could not find scripts directory

Notes:

1. This is new behavior (searching for scripts) starting with v2023.04.16. This works fine with v2022.10.11.

2. This fails on any version of Linux. I've tested RH 8.x and Ubuntu 20.04 and 22.04.

3. It doesn't depend on the version of python.

4. The critical thing is that there is no upwards directory (or file) named xed that would cause the script to return success. That's the reason for picking something close to /.

---

Now, you may consider this not to be a bug and just dismiss. But let me suggest a few things.

One, this is new to 2023.04.16. And it's looking for the scripts directory but not using it. This suggests that your build scripts are not doing what you think they're doing. You may run into more serious problems later and better to clean it out now before it becomes too entrenched.

Two, I often build multiple versions of a package for comparisson. For example, I might want to build xed-old, xed-new, xed-patched, where none are named exactly xed. Why should this not be allowed??

Finally, this change broke my spack recipe and took me nearly a week to track down. :-( (Spack renames all source dirs to spack-src.)

https://github.com/spack/spack/pull/37582

[marjevan]

Thanks for the detailed report.

This is a bug. We will fix it soon.

[marjevan]

I see that you solved the issue with a local workaround. Is this workaround good enough for your usage? Is this an urgent issue for you? Can it wait for a week or two? (I can send you a fixing patch in an urgent case)

[mwkrentel]

Yes, I have a satisfactory workaround for spack. So, there's no rush to commit a fix, take your time. I'm just happy that you consider it a bug.

[marjevan]

Great. Thanks.

[marjevan]

@mwkrentel - Please check External Release v2023.06.07

[mwkrentel]

Yes, it works now, both for renaming xed and mbuild. Thanks!