Install request: Xpra

[heatherkellyucl]

IN:03022221 For proof of concept right now.

Xvnc can't find the xkb rules (apparently you can tell it where they are at build time if you build from source) and users could connect to other people's sessions on the same box if they had the details, so not ideal. Xpra may work better.

https://xpra.org/ https://xpra.org/trac/wiki/Building https://xpra.org/trac/wiki/Dependencies

[heatherkellyucl]

Required dependencies:

• [ ] Xdummy
• [ ] GTK 2.x (hopefully system version is sufficient)
• [ ] python-pillow
• [ ] Cython

Recommended:

• [ ] numpy
• [ ] rencode (python package)

[heatherkellyucl]

The Xorg requirements are (package names)

• [ ] xorg-x11-server-Xorg
• [ ] xorg-x11-drv-dummy
• [ ] xorg-x11-xauth
• [ ] xorg-x11-xkb-utils

[heatherkellyucl]

The plan is to use one of the provided build-everything from source scripts for Xorg, which makes sure everything is done in the right order. Since there has been no full release (they call it a katamari!) since X11R7.7 in 2012 and many packages have updates since then, I am going to use the current versions of everything, download all those specific tar versions first and go from there.

From what I've read, Xpra ought to work with an Xorg installed in a userspace location in this fashion, but NVIDIA's drivers cannot, so this wouldn't work if there was a GPU login node.

[heatherkellyucl]

I have a list of all the wgets we need for apps and util (apps is one of the largest directories). The build.sh script will use tar files in a directory if they are there, so we will know what versions we have built.

[heatherkellyucl]

Finished going through the drivers, on to fonts (of which there aren't too many, by default).

[heatherkellyucl]

Done - got the list of all the current versions of tarfiles. Have commented out ones we definitely don't need (mostly in drivers and fonts) but for the vast majority of things in apps, for example, I have no idea what they do and whether anything needs them so have left them in.

[heatherkellyucl]

Getting all the source on Myriad (removes comments and empty lines first):

wget --input $(grep -v "^#" /shared/ucl/apps/build_scripts/lists/xorg-all.list | grep -v "^$")

This is going in /shared/ucl/apps/xorg/full/source/2018-07-06 for now.

[heatherkellyucl]

You have to clone the current version of build.sh (no current tagged release).

git clone git://anongit.freedesktop.org/git/xorg/util/modular util/modular

I have also put this in the 2018-07-06 directory.

[heatherkellyucl]

We're off to a good start...

./util/modular/build.sh --autoresume resumefile /shared/ucl/apps/xorg/full/2018-07-06

./util/modular/build.sh: line 731: [: missing `]'
./util/modular/build.sh: line 750: ninja: command not found
build.sh: "ninja -C builddir" failed on util/macros
build.sh: error processing:  "util/macros"

    if [ $use_autogen != 0 ] || [ $use_configure != 0]; then

There's a missing space before that last ] in build.sh, and ninja is a dependency that wasn't on the dependency list.

[heatherkellyucl]

https://github.com/ninja-build/ninja

• [x] install ninja on Myriad
• [x] Legion
• [x] Grace
• [x] Thomas
• [ ] ~modulefile~

[heatherkellyucl]

Hmm, I wrote a buildscript for a standalone ninja using the system python, but not sure if it should be installed as part of the python recommended bundles instead.

[heatherkellyucl]

It doesn't have external dependencies, so yes, it should be in the recommended bundles.

[heatherkellyucl]

Ninja was done, for python2 and 3.

After fixing that bracket, build.sh is now building a load of stuff. Waiting for first failure.

[heatherkellyucl]

Ok, we got to app/luit:

make[2]: Entering directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/app/luit-1.1.1'
  CC     charset.o
  CC     iso2022.o
  CC     locale.o
  CC     luit.o
  CC     other.o
  CC     parser.o
  CC     sys.o
sys.c: In function ‘allocatePty’:
sys.c:339:5: error: implicit declaration of function ‘posix_openpt’ [-Werror=implicit-function-declaration]
     pty = posix_openpt(O_RDWR);
     ^
sys.c:339:5: warning: nested extern declaration of ‘posix_openpt’ [-Wnested-externs]
cc1: some warnings being treated as errors
make[2]: *** [sys.o] Error 1
make[2]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/app/luit-1.1.1'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/app/luit-1.1.1'
make: *** [all] Error 2
build.sh: "make " failed on app/luit
build.sh: error processing:  "app/luit"

[heatherkellyucl]

Freedesktop.org patched that in 2013: https://bugs.freedesktop.org/show_bug.cgi?id=47792 Xorg's apps directory doesn't have anything newer than 2012 for luit.

...because it doesn't have a tagged release since that one. https://cgit.freedesktop.org/xorg/app/luit/

[heatherkellyucl]

This configure.ac patch also needs applying first or that one won't: https://cgit.freedesktop.org/xorg/app/luit/commit/?id=445863f8b5059692ac7a4df785af6920849faa82

[heatherkellyucl]

It is actually a series of 3 small patches, which you can see at https://patchwork.freedesktop.org/project/Xorg/patches/?submitter=&state=*&q=luit&archive=both&delegate= (2013-07-13)

[heatherkellyucl]

git applying those three in order works.

[heatherkellyucl]

Because those patches change configure.ac, you then need to recreate the configure script. There was another thing that needed changing in configure.ac as well:

This needs to not have outer [] around AC_MSG_WARN (https://stackoverflow.com/a/37980772). Line 53 configure.ac:

# We don't link against libX11, just use its locale.alias data file
PKG_CHECK_EXISTS(x11, [],
    [AC_MSG_WARN([libX11 not found.  luit may not be able to find locale aliases without it.])])

A fourth patch has been created for this. Need to set the environment variables that build.sh has.

export PREFIX=/shared/ucl/apps/xorg/full/2018-07-06
export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig:$PREFIX/share/pkgconfig:$PKG_CONFIG_PATH
export ACLOCAL="aclocal -I $PREFIX/share/aclocal -I /usr/share/aclocal/"
export LD_LIBRARY_PATH=$PREFIX/lib:$LD_LIBRARY_PATH
export PATH=$PREFIX/bin:$PATH
export PKG_CONFIG=/bin/pkg-config

cd app/luit-1.1.1
rm -r autom4te.cache/
autoreconf -fi
cd ../..
# then continue build
./util/modular/build.sh --autoresume resumefile /shared/ucl/apps/xorg/full/2018-07-06

[heatherkellyucl]

Next failure is for app/xdm-1.1.11:

test -z "/usr/lib/systemd/system" || /bin/mkdir -p "/usr/lib/systemd/system"
 /bin/install -c -m 644 xdm.service '/usr/lib/systemd/system'
/bin/install: cannot create regular file ‘/usr/lib/systemd/system/xdm.service’: Permission denied
make[2]: *** [install-systemdsystemunitDATA] Error 1
make[2]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/app/xdm-1.1.11'
make[1]: *** [install-am] Error 2
make[1]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/app/xdm-1.1.11'
make: *** [install-recursive] Error 1
build.sh: " env LD_LIBRARY_PATH=/shared/ucl/apps/xorg/full/2018-07-06/lib:/shared/ucl/apps/autogen/5.18.12/gnu-4.9.2/lib:/shared/ucl/apps/guile/2.0.11/gnu-4.9.2/lib:/shared/ucl/apps/libbdwgc/7.4.2/gnu-4.9.2/lib:/shared/ucl/apps/xorg/full/2018-07-06/lib:/shared/ucl/apps/python/bundles/python3-3.0.0/venv/lib64:/shared/ucl/apps/python/bundles/python3-3.0.0/venv/lib:/shared/ucl/apps/openblas/0.2.14/gnu-4.9.2/lib:/shared/ucl/apps/python/3.6.3/gnu-4.9.2/lib:/shared/ucl/apps/llvm/3.9.1/gnu-4.9.2/lib:/shared/ucl/apps/bison/3.0.4/gnu-4.9.2/lib:/shared/ucl/apps/giflib/5.1.1/gnu-4.9.2/lib:/shared/ucl/apps/subversion/1.8.13/lib:/shared/ucl/apps/apr-util/1.5.4/lib:/shared/ucl/apps/apr/1.5.2/lib:/shared/ucl/apps/git/2.10.2/gnu-4.9.2/lib64:/shared/ucl/apps/flex/2.5.39/gnu-4.9.2/lib:/shared/ucl/apps/gcc/4.9.2/lib:/shared/ucl/apps/gcc/4.9.2/lib64:/shared/ucl/apps/libtool/2.4.6/lib:/shared/ucl/apps/mesa/13.0.6/gnu-4.9.2/lib make  install" failed on app/xdm
build.sh: error processing:  "app/xdm"

Yeah, that's not gonna work... Need ./configure --with-systemdsystemunitdir=$PREFIX/something for XDM.

[heatherkellyucl]

Will need to have a modfile to give configure options to xdm.

#   Building from a Custom Modules List
#
#   Starting from the list generated using the -L option, remove unwanted
#   modules. You may also add your own module or add specific configure
#   options for some modules to meet your configuration needs. Using the
#   --modfile option, your list replaces the built-in list of the script.

#   Specifying Configuration Options to Specific Modules
#
#   In the modulesfile used by the --modfile option, add any configuration
#   options you want to pass to the modules as it gets configures by autoconf.
#   Write the configure options next to the module name in the file.
#   It could be something like --enable-strict-compilation for example.

[heatherkellyucl]

# get list of modules we were building
./util/modular/build.sh -L > modfile
# add extra config to xdm module
sed -i -e "s|app/xdm|app/xdm --with-systemdsystemunitdir=\$PREFIX/systemd|" modfile
# continue building
./util/modular/build.sh --autoresume resumefile /shared/ucl/apps/xorg/full/2018-07-06 --modfile modfile

Now building more things.

[heatherkellyucl]

Error with apps/xfs

checking for ${prefix}/etc/X11/fontpath.d... no
checking for default font path... ${FONTROOTDIR}/misc/,${FONTROOTDIR}/TTF/,${FONTROOTDIR}/OTF/,${FONTROOTDIR}/Type1/,${FONTROOTDIR}/100dpi/,${FONTROOTDIR}/75dpi/
checking for daemon... yes
checking for XFS... no
configure: error: Package requirements (xproto >= 7.0.17 xfont2 >= 2.0.1 xtrans) were not met:

No package 'xfont2' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables XFS_CFLAGS
and XFS_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
build.sh: "./configure" failed on app/xfs
build.sh: error processing:  "app/xfs"

[heatherkellyucl]

Both libXfont and libXfont2 are in the downloaded tar files - so far it has only built libXfont though.

[heatherkellyucl]

libXfont2 recently replaced libXfont, so can probably remove the 1.x version entirely. (But I'll need to start the build over from clean to make sure everything gets recompiled with the right one).

[heatherkellyucl]

Makes sense to have a modfile1 that goes up to but not including apps/luit, then do the patching and autoreconf, then build with a modfile2 that has luit and the rest of the modules to be built.

[heatherkellyucl]

Still doesn't build libXfont2.

[heatherkellyucl]

Cannot find anything in docs or issues about getting build.sh to build libXfont2. I am going to split modfile1 again, build libXfont2 manually at the point it builds libXfont, then continue.

[heatherkellyucl]

libXfont2 built with no problems, continuing.

[heatherkellyucl]

Error with xserver:

checking for LIBDRM... no
configure: error: Package requirements (libdrm >= 2.4.89) were not met:

Requested 'libdrm >= 2.4.89' but version of libdrm is 2.4.74

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables LIBDRM_CFLAGS
and LIBDRM_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
build.sh: "./configure" failed on xserver
build.sh: error processing:  "xserver"

The system version of libdrm is too old.

[heatherkellyucl]

Still xserver:

checking for DGA... yes
checking for DGA... yes
checking for XORG_MODULES... yes
checking for GLAMOR... yes
checking for GBM... no
configure: error: Glamor for Xorg requires gbm >= 10.2.0
build.sh: "./configure" failed on xserver
build.sh: error processing:  "xserver"

[heatherkellyucl]

gbm appears to be part of mesa (have to build it with that explicitly enabled).

Need a different mesa install that isn't mesa/13.0.6/gnu-4.9.2. Possibly should let build.sh build mesa itself, as it can do that.

[heatherkellyucl]

Of course, mesa comes in the first segment of modules to build, so everything probably has to be redone again.

[heatherkellyucl]

It currently isn't picking up the mesa tarball as being there, because build.sh has pkg_stem="MesaLib" for some reason and the tarball is mesa-18.1.4.tar.xz. It hasn't been called MesaLib-x.x.x.tar.yy since 10.4.7.

(After sprint meeting: I have until the end of Tues 31 Aug to work on this and if it isn't ready, it has to wait until we have containers available as that will greatly simplify the install process).

[heatherkellyucl]

Oh no.

configure: error: LLVM target 'amdgpu' not enabled in your LLVM build. Required by r600.
build.sh: "./configure" failed on mesa/mesa
build.sh: error processing:  "mesa/mesa"

[heatherkellyucl]

Going to look for options to tell mesa not to try to build for amdgpu/r600 (because there are no graphics cards involved here and those are Radeon drivers).

[heatherkellyucl]

--with-gallium-drivers=swrast,swr should help. May also need --enable-gallium-osmesa and --disable-dri (and maybe --with-dri-drivers=)

[heatherkellyucl]

With --with-gallium-drivers=swrast,swr --enable-gallium-osmesa

configure: error: LLVM 4.0.0 or newer is required for swr
build.sh: "./configure" failed on mesa/mesa

[heatherkellyucl]

LLVM 6.0.1 requires GCC 4.8 or above: we are 4.9.2 so that is fine.

Build on:

• [x] Myriad
• [x] Legion
• [x] Grace
• [x] Thomas
• [x] write modulefile

[heatherkellyucl]

Build of llvm at 70% and should finish overnight.

[heatherkellyucl]

It looks like the Myriad make install step successfully completed before the Lustre issues last night, which is good.

[heatherkellyucl]

Currently successfully rebuilt libdrm, modfile1 list, libXfont2, and rebuilding modfile2 list (mesa is in this bit). Following is luit, modfile3 and modfile4. (The last two can probably be recombined because I moved the libdrm build to first as it doesn't depend on the rest).

[heatherkellyucl]

Whole load of missing LLVM things:

/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/drivers/swr/rasterizer/jitter/gen_state_llvm.h:826: undefined reference to `llvm::StructType::create(llvm::ArrayRef<llvm::Type*>, llvm::StringRef, bool)'
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/drivers/swr/rasterizer/jitter/gen_state_llvm.h:829: undefined reference to `llvm::DIBuilder::DIBuilder(llvm::Module&, bool)'
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/drivers/swr/rasterizer/jitter/gen_state_llvm.h:830: undefined reference to `llvm::DIBuilder::createFile(llvm::StringRef, llvm::StringRef, llvm::DIFile::ChecksumKind, llvm::StringRef)'

...

../../../../src/gallium/drivers/llvmpipe/.libs/libllvmpipe.a(lp_tex_sample.o): In function `lp_llvm_texture_member':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/drivers/llvmpipe/lp_tex_sample.c:115: undefined reference to `LLVMBuildGEP'
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/drivers/llvmpipe/lp_tex_sample.c:118: undefined reference to `LLVMBuildLoad'
collect2: error: ld returned 1 exit status
make[4]: *** [gallium_dri.la] Error 1
make[4]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/targets/dri'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium'

[heatherkellyucl]

It is specifically gallium-dri that fails. Top of failure:

make[4]: Leaving directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/state_trackers/dri'
Making all in targets/dri
make[4]: Entering directory `/lustre/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/targets/dri'
  CC       gallium_dri_la-target.lo
  CXXLD    gallium_dri.la
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `init_native_targets()':
/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include/llvm/Support/TargetSelect.h:119: undefined reference to `LLVMInitializeX86TargetInfo'
/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include/llvm/Support/TargetSelect.h:120: undefined reference to `LLVMInitializeX86Target'
/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include/llvm/Support/TargetSelect.h:121: undefined reference to `LLVMInitializeX86TargetMC'
/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include/llvm/Support/TargetSelect.h:133: undefined reference to `LLVMInitializeX86AsmPrinter'
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `gallivm_create_target_library_info':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:182: undefined reference to `llvm::Triple::Triple(llvm::Twine const&)'
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:182: undefined reference to `llvm::TargetLibraryInfoImpl::TargetLibraryInfoImpl(llvm::Triple const&)'
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `lp_get_default_memory_manager':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:783: undefined reference to `llvm::SectionMemoryManager::SectionMemoryManager(llvm::SectionMemoryManager::MemoryMapper*)'
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `lp_free_memory_manager':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:234: undefined reference to `llvm::RTDyldMemoryManager::~RTDyldMemoryManager()'
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `lp_is_function':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:811: undefined reference to `LLVMGetValueKind'
../../../../src/gallium/auxiliary/.libs/libgallium.a(lp_bld_misc.o): In function `lp_build_create_jit_compiler_for_module':
/shared/ucl/apps/xorg/full/source/2018-07-06/mesa/mesa-18.1.4/src/gallium/auxiliary/gallivm/lp_bld_misc.cpp:500: undefined reference to `llvm::EngineBuilder::EngineBuilder(std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >)'

[heatherkellyucl]

I think this may be related to https://github.com/rust-lang/rust/issues/36854 and https://patchwork.freedesktop.org/patch/75163/

We have individual llvm static libraries, and one big libLLVM.so. This is what various llvm-config options give:

llvm-config --libs
-lLLVMLTO -lLLVMPasses -lLLVMObjCARCOpts -lLLVMSymbolize -lLLVMDebugInfoPDB -lLLVMDebugInfoDWARF -lLLVMFuzzMutate -lLLVMTableGen -lLLVMDlltoolDriver -lLLVMLineEditor -lLLVMOrcJIT -lLLVMCoverage -lLLVMMIRParser -lLLVMObjectYAML -lLLVMLibDriver -lLLVMOption -lLLVMWindowsManifest -lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen -lLLVMGlobalISel -lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMDebugInfoCodeView -lLLVMDebugInfoMSF -lLLVMX86Desc -lLLVMMCDisassembler -lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMMCJIT -lLLVMInterpreter -lLLVMExecutionEngine -lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMTarget -lLLVMCoroutines -lLLVMipo -lLLVMInstrumentation -lLLVMVectorize -lLLVMScalarOpts -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser -lLLVMInstCombine -lLLVMTransformUtils -lLLVMBitWriter -lLLVMAnalysis -lLLVMProfileData -lLLVMObject -lLLVMMCParser -lLLVMMC -lLLVMBitReader -lLLVMCore -lLLVMBinaryFormat -lLLVMSupport -lLLVMDemangle

llvm-config --libfiles
# full paths to all .a

llvm-config --libnames
# names of all .a

This talks about trying to make cmake link against the single shared library failing https://groups.google.com/forum/#!topic/llvm-dev/YeEVe7HTasQ

Need to look at mesa's link options.

[heatherkellyucl]

Might need --disable-llvm-shared-libs as it tries to link the shared libs by default and is obviously failing.

[heatherkellyucl]

This is what mesa's config.log has. It does look like it is using llvm-config and I don't think there's any way to get the correct single shared library output from that.

LLVM_CFLAGS='-I/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS'
LLVM_CONFIG='/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/bin/llvm-config'
LLVM_CXXFLAGS='-I/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include -std=c++11 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS'
LLVM_INCLUDEDIR='/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/include'
LLVM_LDFLAGS='-L/lustre/shared/ucl/apps/llvm/6.0.1/gnu-4.9.2/lib '
LLVM_LIBS=''

[heatherkellyucl]

There have been multiple attempts at patching this for older llvm...

llvm-config --libs only reports the single shared library if LLVM was built with -DLLVM_LINK_LLVM_DYLIB=ON. llvm-config --shared-mode reports "shared" in that case, "static" otherwise (even if LLVM was built with -DLLVM_BUILD_LLVM_DYLIB=ON).

Ours is built with -DLLVM_BUILD_LLVM_DYLIB=ON only.

[heatherkellyucl]

We probably do want to set -DLLVM_LINK_LLVM_DYLIB=ON as well for llvm, so it can link both static and shared correctly.

[heatherkellyucl]

• [x] rebuild llvm on Thomas after login node reimaged (The others are going).