esteam error: UNBUNDLEABLES_A: bad array subscript

[hoefling]

Trying out esteam for the first time results in:

$ sudo -s
Password: 
$ esteam
 * Scanning /home/hoefling/steamlib ...
/usr/bin/esteam: line 213: UNBUNDLEABLES_A: bad array subscript

Sys info

```bash emerge --info Portage 2.3.3 (python 3.5.2-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-5.4.0, glibc-2.23-r3, 4.9.6-gentoo x86_64) ================================================================= System uname: Linux-4.9.6-gentoo-x86_64-AMD_A10-5750M_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.3 KiB Mem: 15604580 total, 8808504 free KiB Swap: 17301500 total, 17301500 free Timestamp of repository gentoo: Fri, 27 Jan 2017 10:00:01 +0000 sh bash 4.3_p48-r1 ld GNU ld (Gentoo 2.25.1 p1.1) 2.25.1 distcc 3.2rc1 x86_64-pc-linux-gnu [enabled] app-shells/bash: 4.3_p48-r1::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.22.3_rc4::gentoo dev-lang/python: 2.7.12::gentoo, 3.5.2::gentoo dev-util/cmake: 3.7.2::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.3::gentoo sys-apps/openrc: 0.22.4::gentoo sys-apps/sandbox: 2.10-r1::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69::gentoo sys-devel/automake: 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25.1-r1::gentoo sys-devel/gcc: 5.4.0-r2::gentoo, 6.3.0::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.6-r2::gentoo sys-devel/make: 4.2.1::gentoo sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers) sys-libs/glibc: 2.23-r3::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 hoefling-private location: /usr/local/portage/overlays/hoefling-private sync-type: git sync-uri: git://github.com/hoefling/hoefling-private-overlay masters: gentoo priority: 50 pypi location: /var/lib/layman/pypi sync-type: laymansync sync-uri: gs-pypi pypi masters: gentoo priority: 50 steam-overlay location: /usr/local/portage/overlays/steam-overlay sync-type: git sync-uri: git://github.com/anyc/steam-overlay.git masters: gentoo priority: 50 uwl location: /usr/local/portage/overlays/uwl sync-type: git sync-uri: git://github.com/lexa-uw/gentoo-layman masters: gentoo priority: 50 Installed sets: @esteam ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=bdver2 -mmmx -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mtbm -mavx -msse4.2 -msse4.1 -mlzcnt -mf16c -mprfchw -mfxsr -mxsave --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver2 -fstack-protector-strong" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.3/conf /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -pipe -march=bdver2 -mmmx -msse -msse2 -msse3 -mssse3 -msse4a -mcx16 -msahf -maes -mpclmul -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mtbm -mavx -msse4.2 -msse4.1 -mlzcnt -mf16c -mprfchw -mfxsr -mxsave --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver2 -fstack-protector-strong" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distcc distcc-pump distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo http://mirror.netcologne.de/gentoo/ rsync://mirror.netcologne.de/gentoo/ ftp://mirror.netcologne.de/gentoo/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://ftp.halifax.rwth-aachen.de/gentoo/ http://ftp.halifax.rwth-aachen.de/gentoo/ rsync://ftp.halifax.rwth-aachen.de/gentoo/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ http://ftp.spline.inf.fu-berlin.de/mirrors/gentoo/ ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo http://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ rsync://ftp-stud.hs-esslingen.de/gentoo/ http://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j21 -l5" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git --bwlimit=500k" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cxx dbus declarative dri dts dvd dvdr egl emboss encode exif fam ffmpeg firefox flac fortran gdbm gif glamor gpm iconv icu ipv6 jpeg kde kipi lcms ldap libnotify lm_sensors mad mmx mng modules mp3 mp4 mpeg multilib ncurses networkmanager nls nptl ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds pulseaudio qml qt3support qt5 readline sdl seccomp semantic-desktop session spell sqlite sse sse2 ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vlc vorbis wayland widgets wxwidgets x264 xattr xcb xcomposite xinerama xml xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx fma3 fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python3_5" PYTHON_TARGETS="python3_5" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="amdgpu radeonsi r600 radeon" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON ```

Packages installed from overlay

```bash $ eix --compact --installed --in-overlay steam-overlay [I] games-util/esteam [1] (0.20161113@11/14/2016): Scan Steam libraries to generate and emerge a Portage set [I] games-util/steam-client-meta [1] (0-r20160828@11/13/2016): Meta package for Valve's native Steam client [I] games-util/steam-launcher [1] (1.0.0.54-r1@01/23/2017): Installer, launcher and supplementary files for Valve's Steam client [I] games-util/steam-meta [1] (0-r20160817@11/14/2016): Meta package for Valve's Steam environment [1] "steam-overlay" /usr/local/portage/overlays/steam-overlay Found 4 matches ```

[chewi]

Hmm. Please give me a directory listing of /home/hoefling/steamlib/SteamApps/common. SteamApps may be lower case.

[hoefling]

$ ls -lah /home/hoefling/steamlib/steamapps/common
total 60K
drwxr-xr-x 15 hoefling hoefling 4.0K Jan 27 12:20 .
drwxr-xr-x  7 hoefling hoefling 4.0K Jan 26 22:06 ..
drwxr-xr-x  4 hoefling hoefling 4.0K Nov 27 20:27 'Broken Age'
drwxr-xr-x  7 hoefling hoefling 4.0K Jan 26 15:23 Cities_Skylines
drwxr-xr-x  6 hoefling hoefling 4.0K Dec  2 17:33 dont_starve
drwxr-xr-x  5 hoefling hoefling 4.0K Dec 20 11:51 'Mad Max'
drwxr-xr-x  5 hoefling hoefling 4.0K Nov 27 20:23 'NEON STRUCT'
drwxr-xr-x  3 hoefling hoefling 4.0K Nov 27 20:24 'Oh...Sir! The Insult Simulator'
drwxr-xr-x  4 hoefling hoefling 4.0K Jan 22 23:05 'Please, Don’t Touch Anything'
drwxr-xr-x  7 hoefling hoefling 4.0K Nov 27 20:49 Stacking
drwxr-xr-x 24 hoefling hoefling 4.0K Jan 27 11:47 Stellaris
drwxr-xr-x  6 hoefling hoefling 4.0K Jan 27 11:47 'The Binding of Isaac Rebirth'
drwxr-xr-x  9 hoefling hoefling 4.0K Dec 23 20:09 'This War of Mine'
drwxr-xr-x  5 hoefling hoefling 4.0K Nov 27 20:55 'Total War Attila'
drwxr-xr-x  6 hoefling hoefling 4.0K Jan 12 20:38 'Total War WARHAMMER'

Can spaces in dir names be an issue here? This is how steam downloaded them, never touched anything myself here.

[chewi]

Spaces are fine but I bet it's the !. I'll try fixing this at the weekend.

[hoefling]

So the Oh...Sir! The Insult Simulator is the bad boy? If you need it for testing, I have a copy to spare.

[chewi]

I could just create a directory with that name but it would be useful to test it anyway so I would be grateful. My Steam name is jerseychewi. Thanks!

[hoefling]

Added you on steam; apparently, I can send a gift only to steam friends :-)

[chewi]

I installed that and the error didn't appear. Then I figured it must be the ’ in Please, Don’t Touch Anything so I installed that too (already had it) and I still can't reproduce it. Weird. Your Bash is up to date. I'll have another look later.

[chewi]

You have LANG="en_US.utf8" but just to be sure, please give me the output of locale. It's probably not this but I can't think of anything else.

[hoefling]

That's the exact locale I have set:

$ eselect locale list 
Available targets for the LANG variable:
  [1]   C
  [2]   en_US
  [3]   en_US.iso88591
  [4]   en_US.utf8 *
  [5]   POSIX
  [ ]   (free form)

$ locale -v
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

[EoD]

I have the same issue and I have neither a ! nor a ` in my folder structure. I also have the an utf8 locale.

[chewi]

@EoD, mind sharing your directory listing?

[EoD]

@chewi For example the f2p game "Destination Sol" causes the issue.

125     # Reverse sort so that EM_X86_64 is handled before EM_386. This
126     # ensures that if both a 32-bit and 64-bit JRE are found within
127     # the same directory then 64-bit will take precedence.
128     SCAN_RESULT=$(find "${COMMON}" -type f -name libjvm.so -exec scanelf ${SCAN_ARGS} -BF $'%a\t%F' {} + | sort -r)
129 
130     IFS=$'\n'
131     for SCAN_LINE in ${SCAN_RESULT}; do
132         IFS=$'\t' read EM SCANNED_PATH <<< "${SCAN_LINE}"
133 
134         GAME=${SCANNED_PATH#${COMMON}}
135         GAME=${GAME%%/*}
136 
137         if [[ ! -e "${SCANNED_PATH}" || ${UNBUNDLEABLES_A[${GAME}]} != 1 ]]; then
138             continue
139         fi

SCAN_RESULT contains

EM_X86_64   /steam/SteamApps/common/Destination Sol/lwjreLinux64/lib/amd64/server/libjvm.so
EM_386  /steam/SteamApps/common/Destination Sol/lwjreLinux/lib/i386/server/libjvm.so
EM_386  /steam/SteamApps/common/Destination Sol/lwjreLinux/lib/i386/client/libjvm.so

The problem occurs with ${UNBUNDLEABLES_A[${GAME}] as ${GAME} evaluates to a blank because GAME=${GAME%%/*} removes everything.

What are you trying to achieve with ${GAME%%/*} in line 135?

[chewi]

Something awry here. COMMON is supposed to be /path/to/steam/SteamApps/common/. Is yours really at the top-level? It then strips this from the front of SCANNED_PATH on line 134, leaving Destination Sol/lwjreLinux64/lib/amd64/server/libjvm.so. Then it strips everything but the first component on line 135, leaving Destination Sol.

[EoD]

Yes, it is at the top level. I can confirm what you are are expecting, except the last step.

In order to show you some more details, I added the following echos to the bash script:

        SCAN_RESULT=$(find "${COMMON}" -type f -name libjvm.so -exec scanelf ${SCAN_ARGS} -BF $'%a\t%F' {} + | sort -r)

        echo "a1: ${COMMON}"
        echo "b1: ${SCAN_RESULT}"

        IFS=$'\n'
        for SCAN_LINE in ${SCAN_RESULT}; do
                IFS=$'\t' read EM SCANNED_PATH <<< "${SCAN_LINE}"

                echo "x1: ${GAME}"
                GAME=${SCANNED_PATH#${COMMON}}
                echo "x2: ${GAME}"
                GAME=${GAME%%/*}
                echo "x3: ${GAME}"

And there is the result:

 * Scanning /steam ...
a1: /steam
b1: EM_X86_64   /steam/SteamApps/common/Destination Sol/lwjreLinux64/lib/amd64/server/libjvm.so
EM_386  /steam/SteamApps/common/Destination Sol/lwjreLinux/lib/i386/server/libjvm.so
EM_386  /steam/SteamApps/common/Destination Sol/lwjreLinux/lib/i386/client/libjvm.so
x1: 
x2: /SteamApps/common/Destination Sol/lwjreLinux64/lib/amd64/server/libjvm.so
x3: 
/usr/bin/esteam: line 143: UNBUNDLEABLES_A: bad array subscript

EDIT: Adding a few more echos. I think we found the issue. $COMMON is not what you are expecting.

[chewi]

Hmm. How can COMMON be /steam when line 100 has this?

for COMMON in "${!DIRS[@]}"/steamapps/common/ "${!DIRS[@]}"/SteamApps/common/; do

I'll experiment later if you can't figure it out.

[EoD]

Indeed, there is the bug. As the expansion of ${!DIRS[@]} is done first, it will only append the /steamapps/common to the last element of the array.

You should be able to reproduce the bug, once you have set up several steam folders.

[chewi]

Great, I'll fix this tonight. I'll also try unbundling Destination Sol to see whether the Java unbundling works with that.

[chewi]

I should have realised sooner that someone has already submitted the fix for this in #180. I suck! On the plus side, I have fixed the Java unbundling so that Destination Sol uses a 64-bit rather than 32-bit JRE.