Openstudio 3.6.0 rpm cannot be installed properly under CentOS 7.

wenyikuang commented 6 months ago

Enhancement Request

Detailed Description

Under centos:7 After I downloaded the rpm from the release page. Then install by

yum install -y openstudio.rpm

will raise an exception for requires: /usr/local/bin/wish4.0.

[root@4cd97e37f9b7 Download]# yum install -y OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64.rpm
Loaded plugins: fastestmirror, ovl
Examining OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64.rpm: openstudio-3.6.0-3.6.0-1.x86_64
Marking OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package openstudio-3.6.0.x86_64 0:3.6.0-1 will be installed
--> Processing Dependency: /bin/csh for package: openstudio-3.6.0-3.6.0-1.x86_64
Loading mirror speeds from cached hostfile
 * base: mirror.web-ster.com
 * centos-sclo-rh: mirror.fcix.net
 * centos-sclo-sclo: mirror.umd.edu
 * epel: d2lzkl7pfhq30w.cloudfront.net
 * extras: mirror.siena.edu
 * updates: paducahix.mm.fcix.net
--> Processing Dependency: /usr/local/bin/wish4.0 for package: openstudio-3.6.0-3.6.0-1.x86_64
Error: Package: openstudio-3.6.0-3.6.0-1.x86_64 (/OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64)
           Requires: /usr/local/bin/wish4.0
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

If you run it with --skip-broken flag.

[root@4cd97e37f9b7 Download]# yum install -y OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64.rpm --skip-broken

It will not raise exception but will:

---> Package libXau.x86_64 0:1.0.8-2.1.el7 will be installed
---> Package libXrender.x86_64 0:0.9.10-1.el7 will be installed
---> Package libpng.x86_64 2:1.5.13-8.el7 will be installed
---> Package openstudio-3.6.0.x86_64 0:3.6.0-1 will be installed
--> Processing Dependency: /usr/local/bin/wish4.0 for package: openstudio-3.6.0-3.6.0-1.x86_64
--> Running transaction check
---> Package dejavu-sans-fonts.noarch 0:2.33-6.el7 will be installed
--> Processing Dependency: dejavu-fonts-common = 2.33-6.el7 for package: dejavu-sans-fonts-2.33-6.el7.noarch
---> Package fontpackages-filesystem.noarch 0:1.44-8.el7 will be installed
---> Package openstudio-3.6.0.x86_64 0:3.6.0-1 will be installed
--> Processing Dependency: /usr/local/bin/wish4.0 for package: openstudio-3.6.0-3.6.0-1.x86_64
--> Running transaction check
---> Package dejavu-fonts-common.noarch 0:2.33-6.el7 will be installed
---> Package openstudio-3.6.0.x86_64 0:3.6.0-1 will be installed
--> Processing Dependency: /usr/local/bin/wish4.0 for package: openstudio-3.6.0-3.6.0-1.x86_64
--> Processing Dependency: /usr/local/bin/wish4.0 for package: openstudio-3.6.0-3.6.0-1.x86_64

Packages skipped because of dependency problems:
    dejavu-fonts-common-2.33-6.el7.noarch from base
    dejavu-sans-fonts-2.33-6.el7.noarch from base
    fontconfig-2.13.0-4.3.el7.x86_64 from base
    fontpackages-filesystem-1.44-8.el7.noarch from base
    freetype-2.8-14.el7_9.1.x86_64 from updates
    libX11-1.6.7-4.el7_9.x86_64 from updates
    libX11-common-1.6.7-4.el7_9.noarch from updates
    libXau-1.0.8-2.1.el7.x86_64 from base
    libXft-2.3.2-2.el7.x86_64 from base
    libXrender-0.9.10-1.el7.x86_64 from base
    libicu-50.2-4.el7_7.x86_64 from base
    libicu-devel-50.2-4.el7_7.x86_64 from base
    2:libpng-1.5.13-8.el7.x86_64 from base
    libxcb-1.13-1.el7.x86_64 from base
    openstudio-3.6.0-3.6.0-1.x86_64 from /OpenStudio-3.6.0+860f5de185-CentOS-7.9.2009-x86_64
    1:tcl-8.5.13-8.el7.x86_64 from base
    tcsh-6.18.01-17.el7_9.1.x86_64 from updates
    1:tk-8.5.13-6.el7.x86_64 from base

Possible Implementation

Rebuilt from CentOS with newer depedency meta-info
Update CMakeList to catch up with the new wish dependency

wenyikuang commented 6 months ago

-- Library backtrace found /root/.conan/data/libbacktrace/cci.20210118/_/_/package/1d877a3df840030e6a8abb74c5ffb9088d08b47a/lib/libbacktrace.a
-- Conan: Adjusting language standard
-- Current conanbuildinfo.cmake directory: /Openstudio/build
-- Conan: Compiler GCC>=5, checking major version 8
-- Conan: Checking correct version: 8
-- openstudio: DONE RUNNING CONAN
CMake Error at CMakeLists.txt:648 (message):
  EnergyPlus no longer provides packages for Ubuntu 18.04

If I tried to build it from scratch, raise exception with E+ compatitable issue.

jmarrec commented 6 months ago

@wenyikuang I have this in the bash script I had used to build OpenStudio on a Centos docker.

cmake -G Ninja  [...truncated options...]

echo "Changing shebang line in radiance"
sed -i "s:#\!/usr/local/bin/wish4.0:#\!/usr/bin/env wish:g" radiance-5.0.a.12-Linux/usr/local/radiance/bin/trad

ninja

ninja package

jmarrec commented 6 months ago

cf https://github.com/jmarrec/docker-OpenStudio-CentOS/blob/4e4de0d797c82e7c9bdcc2a198d61b034c2eec1a/docker_container_script.sh#L39-L40

MatthewSteen commented 6 months ago

This is also a problem with 3.7.0. Autodesk Insight product uses the CentOS RPM builds.

jmarrec commented 6 months ago

@wenyikuang Do you want me to address this issue or are you clear? It requires

1) Either :

patching the jenkinsfiles for centos
Add a cmake install(CODE [[ ... ]]) command. This is probably prefered because it means it'll work no matter who builds it
Edit: actually, it's probably just easier and better to just change the radiance tar.gz we have hosted on s3://openstudio-resources to have the correct shebang

2) Rebuilding 3.7.0 centos7. (and maybe 3.6.0)

wenyikuang commented 6 months ago

@wenyikuang I have this in the bash script I had used to build OpenStudio on a Centos docker.

cmake -G Ninja  [...truncated options...]

echo "Changing shebang line in radiance"
sed -i "s:#\!/usr/local/bin/wish4.0:#\!/usr/bin/env wish:g" radiance-5.0.a.12-Linux/usr/local/radiance/bin/trad

ninja

ninja package

I see! Let me try it. Thanks!

wenyikuang commented 6 months ago

Emmmm even after I use the sed it still failed: (under the openstudio-cmake-tools:centos7 image)

When I executing ninja

[1235/9183] Building CXX object src/model/CMakeFi...rces.dir/ruby_OpenStudioModelResources_wrap.cxx.o
FAILED: src/model/CMakeFiles/ruby_OpenStudioModelResources.dir/ruby_OpenStudioModelResources_wrap.cxx.o
/opt/rh/devtoolset-10/root/usr/bin/g++ -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=\"/usr/bin/addr2line\" -DBOOST_STACKTRACE_USE_ADDR2LINE -DBOOST_STACKTRACE_USE_BACKTRACE -DBOOST_STACKTRACE_USE_NOOP -DGEOGRAPHICLIB_SHARED_LIB=0 -DHAVE_BZIP2 -DHAVE_UNISTD_H -DLIBXML_STATIC -DLIBXSLT_STATIC -D_GLIBCXX_USE_CXX11_ABI=0 -D_NO_ASYNCRTIMP -D_NO_PPLXIMP -I/Openstudio/src -I/Openstudio/build/src -I/Openstudio -isystem /opt/rh/rh-python38/root/usr/include/python3.8 -isystem /root/.conan/data/openstudio_ruby/2.7.2/nrel/centos/package/9e91ea8bf0bb98e1b35847c4567cb0a3682d2648/include -isystem /root/.conan/data/openstudio_ruby/2.7.2/nrel/centos/package/9e91ea8bf0bb98e1b35847c4567cb0a3682d2648/include/ruby-2.7.0 -isystem /root/.conan/data/openstudio_ruby/2.7.2/nrel/centos/package/9e91ea8bf0bb98e1b35847c4567cb0a3682d2648/include/ruby-2.7.0/x86_64-linux -isystem /root/.conan/data/sqlite3/3.38.5/_/_/package/b7e8530a7ce4dc98890a185b197886c549602572/include -isystem /root/.conan/data/jsoncpp/1.9.5/_/_/package/82ef5eac51c38971dea2fd342dd55ddf2ddfbbc3/include -isystem /root/.conan/data/minizip/1.2.12/_/_/package/50479a03e6a0f0ad7e9baeddeb9019d627aa692f/include -isystem /root/.conan/data/minizip/1.2.12/_/_/package/50479a03e6a0f0ad7e9baeddeb9019d627aa692f/include/minizip -isystem /root/.conan/data/zlib/1.2.12/_/_/package/19729b9559f3ae196cad45cb2b97468ccb75dcd1/include -isystem /root/.conan/data/bzip2/1.0.8/_/_/package/91a8b22c2c5a149bc617cfc06cdd21bf23b12567/include -isystem /root/.conan/data/fmt/9.1.0/_/_/package/1d886ef72d27d7ab9a1a64be277b3d28a290f3e4/include -isystem /root/.conan/data/pugixml/1.12.1/_/_/package/82ef5eac51c38971dea2fd342dd55ddf2ddfbbc3/include -isystem /root/.conan/data/boost/1.79.0/_/_/package/a762cbb0f4f91f9ac21c111bfd58400aa5d97a0b/include -isystem /root/.conan/data/libbacktrace/cci.20210118/_/_/package/19729b9559f3ae196cad45cb2b97468ccb75dcd1/include -isystem /root/.conan/data/cpprestsdk/2.10.18/_/_/package/336065edd5db47193312b10d982fa47c72ef8211/include -isystem /root/.conan/data/openssl/1.1.1o/_/_/package/19729b9559f3ae196cad45cb2b97468ccb75dcd1/include -isystem /root/.conan/data/websocketpp/0.8.2/_/_/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9/include -isystem /root/.conan/data/geographiclib/1.52/_/_/package/2066548edbbda32321eec39c17ddb5f988c7a268/include -isystem /root/.conan/data/libxml2/2.9.14/_/_/package/f9bf12b05a917e3b694bc35eb5037c71fbf6b7ca/include -isystem /root/.conan/data/libxml2/2.9.14/_/_/package/f9bf12b05a917e3b694bc35eb5037c71fbf6b7ca/include/libxml2 -isystem /root/.conan/data/libiconv/1.16/_/_/package/19729b9559f3ae196cad45cb2b97468ccb75dcd1/include -isystem /root/.conan/data/libxslt/1.1.34/_/_/package/d496057d7c19e1765bd39bf20d3383470a1603b0/include -fdiagnostics-color=always  -Wall -fPIC -fno-strict-aliasing -Winvalid-pch -Wnon-virtual-dtor -Wno-narrowing -Wno-narrowing -Werror -Wno-overloaded-virtual -Wno-maybe-uninitialized -O3 -DNDEBUG -std=c++2a -Wno-deprecated-declarations -Wno-sign-compare -Wno-register -Wno-conversion-null -Wno-misleading-indentation -fno-gnu-unique -MD -MT src/model/CMakeFiles/ruby_OpenStudioModelResources.dir/ruby_OpenStudioModelResources_wrap.cxx.o -MF src/model/CMakeFiles/ruby_OpenStudioModelResources.dir/ruby_OpenStudioModelResources_wrap.cxx.o.d -o src/model/CMakeFiles/ruby_OpenStudioModelResources.dir/ruby_OpenStudioModelResources_wrap.cxx.o -c /Openstudio/build/src/model/ruby_OpenStudioModelResources_wrap.cxx
g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
[1244/9183] Building CXX object src/model/CMakeFi...ModelHVAC.dir/ruby_OpenStudioModelHVAC_wrap.cxx.o
ninja: build stopped: subcommand failed.

wenyikuang commented 6 months ago

Emmmm and I tried to run the script in docker-openstudio-centos but not works well. Does it need any pre-request?

Thanks,

jmarrec commented 6 months ago

That error has nothing to do with the shebang. It happens because it likely runs out of memory during the linking phase.

g++: fatal error: Killed signal terminated program cc1plus

Try redoing ninja or even ninja -j 1. And you should go check your docker settings allocations for memory and cpu and see if they make sense.

Your ruby bindings are HUGE files and so it bites through the available memory quickly when done in //. I have an ubuntu latptop with 12 threads and only 16 GB of ram and I get this issue on it. In my case it rarely crashes, because I have a large swap, but the system is often unresponsive

wenyikuang commented 6 months ago

That error has nothing to do with the shebang. It happens because it likely runs out of memory during the linking phase.
g++: fatal error: Killed signal terminated program cc1plus
Try redoing ninja or even ninja -j 1. And you should go check your docker settings allocations for memory and cpu and see if they make sense.

Your ruby bindings are HUGE files and so it bites through the available memory quickly when done in //. I have an ubuntu latptop with 12 threads and only 16 GB of ram and I get this issue on it. In my case it rarely crashes, because I have a large swap, but the system is often unresponsive

Yeah finally I made it(build successfully) with limiting the ninja -j 4 and larger ram.

And face the same issue:

Error: Package: openstudio-3.7.0-3.7.0-1.x86_64 (/OpenStudio-3.7.0+1ce97f5651-CentOS-7.9.2009-x86_64)
           Requires: /usr/local/bin/wish4.0
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

jmarrec commented 6 months ago

Cd build/_CPack_Packages Grep -R wish

I reckon the sed command didn't do the replace.

wenyikuang commented 6 months ago

I reckon the sed command didn't do the replace.

And I didn't find the wish as an interpretor under the /env/ dir.

the result from grep is (I directly grep wish4.0 to skip some random stuff)

root@_CPack_Packages$ grep -R wish4.0
Binary file Linux/RPM/RPMS/OpenStudio-3.7.0+1ce97f5651-CentOS-7.9.2009-x86_64.rpm matches
Linux/RPM/OpenStudio-3.7.0+1ce97f5651-CentOS-7.9.2009-x86_64/usr/local/openstudio-3.7.0/Radiance/bin/trad:#!/usr/local/bin/wish4.0
Linux/RPM/rpmbuildopenstudio-3.7.0.out:Requires: /bin/csh /usr/bin/perl /usr/local/bin/wish4.0 ld-linux-x86-64.so.2()(64bit) ld-linux-x86-64.so.2(GLIBC_2.2.5)(64bit) ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libX11.so.6()(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.10)(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.15)(64bit) libc.so.6(GLIBC_2.17)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.7)(64bit) libc.so.6(GLIBC_2.8)(64bit) libc.so.6(GLIBC_2.9)(64bit) libcrypt.so.1()(64bit) libdl.so.2()(64bit) libdl.so.2(GLIBC_2.2.5)(64bit) libenergyplusapi.so.23.2.0()(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libgcc_s.so.1(GCC_3.4)(64bit) libgcc_s.so.1(GCC_4.0.0)(64bit) libgomp.so.1()(64bit) libgomp.so.1(GOMP_4.0)(64bit) libgomp.so.1(OMP_1.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libopenstudiolib.so()(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.12)(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpthread.so.0(GLIBC_2.3.2)(64bit) libpthread.so.0(GLIBC_2.3.3)(64bit) libpython3.8.so.rh-python38-1.0()(64bit) librt.so.1()(64bit) librt.so.1(GLIBC_2.2.5)(64bit) librt.so.1(GLIBC_2.3.3)(64bit) libstdc++.so.6()(64bit) libstdc++.so.6(CXXABI_1.3)(64bit) libstdc++.so.6(CXXABI_1.3.1)(64bit) libstdc++.so.6(CXXABI_1.3.2)(64bit) libstdc++.so.6(CXXABI_1.3.3)(64bit) libstdc++.so.6(CXXABI_1.3.5)(64bit) libstdc++.so.6(GLIBCXX_3.4)(64bit) libstdc++.so.6(GLIBCXX_3.4.11)(64bit) libstdc++.so.6(GLIBCXX_3.4.13)(64bit) libstdc++.so.6(GLIBCXX_3.4.14)(64bit) libstdc++.so.6(GLIBCXX_3.4.15)(64bit) libstdc++.so.6(GLIBCXX_3.4.17)(64bit) libstdc++.so.6(GLIBCXX_3.4.18)(64bit) libstdc++.so.6(GLIBCXX_3.4.19)(64bit) libstdc++.so.6(GLIBCXX_3.4.5)(64bit) libstdc++.so.6(GLIBCXX_3.4.6)(64bit) libstdc++.so.6(GLIBCXX_3.4.9)(64bit) libutil.so.1()(64bit) libutil.so.1(GLIBC_2.2.5)(64bit) perl(File::Temp) perl(Math::Trig) perl(POSIX) perl(strict) perl(warnings) rtld(GNU_HASH)
Linux/TGZ/OpenStudio-3.7.0+1ce97f5651-CentOS-7.9.2009-x86_64/usr/local/openstudio-3.7.0/Radiance/bin/trad:#!/usr/local/bin/wish4.0```

Tried to modify radiance-5.0.a.12-Linux/usr/local/radiance/bin/trad in vi and build again, let's see.

wenyikuang commented 6 months ago

Nice, after I modify the radiance/../trad with vi, the installation works!

TODO:

Fix the sed script
Fix the CI to build the package correctly.

jmarrec commented 6 months ago

#!/usr/bin/env xxxx [opts] is a shebang that just instructs the terminal to look for the executable xxx in the PATH (and potentially pass opts to it)

It basically resolves it like which xxx would (with the caveat that it also evaluates anything you have aliased first)

wenyikuang commented 6 months ago

get it, but why does it comes from wish4.0? it looks something really old.

and the sed should be:

sed -i 's:^#!/usr/local/bin/wish4.0:#!/usr/bin/env wish:' radiance-5.0.a.12-Linux/usr/local/radiance/bin/trad

jmarrec commented 6 months ago

We haven't update Radiance in the last 10 years I think We should patch the tar.gz and reupload to s3 to avoid bumping into this again.

As far as sed, it's a nifty but confusing tool, in the sense that there are often major differences in which flavor is installed. The freedesktop one for eg does not support -i without a suffix. And that's the one shipped on many older systems and on macos. Got bit by that this week again while creating a CI workflow.

jmarrec commented 6 months ago

Also I don't think we ever use that file because afaik wish is a gui. https://en.m.wikipedia.org/wiki/Wish_(Unix_shell)

MatthewSteen commented 6 months ago

Here's the Radiance source, which was changed from wish4.0 in 2019...

https://github.com/LBNL-ETA/Radiance/blob/master/src/util/trad.wsh

NREL / OpenStudio