Closed piyushgupta1551 closed 3 years ago
Update - I did manage to eventually build the uperf image on ppc64le. What I did additionally to make it happen -
atlas-devel libffi-devel gcc gcc-c++ gcc-gfortran libffi-devel make openssl-devel
(added them in line 3 of the Dockerfiles) - this seems to be a known requirement for ppc64le in general for both scipy & numpyRUN python3 -m pip install --upgrade cython numpy
importlib_metadata
& urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
(version clash and eventual failure while running workload if version specifics not added)Since this requirement is specific to ppc64le systems only, can we probably add this info as a small Note in the README doc somewhere for future reference?
Update - I did manage to eventually build the uperf image on ppc64le. What I did additionally to make it happen -
- Install a couple of rpm packages before scipy and numpy from the Dockerfile -
atlas-devel libffi-devel gcc gcc-c++ gcc-gfortran libffi-devel make openssl-devel
(added them in line 3 of the Dockerfiles) - this seems to be a known requirement for ppc64le in general for both scipy & numpy- I had to add this to the Dockerfile as the same somehow doesn't work by adding as dependency in setup.cfg -
RUN python3 -m pip install --upgrade cython numpy
- I had to add a couple of python packages to setup.cfg -
importlib_metadata
&urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
(version clash and eventual failure while running workload if version specifics not added)Since this requirement is specific to ppc64le systems only, can we probably add this info as a small Note in the README doc somewhere for future reference?
Interesting on the ppc64 specific needs. I wonder if we can have a uperf Dockerfile that is ppc64 specific?
Would you like to do a PR against benchmark-wrapper to include this ppc64. And if you have a container image, maybe you can share that, so I can push to Quay?
@jtaleric , yes I will raise the PR for ppc64le Dockerfiles. In my opinion all dockerfiles except iperf will need these additions to be ppc64le compliant. So far I've created 4 images for ppc64le -
How do you want me to add in the python packages that I added in setup.cfg? Should I add them in the Dockerfiles directly? Adding them in setup.cfg might create a conflict as they are not really required for x86.
Hi @jtaleric @vasukulkarni Below are the images that could be created for benchmark-operator and benchmark-wrapper workloads for ppc64le arch -
quay.io/piyushgupta1551/benchmark-operator:ppc64le
quay.io/piyushgupta1551/iperf3-m:ppc64le
quay.io/piyushgupta1551/fs-drift-m:ppc64le
quay.io/piyushgupta1551/fio-m:ppc64le
quay.io/piyushgupta1551/ycsb-m:ppc64le
quay.io/piyushgupta1551/smallfile-m:ppc64le
quay.io/piyushgupta1551/upgrade-m:ppc64le
quay.io/piyushgupta1551/scale-openshift-m:ppc64le
quay.io/piyushgupta1551/uperf-m:ppc64le
The remaining (oslat, hammerdb, sysbench, pgbench, cyclictest, vegeta) could not be created as dependent rpms, installers etc were not available for the required distros for ppc64le arch.
Open Issues for ppc64le workload images -
oslat - rt-tests-1.8-11.el8.x86_64.rpm not available for ppc64le for the required distro (Only OpenSUSE available)
hammerdb - https://github.com/TPC-Council/HammerDB/releases/download/v3.3/HammerDB-3.3-Linux-x86-64-Install' not available for ppc64le
sysbench - rpm not available for ppc64le for the required distro
stress-ng - not available for the required distro for ppc64le
pgbench - https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm - ppc64le available only for EL7
cyclictest - rt-tests-1.8-11.el8.x86_64.rpm available only for OpenSUSE distro for ppc64le
vegeta - github release not available for ppc64le
FYI @vasukulkarni
Hi Team, I am trying to build the uperf image from benchmark-wrapper repository for ppc64le but it keeps failing at the last step -
RUN pip3 install -e /opt/snafu/
with errors pointing to missing packages/dependencies as below -Adding these 'missing' packages is not straight as a new one shows up with every image build failure. The expectation was that all dependencies mentioned in setup.cfg would be sufficient but that isn't the case ppc64le. Main issue seems to be scipy & numpy packages.
As per a workaround suggested on perf-scale slack, I tried removing scipy & numpy in setup.cfg but it didn't help and the build failed with error as numpy is again required by prometheus_api_client -
Please help suggest a fix or a workaround for the same.