POWER8 IBM - i386 Intel SSE headers

[polsm91]

Hi, We are trying to deploy Scylla on a Power8 system (PPC64). However, we've run into problems when building seastar (commit 5088065a503392135953854adb0db920759317f8).

These files: https://github.com/scylladb/seastar/blob/master/util/spinlock.hh#L25 https://github.com/scylladb/seastar/blob/master/core/reactor.cc#L92 (#L2804)

use SSE instructions (at least _mm_pause), included in the header "xmmintrin.h" (i386 header, Intel SSE instructions).

Could it be possible to replace these instructions with architecture specific?

[penberg]

I don't think there are portable versions of _mm_pause(). We can create our own wrapper similar to Linux cpu_relax(), of course.

/cc @avikivity

[avikivity]

Yup. For starters, you can just #ifdef it out, though it's not good for SMT.

You will run into another problem as we use the Intel crc32 instruction (for generating crc32c checksums). I don't think there is a drop-in replacement on ppc64, so we'll have to do something more involved.

Lastly, I recommend using little endian mode. While we've been trying to keep the code endian-clean, it was never tested. Best to start with little endian and switch to big endian after it works.

[polsm91]

Thanks for your suggestions, we took the approach @avikivity suggested for the _mm_pause instruction. Currently we are solving other compilations errors related to dependencies and versions, when we get to the crc32 problem we will get back to you.

[Fale]

Is there any news on this?

[dorlaor]

Yes, it should be contributed any time. I'll check

On Sun, Oct 1, 2017 at 11:58 AM, Fabio Alessandro Locati < notifications@github.com> wrote:

Is there any news on this?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/scylladb/seastar/issues/270#issuecomment-333398434, or mute the thread https://github.com/notifications/unsubscribe-auth/ABp6RcvenrVoB1aaEKrNMcMxNvjacfR4ks5sn-C5gaJpZM4Mvop2 .

[vladzcloudius]

@polsm91 @Fale There is a work-in-progress Power8 port effort. I'll work on integrating this work in our trees but for now you can get it as follows:

1) Clone the scylla's master branch in a regular way: 1) git clone https://github.com/scylladb/scylla.git 2) cd scylla 3) git submodule update --init --recursive

2) Add my repos as remotes to `scylla` and `seastar` correspondingly:
    1) git remote add vladz https://github.com/vladzcloudius/scylla.git repository
    2) cd seastar
    3) git remote add vladz https://github.com/vladzcloudius/seastar.git
3) Fetch and checkout the branches above:
    1) cd ..
        (you should be in 'scylla' directory now.)
    2) git fetch vladz
    3) git checkout ppc64_port_scylla-5
    4) cd seastar
    5) git fetch vladz
    6) git checkout ppc64_port-5
    7) sudo ./install-dependencies.sh
    8) cd ..
    9) sudo ./install-dependencies.sh
4) Configure and compile scylla:
    1) ./configure.py --huge-page-size-mb=<huge page size in MB on your platform> --target="" --page-bits=12
    Note that with the parameters above seastar is going to use 4K pages for internal memory management and that's ok. There is some problem to use the native page and I'm going to hunt it down soon.
    2) ninja ./build/release/scylla (or whatever you want to build)
    3) Enjoy!

[polsm91]

@vladzcloudius Thanks for your efforts, we will try and let you know if we encounter any issue.

[polsm91]

@vladzcloudius

We mostly work with supercomputers without containers, so doing a sudo install is not an option nor using the system package manager. That's why it takes so long for us to build Scylla. We mostly rely on manual installing or the LLNL/Spack project to install dependencies. I am not done installing dependencies but noticed a couple of things:

The scylla's configure script assumes Python3 is in /usr/bin. I ran it with Python3 and removed that line. but got the error: Configure scylla: error: argument --huge-page-size-mb: invalid int value: ''

[vladzcloudius]

On Oct 28, 2017 6:25 AM, "polsm91" notifications@github.com wrote:

@vladzcloudius https://github.com/vladzcloudius

We mostly work with supercomputers without containers, so doing a sudo install is not an option nor using the system package manager. That's why it takes so long for us to build Scylla.

We are working on creating proper packaging for Power8 but it's not ready yet.

We mostly rely on manual installing or the LLNL/Spack project to install dependencies. I am not done installing dependencies but noticed a couple of things:

The scylla's configure script assumes Python3 is in /usr/bin. I ran it with Python3 and removed that line. but got the error: Configure scylla: error: argument --huge-page-size-mb: invalid int value: ''

It seems like you didn't give this parameter a value and it's mandatory.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scylladb/seastar/issues/270#issuecomment-340190699, or mute the thread https://github.com/notifications/unsubscribe-auth/AF0jrdCrlDWjbdQiFdUbG6jbJK32gwywks5swyszgaJpZM4Mvop2 .

[polsm91]

Hi @vladzcloudius,

I apologize for not coming back to you earlier. We successfully deployed ScyllaDB on the Power 8 cluster (high-density memory and NVME) a few months ago. We observed a write performance of 600 kiops deploying Scylla in a single node, measured with the cassandra-stress tool and the model map<<long,long>,long>. Cassandra performed at 160 kiops under the same conditions.

We encountered some problems during the installation which were solved by downloading the RPM packages to a local folder, and then uncompressing, instead of manually installing the dependencies.

Thank you very much for your support!

[vladzcloudius]

On 05/17/2018 06:45 AM, polsm91 wrote:

Hi @vladzcloudius https://github.com/vladzcloudius,

I apologize for not coming back to you earlier. We successfully deployed ScyllaDB on the Power 8 cluster (high-density memory and NVME) a few months ago. We observed a write performance of 600 kiops deploying Scylla in a single node, measured with the cassandra-stress tool and the model map<<long,long>,long>. Cassandra performed at 160 kiops under the same conditions.

We encountered some problems during the installation which were solved by downloading the RPM packages to a local folder, and then uncompressing, instead of manually installing the dependencies.

Thank you very much for your support!

Hi! I'm glad you are enjoying our product. If you would like me to take a look in order to estimate if scylla performance can get even better just let me know.

Best regards, Vlad

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/scylladb/seastar/issues/270#issuecomment-389824780, or mute the thread https://github.com/notifications/unsubscribe-auth/AF0jrerUC5zQ2XByKTRJBNfCkIDbwZE9ks5tzVS3gaJpZM4Mvop2.

[polsm91]

Hi! It is good to know that we can come to you to tune the performance. We are satisfied with the current results but we still need to test Scylla under a variety of scenarios. If there is anything we will let you know.

Best, Pol