search

uccross / skyhookdm-ceph-cls

Skyhook Data Management: Storage and management of tabular data in Ceph.
https://www.skyhookdm.com
GNU Lesser General Public License v2.1
13 stars 9 forks source link

fix CI pipeline on centos-7.8 #30

Closed ivotron closed 4 years ago

ivotron commented 4 years ago

as part of #3 , we ported the CI pipeline of the luminous branch to centos-7.8. However, this is causing unit tests to fail (see travis#13). Error being thrown is -95, which it's apparently Operation not supported on transport endpoint.

ivotron commented 4 years ago

@jlefevre any hints of what could be causing this? Is there a way of getting verbose output? or can you share instructions on how to debug this type of issue?

jlefevre commented 4 years ago

I will take a look. I am not sure with nautilus, but this error appears due to the cls lib not being present or linked correctly to run-query. The first several test queries succeed, these are of the "NoCLS" flavor meaning they execute on the client side only. The first query that uses "CLS" (server side) fails. Last time I think this was due to a linking issue. To test you can run any query with the --use-cls flag and --debug flag.
for example this test query "b" bin/run-query --num-objs 10 --pool tpch --wthreads 1 --qdepth 10 --quiet --extended-price 91400.0 --query b --use-cls

Logging will be in the ceph logs of the osd, which you can grep for something like: grep "" /skyhookdm-ceph/build/out/osd.0.log

jlefevre commented 4 years ago

can you check if libcls_tabular getting copied to the correct location for centos? since the regular queries work without cls, the code is working (uses same code) but the cls liblocation is different than ubuntu. here is the dir for centos: https://github.com/uccross/skyhookdm-ceph/blob/skyhook-luminous/src/progly/cloudlab_cluster_setup/deploy_skyhooklib.sh#L22

also can you add this cloudlab_cluster_setup folder over to the cls repo?

ivotron commented 4 years ago

can you check if libcls_tabular getting copied to the correct location for centos? since the regular queries work without cls, the code is working (uses same code) but the cls liblocation is different than ubuntu. here is the dir for centos: https://github.com/uccross/skyhookdm-ceph/blob/skyhook-luminous/src/progly/cloudlab_cluster_setup/deploy_skyhooklib.sh#L22

yes, that it's done correctly https://github.com/uccross/skyhookdm-ceph-cls/blob/luminous/ci/Dockerfile.release#L12

also can you add this cloudlab_cluster_setup folder over to the cls repo?

is it ok if we add the cluster setup to the https://github.com/uccross/skyhookdm-workflows repo that contains deployment and experimentation scripts? that would allow us to keep this repo 1) small and 2) only focused on the library code. What I see is that that folder has bigish files that in the long run become a problem to deal with in Git. So another alternative is to only checkin scripts and leave tarballs out somewhere else.

jlefevre commented 4 years ago

yes the workflows dir is good, thanks.