OS not supported - Githubissues

jorpilo commented 7 years ago

I am using: CentOS Linux release 7.4.1708 Kernel: 4.13.4-1.el7.elrepo.x86_64

When trying to create a GW it says Failed : iscsi0 failed package validation checks - OS is unsupported

why can it be, what OS do I need?

lxbsz commented 7 years ago

From rbd-target-api.py, you can see that:

valid_dists = ["redhat"]
valid_versions = ['7.4']

required_rpms = [
    {"name": "python-rtslib",
     "version": "2.1.fb64",
     "release": "0.1"},
    {"name": "tcmu-runner",
     "version": "1.3.0",
     "release": "0.2.3"}
]

k_vers = '3.10.0'
k_rel = '695.el7'

for now only "redhat" OS is supported.

jorpilo commented 7 years ago

Totally true, mainline checks for RedHat while 2.5 release doesn't do that checking

mikechristie commented 7 years ago

We are working on removing those checks. The code was originally tested/written against RHEL.

If you have a patch to check for upstream packages feel free to submit it. Ideally, we get rid of version checks and we can just dynamically detect if the kernel and various libs/tools supports what we need.

lxbsz commented 7 years ago

@mikechristie

k_rel = '695.el7',

One more question, does this kernel version has backported all the LIO/TCMU patches needed ?

mikechristie commented 7 years ago

No. We hit some issues with it. We are working on a new RHEL kernel. It's not done so I do not have the new version yet.

shin2344234 commented 7 years ago

I have installed cli from source and have it almost working save for:

Failed : sbd-cephnode01 failed package validation checks - OS is unsupported,Kernel version too old - 3.10.0-695.el7 or above needed

When I try to add a gateway.

Is there any way to get a workaround for this temporarily, for centos, so I can test it? I'm not a python coder so I don't want to start mucking around.

Kernel/dist is:

CentOS Linux release 7.4.1708 (Core) Linux sbd-cephnode01 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

rdoerr01 commented 6 years ago

I'm also trying to do a proof of concept using CEPH Luminous with iSCSI on CentOS 7.4. I checked the values returned by Python for "dist, rel, dist_id = platform.linux_distribution(full_distribution_name=0)" and then are in order "centos", "7.4.1708" and "Core". I even tried to modify the Python script at "/usr/bin/rbd-target-api", but even adding a simple "logger.info" doesn't execute, so I'm thinking the file must need to be modified elsewhere and then is compiled compress and storage elsewhere when you run "gwcli" since it doesn't execute this line of code. cat /etc/centos-release: CentOS Linux release 7.4.1708 (Core) uname -a: Linux ceph-mon-1 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

zerocarbthirty commented 6 years ago

Can we maybe get someone to put in the docs that this absolutely only works in RHEL 7.4 so that people don't waste large swaths of time trying to get this running in CentOS?

jorpilo commented 6 years ago

I does work with others OS but kernel and code fixes are needed. I dont even know if it totally works for RHEL 7.4 as kernel fixes are planned for 7.5 http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-November/022779.html

mikechristie commented 6 years ago

Hi,

For the distro issue you should be able to use skipchecks=true when setting up the gateway. I also just did a PR to remove the RH checks from that function and make them more generic:

https://github.com/ceph/ceph-iscsi-cli/pull/53

There is sill one piece you will be missing and that is the kernel. Without the patches kernel you will get errors during device setup about the kernel being unsupported. I am trying to get a upstream kernel uploaded with the required patches to github.com/ceph/ceph-client by tomorrow so you can test against that. I will update this Issue when that gets done.

mikechristie commented 6 years ago

Oh yeah, if you find more RH distro specific checks please let me know and I will kill them.

mikechristie commented 6 years ago

I am trying to get a upstream kernel uploaded with the required patches to github.com/ceph/ceph-client by tomorrow

Hey, Just a quick update. A bug was found with the upstream code. I am trying to get it fixed and this stuff pushed by Wed.

gelato commented 6 years ago

Or... You could just get them from here: https://shaman.ceph.com/repos/ceph-iscsi-config/master/ https://shaman.ceph.com/repos/ceph-iscsi-cli/master/

gelato commented 6 years ago

All of them, mate... https://shaman.ceph.com/ You can install everything related to ceph iscsi gateway from there.

jorpilo commented 6 years ago

Thanks a lot for the repo. Really useful. Are all the rpms updated? Do main kernels there have all the patches needed to run ceph-iscsi?

mikechristie commented 6 years ago

The kernels are not updated. I am trying to get it uploaded this week. Note that the updated upstream kernel will require a updated tcmu-runner which will be available on runner's github if you are using releases from there.

99% testing is now done. I am just building packages and trying to finish testing a tcmu-runner patch to add support for ESX ATS/COMPARE_AND_WRITE offload support.

squarejester commented 6 years ago

Thanks Gelato for going over the shaman repositories. It's like a light went off in my head. I see everything now and OMG! You even have ceph at version 13 in there. However, I did notice that it does not have anything for python-rtslib. Since that is a prerequisite to make anything work why is it not there as well.

squarejester commented 6 years ago

Any word on when we can expect the new kernel? I see that its what is needed to solve a number of issues that are currently out there.

mikechristie commented 6 years ago

I am still working on the last bug mentioned in my last comment.

Hoping this week. Hit a bug/regression in the iscsi code that is going to take a while to fix with 4.14 so rebasing to 4.13.

I will promose that I will not go on Christmas/New year break until it is done!

jorpilo commented 6 years ago

Thanks for your effort, I am willing to see the iSCSI as a Christmas gift!

If there is anything I could help I would love to get my hands dirty

mikechristie commented 6 years ago

Just FYI, here is a status update posted to ceph-users.

I am just waiting for the upstream target layer maintainer to at least say that the kernel/user API being added in this patchset

https://www.spinics.net/lists/target-devel/msg16372.html

is ok. When I at least get that, I have a kernel that I can push to

https://github.com/ceph/ceph-client

with everything needed.

A updated tcmu-runner rpm will also needed, but I am the maintainer and that is just waiting for the kernel patches to get the OK.

zerocarbthirty commented 6 years ago

Mike we appreciate your efforts.

On Dec 22, 2017 5:49 PM, "mikechristie" notifications@github.com wrote:

Just FYI, here is a status update posted to ceph-users.

I am just waiting for the upstream target layer maintainer to at least say that the kernel/user API being added in this patchset

https://www.spinics.net/lists/target-devel/msg16372.html

is ok. When I at least get that, I have a kernel that I can push to

https://github.com/ceph/ceph-client

with everything needed.

A updated tcmu-runner rpm will also needed, but I am the maintainer and that is just waiting for the kernel patches to get the OK.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-353686565, or mute the thread https://github.com/notifications/unsubscribe-auth/AP5WwsECBpgf702h98-RbKcffMRfJ9zaks5tDDH2gaJpZM4Pu8bq .

cookeem commented 6 years ago

Same issue, I have try use https://github.com/ceph/ceph-iscsi-config and https://github.com/ceph/ceph-iscsi-cli master repo, and I have tried different types of OS, but both are failed:

[root@devops-ceph01 ~]# uname -sr
Linux 4.14.8-1.el7.elrepo.x86_64
[root@devops-ceph01 ~]# cat /etc/redhat-release 
CentOS Linux release 7.4.1708 (Core) 

[root@devops-ceph03 ~]# uname -sr
Linux 3.10.0-693.el7.x86_64
[root@devops-ceph03 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.4 (Maipo)

My package:

[root@devops-ceph03 ~]# pip show targetcli-fb
---
Metadata-Version: 1.1
Name: targetcli-fb
Version: 2.1.fb47

[root@devops-ceph03 ~]# pip show rtslib-fb
---
Metadata-Version: 1.1
Name: rtslib-fb
Version: 2.1.64

[root@devops-ceph03 ~]# yum info tcmu-runner
已加载插件：fastestmirror, product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Loading mirror speeds from cached hostfile
已安装的软件包
名称    ：tcmu-runner
架构    ：x86_64
版本    ：v1.3.0
发布    ：0.4.11.g618f619
大小    ：673 k
源    ：installed

[root@devops-ceph03 ~]# pip list | grep ceph-iscsi
ceph-iscsi-config (2.3)

cat /var/log/rbd-target-api.log

My log show this:

2017-12-28 16:10:28,222    DEBUG [common.py:147:_get_ceph_config()] - (_get_rbd_config) config object contains '{
    "clients": {},
    "created": "2017/12/28 07:48:20",
    "disks": {},
    "epoch": 4,
    "gateways": {
        "created": "2017/12/28 07:48:51",
        "devops-ceph03": {
            "active_luns": 0,
            "created": "2017/12/28 07:49:11",
            "gateway_ip_list": [
                "192.168.20.15",
                "192.168.20.16"
            ],
            "inactive_portal_ips": [
                "192.168.20.16"
            ],
            "iqn": "iqn.2003-01.com.redhat.iscsi-gw:ceph-gw",
            "portal_ip_address": "192.168.20.15",
            "tpgs": 2,
            "updated": "2017/12/28 07:49:16"
        },
        "devops-ceph04": {
            "active_luns": 0,
            "created": "2017/12/28 07:49:17",
            "gateway_ip_list": [
                "192.168.20.15",
                "192.168.20.16"
            ],
            "inactive_portal_ips": [
                "192.168.20.15"
            ],
            "iqn": "iqn.2003-01.com.redhat.iscsi-gw:ceph-gw",
            "portal_ip_address": "192.168.20.16",
            "tpgs": 2,
            "updated": "2017/12/28 07:49:17"
        },
        "ip_list": [
            "192.168.20.15",
            "192.168.20.16"
        ],
        "iqn": "iqn.2003-01.com.redhat.iscsi-gw:ceph-gw"
    },
    "groups": {},
    "updated": "2017/12/28 07:49:17",
    "version": 3
}'
2017-12-28 16:10:28,235    DEBUG [lun.py:328:allocate()] - LUN.allocate starting, listing rbd devices
2017-12-28 16:10:28,250    DEBUG [lun.py:331:allocate()] - rados pool 'rbd' contains the following - [u'rbda']
2017-12-28 16:10:28,250    DEBUG [lun.py:336:allocate()] - Hostname Check - this host is devops-ceph03, target host for allocations is devops-ceph03
2017-12-28 16:10:28,311    DEBUG [common.py:249:add_item()] - (Config.add_item) config updated to {u'updated': u'2017/12/28 07:49:17', u'disks': {'rbd.rbda': {'created': '2017/12/28 08:10:28'}}, u'created': u'2017/12/28 07:48:20', u'clients': {}, u'epoch': 4, u'version': 3, u'gateways': {u'devops-ceph04': {u'gateway_ip_list': [u'192.168.20.15', u'192.168.20.16'], u'active_luns': 0, u'created': u'2017/12/28 07:49:17', u'updated': u'2017/12/28 07:49:17', u'iqn': u'iqn.2003-01.com.redhat.iscsi-gw:ceph-gw', u'inactive_portal_ips': [u'192.168.20.15'], u'portal_ip_address': u'192.168.20.16', u'tpgs': 2}, u'iqn': u'iqn.2003-01.com.redhat.iscsi-gw:ceph-gw', u'ip_list': [u'192.168.20.15', u'192.168.20.16'], u'devops-ceph03': {u'gateway_ip_list': [u'192.168.20.15', u'192.168.20.16'], u'active_luns': 0, u'created': u'2017/12/28 07:49:11', u'updated': u'2017/12/28 07:49:16', u'iqn': u'iqn.2003-01.com.redhat.iscsi-gw:ceph-gw', u'inactive_portal_ips': [u'192.168.20.16'], u'portal_ip_address': u'192.168.20.15', u'tpgs': 2}, u'created': u'2017/12/28 07:48:51'}, u'groups': {}}
2017-12-28 16:10:28,312    DEBUG [lun.py:392:allocate()] - Check the rbd image size matches the request
2017-12-28 16:10:28,336    DEBUG [lun.py:413:allocate()] - rbd image rbd.rbda size matches the configuration file request
2017-12-28 16:10:28,336    DEBUG [lun.py:415:allocate()] - Begin processing LIO mapping
2017-12-28 16:10:28,337     INFO [lun.py:606:add_dev_to_lio()] - (LUN.add_dev_to_lio) Adding image 'rbd.rbda' to LIO
2017-12-28 16:10:28,398    ERROR [lun.py:639:add_dev_to_lio()] - Could not set LIO device attribute cmd_time_out/qfull_time_out for device: rbd.rbda. Kernel not supported. - error(Cannot find attribute: qfull_time_out)
2017-12-28 16:10:28,403    ERROR [rbd-target-api:730:_disk()] - LUN alloc problem - Could not set LIO device attribute cmd_time_out/qfull_time_out for device: rbd.rbda. Kernel not supported. - error(Cannot find attribute: qfull_time_out)
2017-12-28 16:10:28,405     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [28/Dec/2017 16:10:28] "PUT /api/_disk/rbd.rbda HTTP/1.1" 500 -
2017-12-28 16:10:28,406    ERROR [rbd-target-api:1429:call_api()] - _disk change on 127.0.0.1 failed with 500
2017-12-28 16:10:28,406    DEBUG [rbd-target-api:1446:call_api()] - failed on devops-ceph03. LUN allocation failure
2017-12-28 16:10:28,407     INFO [_internal.py:87:_log()] - 127.0.0.1 - - [28/Dec/2017 16:10:28] "PUT /api/disk/rbd.rbda HTTP/1.1" 500 -
2017-12-28 16:10:29,701     INFO [_internal.py:87:_log()] - 192.168.20.15 - - [28/Dec/2017 16:10:29] "GET /api/_ping HTTP/1.1" 200 -
2017-12-28 16:10:39,701     INFO [_internal.py:87:_log()] - 192.168.20.15 - - [28/Dec/2017 16:10:39] "GET /api/_ping HTTP/1.1" 200 -

shin2344234 commented 6 years ago

@mikechristie Thanks for all the work! When this is working on centos 7 I think we will all be really grateful.

donnydavis commented 6 years ago

If you are really determined, you can install elrepo kernel-ml to get the correct kernel version and then patch /usr/bin/rbd-target-api

valid_dists = ["centos"]
valid_versions = ['7.4.1708']

Still have to get python-rtslib and tcmu-runner patched, but you can pull those source rpms and build them...

Not sure of the implications of all of this, so good luck

zerocarbthirty commented 6 years ago

Actually elrepo latest doesnt have the changes needed but I patched the changes into this kernel and it has the RPMSs and the whole build tree.

enjoy

https://1drv.ms/u/s!Ao0RqVkLrZ91kxuuetSexhRSCKc5

On Dec 30, 2017 9:19 AM, "Donny Davis" notifications@github.com wrote:

If you are really determined, you can install elrepo kernel-ml to get the correct kernel version and then patch /usr/bin/rbd-target-api

valid_dists = ["centos"] valid_versions = ['7.4.1708']

Still have to get python-rtslib and tcmu-runner patched, but you can pull those source rpms and build them...

Not sure of the implications of all of this, so good luck

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-354548583, or mute the thread https://github.com/notifications/unsubscribe-auth/AP5Wwt9JA17jahNWL7gGhrtJHaRPl5pQks5tFkZ4gaJpZM4Pu8bq .

cotma commented 6 years ago

Happy New Year ladies and gents,

Just compiled Kernel 4.14.10 with the 19 patches. Got rid of that OS unsupported insufferable only to now get a Malformed REST API response when creating the initial gateway.

Did I miss a step? could I have not created or configured something in Ceph? I did see someone mentionning RBD pool / storage / thingny was named incorrectly on his setup, well, mine is rbd and not rdb... as I usually do.

any other slipups I might have done, any logs or config you'd like to see to TS.?

Thanks so much for all the work, we are so hopeful for this product. Ideally, a good integration with Hyper-V since it's our main plateform, personally I see much more with Ceph but we need to start somewhere.

Marc

mikechristie commented 6 years ago

@cotma For your specific bug could you open a new issue with the /var/log/rbd-target-api.log and /var/log/rbd-target-gw.log system log /var/log/messages attached for the period you hit the error.

Also, there are 20 patches that I am waiting on getting merged upstream:

https://www.spinics.net/lists/target-devel/msg16283.html https://www.spinics.net/lists/target-devel/msg16372.html

You probably just do not have https://www.spinics.net/lists/target-devel/msg16286.html which was posted on the list by another user. Note that the patch has nothing to do with the error you saw.

You will also need a updated tcmu-runner which this patchset:

https://github.com/open-iscsi/tcmu-runner/pull/339

That is not related to your issue, but has fixes for some issues you could hit later.

cotma commented 6 years ago

Good to know, I'll collect the info tomorrow morning and post.

Thanks.

M.

On Tue, Jan 2, 2018 at 4:33 PM, mikechristie notifications@github.com wrote:

@cotma https://github.com/cotma For your specific bug could you open a new issue with the /var/log/rbd-target-api.log and /var/log/rbd-target-gw.log system log /var/log/messages attached for the period you hit the error.

Also, there are 20 patches that I am waiting on getting merged upstream:

https://www.spinics.net/lists/target-devel/msg16283.html https://www.spinics.net/lists/target-devel/msg16372.html

You probably just do not have https://www.spinics.net/lists/target-devel/msg16286.html which was posted on the list by another user. Note that the patch has nothing to do with the error you saw.

You will also need a updated tcmu-runner which this patchset:

open-iscsi/tcmu-runner#339 https://github.com/open-iscsi/tcmu-runner/pull/339

That is not related to your issue, but has fixes for some issues you could hit later.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-354880985, or mute the thread https://github.com/notifications/unsubscribe-auth/Aha6QhBBupIHoiX6T1oZmjeHkGc3KobAks5tGqDDgaJpZM4Pu8bq .

mikechristie commented 6 years ago

Hi,

All the required patches are now merged upstream and the docs are almost all updated (just missing a updated package version but that info is below) . If you are using upstream packages, instructions for setup can be found here:

http://docs.ceph.com/docs/master/rbd/iscsi-overview/

If you are using packages built by a distro then you should use their documentation, because they may have added extra patches.

If you want the most cutting edge development kernel you can get the needed patches from the target layer maintainer's tree here:

https://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending.git/log/?h=for-next

I would only use that if you are doing kernel development. I am not sure if the target maintainer is going to push the patches to linus for 4.16 or 4.17, so I put all the required patches in a tree based on 4.15 here:

https://github.com/ceph/ceph-client/tree/ceph-iscsi-test

Centos/RHEL and ubuntu kernel builds are here:

[Make sure you check the dates on the builds. The builds from 10 days ago are for 4.13 and the builds from today are for 4.15] https://shaman.ceph.com/repos/kernel/ceph-iscsi-test

If you build the kernel yourself you need these items enabled:

CONFIG_TARGET_CORE=m CONFIG_TCM_USER2=m CONFIG_ISCSI_TARGET=m

There are more instructions for building all the needed packages here http://docs.ceph.com/docs/master/rbd/iscsi-target-cli-manual-install/

Warning: The userspace requirement info in the docs lists a older cpeh-iscsi-cli version. You must update tcmu-runner, ceph-iscsi-cli and ceph-iscsi-config packages to the newest versions:

tcmu-runner 1.3.0: https://github.com/open-iscsi/tcmu-runner/releases
ceph-iscsi-config 2.4: https://github.com/ceph/ceph-iscsi-config/releases
ceph-iscsi-cli 2.6: https://github.com/ceph/ceph-iscsi-cli/releases

The -fb versions of rtslib, targetcli and configshell. They can be found here: https://github.com/open-iscsi/

If you build these by hand then make sure you follow the manual doc: http://docs.ceph.com/docs/master/rbd/iscsi-target-cli-manual-install/ because there are some awkward steps.

yangxg89 commented 6 years ago

I have successfully installed iscsi target by using command line interface followed http://docs.ceph.com/docs/master/rbd/iscsi-target-cli , and got the rpm packages from https://shaman.ceph.com (kernel version: 4.16.0). In some cases, I need to use an older version of kernel(such as 4.6), what should I do?

mikechristie commented 6 years ago

You would need to port the patches to whatever kernel you wanted to use or just not use HA.

yangxg89 commented 6 years ago

Thanks for your reply! I feel sorry about that I do not know what patches do you mean. Could you please give me some more detailed instructions? Thanks again.

skunkr commented 5 years ago

Hi Everyone Any ideea if ceph-iscsi suite is fully working on Centos at the minute ? If so, could be someone so kind to provide me what is the fully supported centos version, and what whould be the installation steps ? Thank you very much ! Leo

dillaman commented 5 years ago

@skunkr CentOS 7.5 and above works [1]

[1] http://docs.ceph.com/docs/master/rbd/iscsi-targets/

mikechristie commented 5 years ago

7.5 should work with the newest 7.5 z kernel.

You should just use 7.6. The kernel shipped with it has lots of features and bug fixes in the iscsi, core target layer and target_core_user driver we use.

skunkr commented 5 years ago

This is just perfect ! Thank you very much guys.

Leo

On Sun, Dec 9, 2018, 04:32 mikechristie <notifications@github.com wrote:

7.5 should work with the newest 7.5 z kernel.

You should just use 7.6. The kernel shipped with it has lots of features and bug fixes in the iscsi, core target layer and target_core_user driver we use.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-445505598, or mute the thread https://github.com/notifications/unsubscribe-auth/Aea8GMvDtx_kShnDxHx963E_limpBD5bks5u3HZQgaJpZM4Pu8bq .

zerocarbthirty commented 5 years ago

By the way, how has the performance been coming along? Getting better?

On Sun, Dec 9, 2018, 3:54 AM skunkr <notifications@github.com wrote:

This is just perfect ! Thank you very much guys.

Leo

On Sun, Dec 9, 2018, 04:32 mikechristie <notifications@github.com wrote:

7.5 should work with the newest 7.5 z kernel.

You should just use 7.6. The kernel shipped with it has lots of features and bug fixes in the iscsi, core target layer and target_core_user driver we use.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-445505598 , or mute the thread < https://github.com/notifications/unsubscribe-auth/Aea8GMvDtx_kShnDxHx963E_limpBD5bks5u3HZQgaJpZM4Pu8bq

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ceph/ceph-iscsi-cli/issues/36#issuecomment-445520975, or mute the thread https://github.com/notifications/unsubscribe-auth/AP5WwmntiBtQHL8FufmmSH5sGt9nCmlSks5u3M_IgaJpZM4Pu8bq .

mikechristie commented 5 years ago

It is still much lower than krbd and using librbd directly.

borisbardak commented 5 years ago

:( It's a shame that iscsci peformance is so slow, practical usage for vmware is unsable compared to block storage :(

ceph / ceph-iscsi-cli

OS not supported #36