Method Not Allowed error

[aticiburak46]

Hey Mark,

I'm Burak, we have talked about hmci app while ago. Now im trying your svci app and it looks like cool. Firstly thanks for your works.

We have some old version storages and newer version ones, when we try to get data from 8.4.0.3 ( IBM FlashSystem 7200 ) we get the error " Method Not Allowed " but when we try with 8.5.0.9 ( this one is FlashSystem 9100 ) we can get data.

Do you have any suggestions or anything we can do without upgrading storages?

Thanks and Kind Regards, Burak

[mnellemann]

Hi Burak,

Glad to hear you find the software useful :)

Older versions of FlashSystem firmware (before 8.6 i think) did not support download of iostats through the HTTP API. In that case, svci will try to "fallback" and use ssh/scp to download the xml files for processing.

I am not 100% sure this is the case for you, but you could try to enable extended stats. SSH to your flashsystem controller and run:

svctask stopstats
svctask startstats -interval 1
lsdumps -prefix /dumps/iostats

Let me know if this works for you :)

Best regards, Mark

[mnellemann]

@olemyk did write some information about this here.

[aticiburak46]

We also collecting data with different tools and this is a big environment so i can't simply use stopstats and startstats. But also i have checked lsdumps with the prefix that you provide. You can see the output below.

124 Nvstats-1_240809_170209 125 Ndstats-1_240809_170209 126 Nnstats-1_240809_170209 127 Nmstats-1_240809_170209

Here is a output of service's logs;

Aug 09 17:07:58 hmci-test svci[477182]: [site4] [INFO] RestClient - Connecting to SVC - ibmmonitor @ https://:7443/rest/v1/auth Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - login() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - logon() - error: Unexpected code: Response{protocol=http/1.1, code=403, message=, url=https://:7443/rest/v1/auth} Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lssystem <= "null" => "MissingToken: Missing token" Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getSystem() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsnodestats <= "null" => "MissingToken: Missing token" Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getNodeStats() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsenclosurestats <= "null" => "MissingToken: Missing token" Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getEnclosureStats() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsmdiskgrp <= "null" => "MissingToken: Missing token" Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getMDiskGroups() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsdumps <= "{"prefix":"/dumps/iostats"}" => "MissingToken: Missing token" Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:07:59 hmci-test svci[477182]: [site4] [ERROR] VolumeController - listDumps() - error: postRequest() - Unexpected response: 403

Aug 09 17:08:29 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lssystem <= "null" => "MissingToken: Missing token" Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getSystem() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsnodestats <= "null" => "MissingToken: Missing token" Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getNodeStats() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsenclosurestats <= "null" => "MissingToken: Missing token" Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getEnclosureStats() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsmdiskgrp <= "null" => "MissingToken: Missing token" Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] VolumeController - getMDiskGroups() - error 2: postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [WARN] RestClient - 403: https://:7443/rest/v1/lsdumps <= "{"prefix":"/dumps/iostats"}" => "MissingToken: Missing token" Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] RestClient - postRequest() - Unexpected response: 403 Aug 09 17:08:29 hmci-test svci[477182]: [site4] [ERROR] VolumeController - listDumps() - error: postRequest() - Unexpected response: 403

One another configuration i want to share with you, here is our toml configuration file;

root@hmci-test:/etc# cat /etc/svci.toml

SVCi Configuration

Copy this file into /etc/svci.toml and customize it to your environment.

Define one InfluxDB to save metrics into

There must be only one and it should be named [influx]

InfluxDB v1.x example

[influx] url = "http://localhost:8086" username = "root" password = "" database = "svci"

InfluxDB v2.x example

[influx]

url = "http://localhost:8086"

org = "myOrg"

token = "rAnd0mT0k3nG3neRaT3dByInF1uxDb=="

bucket = "svci"

Define one or more SVC's to query for metrics

Each entry must be named [svc.]

Define one or more SVC's to query for metrics

Each entry must be named [svc.]

[svc.site1] hostname = "ipaddress1" username = "usernamewedefined" password = "passwordwedefined" refresh = 30 # How often to query SVC for data - in seconds trust = true # Ignore SSL cert. errors (due to default self-signed cert.)

[svc.site2] hostname = "ipaddress2" username = "usernamewedefined" password = "passwordwedefined" refresh = 30 # How often to query SVC for data - in seconds trust = true # Ignore SSL cert. errors (due to default self-signed cert.)

[svc.site3] hostname = "ipaddress3" username = "usernamewedefined" password = "passwordwedefined" refresh = 30 # How often to query SVC for data - in seconds trust = true # Ignore SSL cert. errors (due to default self-signed cert.)

[svc.site4] hostname = "ipaddress4" username = "usernamewedefined" password = "passwordwedefined" refresh = 30 # How often to query SVC for data - in seconds trust = true # Ignore SSL cert. errors (due to default self-signed cert.)

[olemyk]

Hi

For the simple stats, not the detailed stats from xml. (XML files Are Downloaded from restapi or scp) this will use restapi to call things like lsnodestats, lsxxstats.

You will need a Virtualize code 8.4.2 code or later. As there is a New restapi interface from 8.4.2)

8.4.0.x - don't have the new rest api version rest/v1/x - Then you need change api call to be without v1/

8.1.3.x - don't have the new rest api version rest/v1/x - then you need change api call to be without v1/

8.1.2.x - getting errors from api calls, 404, 500. to old code to use time on.

[aticiburak46]

I had lots of questions for you but here is the last one :)

We have almost 20 storages working currently, and most of them works between 8.4-8.6 versions. I tried to get metrics with curl and i was able to get token then use that token to get volume details. It worked.

Want to ask you if there is an any config file to change api addresses to remove v1 for api call. Because would be great if i can collect all the data from all the storages.

( Thanks both of you for quick responses.. )

Best Regards, Burak

[mnellemann]

Hi Burak,

Your configuration looks fine. There's really not anything you can tweak - only provide correct hostname and credentials :)

Good to hear it works with your 8.5 systems.

When running against 8.4, the code will print "Method not allowed" (as an API call does not exists in this version) and should fallback to scp/ssh, and try do download the xml iostats. Does this work in your case ?

Even if we could make some changes to allow your older systems to work, you will only get some of the data. Would this be useful at all? And they are quite old and not supported :) Is there any way for you to upgrade the software?

https://www.ibm.com/support/pages/support-statement-ibm-storage-virtualize-family-products-and-related-software

Best regards, Mark

[mnellemann]

I might be mistaken as I haven't really looked at this for some time.

When I test against a FS5100 running 8.5.0.8 I get the following in my svci output:

aug 09 17:46:29 lab-mgt.lab.local.net systemd[1]: Started SVC Insights Service.
aug 09 17:46:30 lab-mgt.lab.local.net svci[1398447]: [main] [INFO] DefaultIoServiceFactoryFactory - No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory
aug 09 17:46:30 lab-mgt.lab.local.net svci[1398447]: [fs5100] [INFO] RestClient - Connecting to SVC - svci @ https://10.32.64.183:7443/rest/v1/auth
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [WARN] RestClient - 404: https://10.32.64.183:7443/rest/v1/download <= "{"prefix":"/dumps/iostats","filename":"Nn_stats_78E07H8-2_240809_172949"}" => "Command 'download' does not exist"
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] RestClient - postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] VolumeController - getFile() - error: postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [WARN] RestClient - 404: https://10.32.64.183:7443/rest/v1/download <= "{"prefix":"/dumps/iostats","filename":"Nm_stats_78E07H8-2_240809_172949"}" => "Command 'download' does not exist"
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] RestClient - postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] VolumeController - getFile() - error: postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [WARN] RestClient - 404: https://10.32.64.183:7443/rest/v1/download <= "{"prefix":"/dumps/iostats","filename":"Nd_stats_78E07H8-2_240809_172949"}" => "Command 'download' does not exist"
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] RestClient - postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [ERROR] VolumeController - getFile() - error: postRequest() - Unexpected response: 404
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [INFO] ShellClient - getSession() - Creating SSH session
aug 09 17:46:31 lab-mgt.lab.local.net svci[1398447]: [fs5100] [INFO] DefaultConfigFileHostEntryResolver - resolveEffectiveResolver(svci@10.32.64.183:22/null) no configuration file at /.ssh/config

telling us that the download API call does not work (in 8.5, only from 8.6) and tries to use SSH.