Does it support python-swiftclient 3.2.1.dev?

[tommyJin]

Hi there,

The issue is that I followed each step and when I ran the systemctl start swifthlm.dispatcher and systemctl status swifthlm.dispatcher -l in order, and the output showed:

`[osddev@localhost ~]$ systemctl start swifthlm.dispatcher [osddev@localhost ~]$ systemctl status swifthlm.dispatcher -l ● swifthlm.dispatcher.service - SwiftHLM Dispatcher Loaded: loaded (/etc/systemd/system/swifthlm.dispatcher.service; disabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2017-05-31 11:59:45 CST; 5s ago Process: 6416 ExecStart=/usr/bin/python -m swifthlm.dispatcher (code=exited, status=217/USER) Main PID: 6416 (code=exited, status=217/USER)

May 31 11:59:44 localhost.localdomain systemd[1]: Started SwiftHLM Dispatcher. May 31 11:59:44 localhost.localdomain systemd[1]: Starting SwiftHLM Dispatcher... May 31 11:59:45 localhost.localdomain systemd[1]: swifthlm.dispatcher.service: main process exited, code=exited, status=217/USER May 31 11:59:45 localhost.localdomain systemd[1]: Unit swifthlm.dispatcher.service entered failed state. May 31 11:59:45 localhost.localdomain systemd[1]: swifthlm.dispatcher.service failed. `

I installed swift by SAIO - Swift All In One

Is it possible that the issue is about the version of python-swiftclient? Or other reasons cause the issue?

Thanks.

[hseipp]

Hi Tommy,

unfortunately the error output is not very informative, so we need more information like

• which OS flavor and version are you using?
• did you configure SwiftHLM and the HLM backend and installed both according to the README.txt before starting SAIO and the SwiftHLM dispatcher?
  • can you please start python -m swifthlm.dispatcher as a standalone process and capture the output?

Note that python-swiftclient is not used at all, but of course following the SAIO instructions will give you the master branch copy of the current swift code that is changing almost every day, so it might happen that our code does not work with that latest code.

[tommyJin]

Hi @hseipp ,

Thanks for your reply! Sorry for the question format, I'll list my versions and questions below. I'm new to openstack swift world and SwiftHLM, so please don't mind if I put forward some stupid questions :)

Versions: OS: CentOS7 in VMWare Python: 2.7.5 Openstack swift: 2.10.0(from PKG-INFO)

Process: I followed step by step, and it was okay until step 4. In /etc/passwd file, there is no user named 'swift' but a user named 'osddev'(under /home directory, the subdirectory is osddev, not swift) osddev:x:1000:1000:OSd Devel:/home/osddev:/bin/bash And I am confused the number of servers I need to run SwiftHLM. Currently, I have one server with openstack swift installed by SAIO and swift works fine alone. In README.txt, it mentions that

4.2.2 Setup key-based ssh for swift user:

• Use ssh-keygen to generate RSA keys on Dispatcher node
• cp content of /home/swift/.ssh/id_rsa.pub from Dispatcher node into /home/swift/.ssh/authorized_keys on storage nodes Does that mean I need to build 2 servers with both swift and swiftHLM installed? Or the dispatch node and storage node are just directories? In order to push forward, I skipped the 4.2.2 step and went straight to step 5.

When I ran swift-init main reload, output shows a error 'ImportError: No module named paramiko', so I installed the paramiko. Then I modified file '/etc/systemd/system/swifthlm.dispatcher.service', and set the value of key 'User' from 'swift' to 'osddev'.

I paste the output below:

[osddev@localhost swift]$ swift-init main reload Signal proxy-server pid: 35175 signal: 1 proxy-server (35175) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting proxy-server...(/etc/swift/proxy-server.conf) Signal container-server pid: 35190 signal: 1 Signal container-server pid: 35191 signal: 1 Signal container-server pid: 35192 signal: 1 Signal container-server pid: 35193 signal: 1 container-server (35192) appears to have stopped container-server (35193) appears to have stopped container-server (35190) appears to have stopped container-server (35191) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting container-server...(/etc/swift/container-server/1.conf) Starting container-server...(/etc/swift/container-server/2.conf) Starting container-server...(/etc/swift/container-server/3.conf) Starting container-server...(/etc/swift/container-server/4.conf) Signal account-server pid: 35249 signal: 1 Signal account-server pid: 35250 signal: 1 Signal account-server pid: 35251 signal: 1 Signal account-server pid: 35252 signal: 1 account-server (35249) appears to have stopped account-server (35250) appears to have stopped account-server (35251) appears to have stopped account-server (35252) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting account-server...(/etc/swift/account-server/1.conf) Starting account-server...(/etc/swift/account-server/2.conf) Starting account-server...(/etc/swift/account-server/3.conf) Starting account-server...(/etc/swift/account-server/4.conf) Signal object-server pid: 35305 signal: 1 Signal object-server pid: 35306 signal: 1 Signal object-server pid: 35307 signal: 1 Signal object-server pid: 35308 signal: 1 object-server (35306) appears to have stopped object-server (35307) appears to have stopped object-server (35305) appears to have stopped object-server (35308) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting object-server...(/etc/swift/object-server/1.conf) Starting object-server...(/etc/swift/object-server/2.conf) Starting object-server...(/etc/swift/object-server/3.conf) Starting object-server...(/etc/swift/object-server/4.conf) [osddev@localhost swift]$ clear  [osddev@localhost swift]$ swift-init main reload Signal proxy-server pid: 35454 signal: 1 proxy-server (35454) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting proxy-server...(/etc/swift/proxy-server.conf) Signal container-server pid: 35468 signal: 1 Signal container-server pid: 35469 signal: 1 Signal container-server pid: 35470 signal: 1 Signal container-server pid: 35471 signal: 1 container-server (35468) appears to have stopped container-server (35469) appears to have stopped container-server (35470) appears to have stopped container-server (35471) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting container-server...(/etc/swift/container-server/1.conf) Starting container-server...(/etc/swift/container-server/2.conf) Starting container-server...(/etc/swift/container-server/3.conf) Starting container-server...(/etc/swift/container-server/4.conf) Signal account-server pid: 35525 signal: 1 Signal account-server pid: 35526 signal: 1 Signal account-server pid: 35527 signal: 1 Signal account-server pid: 35528 signal: 1 account-server (35525) appears to have stopped account-server (35528) appears to have stopped account-server (35526) appears to have stopped account-server (35527) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting account-server...(/etc/swift/account-server/1.conf) Starting account-server...(/etc/swift/account-server/2.conf) Starting account-server...(/etc/swift/account-server/3.conf) Starting account-server...(/etc/swift/account-server/4.conf) Signal object-server pid: 35581 signal: 1 Signal object-server pid: 35582 signal: 1 Signal object-server pid: 35583 signal: 1 Signal object-server pid: 35584 signal: 1 object-server (35584) appears to have stopped object-server (35581) appears to have stopped object-server (35582) appears to have stopped object-server (35583) appears to have stopped WARNING: Unable to modify file descriptor limit. Running as non-root? WARNING: Unable to modify max process limit. Running as non-root? Starting object-server...(/etc/swift/object-server/1.conf) Starting object-server...(/etc/swift/object-server/2.conf) Starting object-server...(/etc/swift/object-server/3.conf) Starting object-server...(/etc/swift/object-server/4.conf) [osddev@localhost swift]$ systemctl start swifthlm.dispatcher Warning: swifthlm.dispatcher.service changed on disk. Run 'systemctl daemon-reload' to reload units. [osddev@localhost swift]$ cat /etc/swift/ account.builder container-reconciler.conf object-1.ring.gz object.ring.gz account.ring.gz container.ring.gz object-2.builder object-server/ account-server/ container-server/ object-2.ring.gz proxy-server.conf backups/ container-sync-realms.conf object.builder swift.conf container.builder object-1.builder object-expirer.conf test.conf [osddev@localhost swift]$ systemctl daemon-reload [osddev@localhost swift]$ systemctl start swifthlm.dispatcher [osddev@localhost swift]$ systemctl status swifthlm.dispatcher -l ● swifthlm.dispatcher.service - SwiftHLM Dispatcher Loaded: loaded (/etc/systemd/system/swifthlm.dispatcher.service; disabled; vendor preset: disabled) Active: active (running) since Mon 2017-06-05 11:45:44 CST; 15s ago Main PID: 36661 (python) CGroup: /system.slice/swifthlm.dispatcher.service └─36661 /usr/bin/python -m swifthlm.dispatcher Jun 05 11:45:44 localhost.localdomain systemd[1]: Started SwiftHLM Dispatcher. Jun 05 11:45:44 localhost.localdomain systemd[1]: Starting SwiftHLM Dispatcher... Jun 05 11:45:45 localhost.localdomain hlm-middleware[36661]: 196 [middleware.py: init():245] info: Initialized SwiftHLM Middleware Jun 05 11:45:45 localhost.localdomain hlm-dispatcher[36661]: 196 [dispatcher.py: init():101] info: Initialized Dispatcher Jun 05 11:45:45 localhost.localdomain swift[36661]: - - 05/Jun/2017/03/45/45 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - txa96a333144d8480987c74-005934d3e9 - 0.0331 - - 1496634345.197978973 1496634345.231062889 - Jun 05 11:45:50 localhost.localdomain swift[36661]: - - 05/Jun/2017/03/45/50 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx8e4b7367549747be83574-005934d3ee - 0.0013 - - 1496634350.233664036 1496634350.234992027 - Jun 05 11:45:55 localhost.localdomain swift[36661]: - - 05/Jun/2017/03/45/55 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx49ac83008a234a2283635-005934d3f3 - 0.0149 - - 1496634355.236531019 1496634355.251461983 -

It seems that SwiftHLM is running? Right?

I got 3 questions so far, Q1: Would it crushes or shutdown unexpectedly if I don't set step 4.2.2? Q2: If I want to do a 'swift<-->swifthlm<-->optical disk' mode instead of tape, what am I supposed to do to implement it? I got the optical library and it works fine.
Q3: Did I operate correctly above? If not, what am I supposed to modify?

A lot of thanks to you! :)

Good day.

Tommy

[hseipp]

Hi @tommyJin

although we mention servers/nodes, both Swift as well as SwiftHLM with all their services can be installed on a single host. I do run a Centos 7 VM with all the services as well. You can't skip step 4.2.2 as even for a single system, SwiftHLM uses ssh to connect to the node running the object server hosting the object to get object<->file mapping information so in your case the process needs to have passwordless ssh access to the very same node. If you haven't set up a swift user, you need to specify the user you use for SAIO (see the SAIO documentation - they refer to it as ${USER}.

From the two swift-init main reload outputs you provided it is not clear if the Swift service is running. Can you access Swift, does swift stat provide a result?

SwiftHLM is most probably running if your Swift proxy server starts while the SwiftHLM middleware is active in the pipeline. Note that the swifthlm.dispatcher process is just a piece of the overall solution (picking up requests from the internal request queue that are fed in through the proxy server and the middleware), but the above output looks OK.

A1: Without step 4.2.2 at least object GET and SwiftHLM status commands will not work as they need to reach out to the object server node using ssh to retrieve file<->object mapping information to determine if the object is on low-latency media or high-latency media. A2: You need to write your own backend connector, we actually work with a company currently doing similar things, contact me via PM for more details. A3: The second swift-init main reload does not look good as I mentioned above, but the output of the swifthlm.dispatcher process looks OK as it can identify and access the request queue.

Your next step could be to test SwiftHLM API requests or run/dig into our automated test cases in the tests/ directory.

Regards, Harald

[tommyJin]

Hi @hseipp ,

Part One When installing SAIO, I point user 'osddev' as the swift user.

• Use ssh-keygen to generate RSA keys on Dispatcher node
  • cp content of /home/swift/.ssh/id_rsa.pub from Dispatcher node into /home/swift/.ssh/authorized_keys on storage nodes

From above description, I don't get what are Dispatcher node and storages nodes. I use the user 'osddev', so in my case, the path is like this '/home/osddev', and sdb1 is mounted on '/mnt/sdb1', and there are 4 directories under '/mnt/sdb1' called '1', '2', '3', '4' respectively. When I create containers or upload objects, new files would be created like '/mnt/sdb1/1/node/sdb1/containers/203/053/ajwoiwatohgoahogawgojawo' and '/mnt/sdb1/1/node/sdb1/objects/4/7eb/12312415.123910.data'.

Also, there is a directory named '/etc/swift' which contains many .conf files and directories called 'account-server', 'container-server', 'object-server'.

But I can't find anything related to the dispatcher except '/opt/swifthlm/swifthlm/dispatcher.py'. In addition, is '/mnt/sdb1' the same as storage node? When I 'ssh-keygen' under a specific directory, the '.ssh' folder only owns files 'id_rsa' and 'id_rsa.pub'. Shall I create file 'authorized_keys' manually?

Part Two When I run 'swift stat', the output is like below:

[osddev@localhost system]$ swift stat Auth version 1.0 requires ST_AUTH, ST_USER, and ST_KEY environment variables to be set or overridden with -A, -U, or -K. Auth version 2.0 requires OS_AUTH_URL, OS_USERNAME, OS_PASSWORD, and OS_TENANT_NAME OS_TENANT_ID to be set or overridden with --os-auth-url, --os-username, --os-password, --os-tenant-name or os-tenant-id. Note: adding "-V 2" is necessary for this.

Part Three With current settings(no 4.2.2), and run 'swift-init main reload', 'systemctl start swifthlm.dispatcher' and 'systemctl status swifthlm.dispatcher -l' in order, the output is like

[osddev@localhost system]$ sudo swift-init main reload [sudo] password for osddev: Signal proxy-server pid: 42834 signal: 1 proxy-server (42834) appears to have stopped Starting proxy-server...(/etc/swift/proxy-server.conf) Signal container-server pid: 42849 signal: 1 Signal container-server pid: 42850 signal: 1 Signal container-server pid: 42851 signal: 1 Signal container-server pid: 42852 signal: 1 container-server (42849) appears to have stopped container-server (42850) appears to have stopped container-server (42851) appears to have stopped container-server (42852) appears to have stopped Starting container-server...(/etc/swift/container-server/1.conf) Starting container-server...(/etc/swift/container-server/2.conf) Starting container-server...(/etc/swift/container-server/3.conf) Starting container-server...(/etc/swift/container-server/4.conf) Signal account-server pid: 42906 signal: 1 Signal account-server pid: 42907 signal: 1 Signal account-server pid: 42908 signal: 1 Signal account-server pid: 42909 signal: 1 account-server (42908) appears to have stopped account-server (42906) appears to have stopped account-server (42907) appears to have stopped account-server (42909) appears to have stopped Starting account-server...(/etc/swift/account-server/1.conf) Starting account-server...(/etc/swift/account-server/2.conf) Starting account-server...(/etc/swift/account-server/3.conf) Starting account-server...(/etc/swift/account-server/4.conf) Signal object-server pid: 42963 signal: 1 Signal object-server pid: 42964 signal: 1 Signal object-server pid: 42965 signal: 1 Signal object-server pid: 42966 signal: 1 object-server (42963) appears to have stopped object-server (42964) appears to have stopped object-server (42965) appears to have stopped object-server (42966) appears to have stopped Starting object-server...(/etc/swift/object-server/1.conf) Starting object-server...(/etc/swift/object-server/2.conf) Starting object-server...(/etc/swift/object-server/3.conf) Starting object-server...(/etc/swift/object-server/4.conf) [osddev@localhost system]$ systemctl start swifthlm.dispatcher [osddev@localhost system]$ systemctl status swifthlm.dispatcher -l ● swifthlm.dispatcher.service - SwiftHLM Dispatcher Loaded: loaded (/etc/systemd/system/swifthlm.dispatcher.service; disabled; vendor preset: disabled) Active: active (running) since Mon 2017-06-05 11:45:44 CST; 3 days ago Main PID: 36661 (python) CGroup: /system.slice/swifthlm.dispatcher.service └─36661 /usr/bin/python -m swifthlm.dispatcher

Jun 08 17:50:06 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/06 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx8f5f4f871d224a318de3d-0059391dce - 0.0224 - - 1496915406.496454954 1496915406.518866062 - Jun 08 17:50:11 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/11 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx8df6fcfd89c8442189a86-0059391dd3 - 0.0032 - - 1496915411.524243116 1496915411.527411938 - Jun 08 17:50:16 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/16 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx711eb1afd214441f894ef-0059391dd8 - 0.0193 - - 1496915416.530283928 1496915416.549582958 - Jun 08 17:50:21 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/21 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - txc268b5a675214b8a88a5e-0059391ddd - 0.0013 - - 1496915421.553831100 1496915421.555177927 - Jun 08 17:50:26 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/26 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx44b304460d974bb2954c5-0059391de2 - 0.0201 - - 1496915426.557710886 1496915426.577801943 - Jun 08 17:50:31 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/31 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx45505cf274204588a53b4-0059391de7 - 0.0032 - - 1496915431.583050966 1496915431.586251974 - Jun 08 17:50:36 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/36 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - txfdd07dd6ba4646768b628-0059391dec - 0.0259 - - 1496915436.591353893 1496915436.617301941 - Jun 08 17:50:41 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/41 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx52f5225e2d124fe0af791-0059391df1 - 0.0041 - - 1496915441.622495890 1496915441.626568079 - Jun 08 17:50:46 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/46 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - txc97cf62f03d74007b8f71-0059391df6 - 0.0172 - - 1496915446.628108025 1496915446.645265102 - Jun 08 17:50:51 localhost.localdomain swift[36661]: - - 08/Jun/2017/09/50/51 GET /v1/.swifthlm/pending-hlm-requests%3Fformat%3Djson%26marker%3D%26end_marker%3D HTTP/1.0 404 - SwiftHLM%20Middleware - - 70 - tx77c06202aeb5402ca9d8e-0059391dfb - 0.0018 - - 1496915451.650019884 1496915451.651850939 -

After running test, the output is like:

OK (SKIP=14) /home/osddev/swift [osddev@localhost swift]$ /home/osddev/swift/.unittests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S..S..........................................................................................................................................................................................................................................................................................................................................................................................................................................................SSSS..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S......................................SSS.............................................................................................................S.............................................SSS...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... Name Stmts Miss Branch BrMiss Cover Missing

swift 14 6 0 0 57% 28-34, 41 swift.account 0 0 0 0 100%
swift.account.auditor 89 0 18 2 98%
swift.account.backend 211 6 80 7 96% 342, 423, 450-451, 503, 548 swift.account.reaper 277 19 107 17 91% 121-123, 164, 168, 171, 185, 189, 280-281, 287-290, 317, 396, 438-443 swift.account.replicator 7 0 0 0 100%
swift.account.server 192 3 67 3 98% 122-123, 153 swift.account.utils 61 1 26 1 98% 61 swift.cli 0 0 0 0 100%
swift.cli.form_signature 93 6 15 1 94% 67-68, 74-75, 81-82 swift.cli.info 334 14 170 19 93% 85, 87, 89, 91, 217-218, 351, 410-412, 435-436, 451-454 swift.cli.recon 699 195 342 140 68% 55, 84, 88, 93, 130, 135, 167, 198-204, 250-251, 258-259, 262-278, 298, 308-310, 334, 356, 422-442, 488-490, 494, 517-535, 544-583, 604-605, 647, 649, 656-658, 671, 673, 680-682, 711, 745, 755-780, 806, 840, 856-858, 864, 897-898, 913, 1014-1015, 1022-1026, 1046-1067, 1070-1075, 1077, 1079, 1086-1091, 1093-1098, 1102, 1104, 1107-1108, 1110, 1112, 1114, 1116, 1124 swift.cli.ring_builder_analyzer 160 17 59 5 90% 303-304, 330-349 swift.cli.ringbuilder 657 61 262 37 89% 85-86, 92, 132-133, 162-164, 174, 196-197, 244-245, 340, 372-373, 452, 556-559, 607-608, 613-614, 645-646, 652, 753-754, 802-813, 886-888, 891-899, 908-916, 973-975, 982, 992-993, 1048-1053, 1069-1071, 1075-1077, 1081, 1225, 1273, 1284-1285 swift.common 0 0 0 0 100%
swift.common.base_storage_server 36 0 24 1 98%
swift.common.bufferedhttp 110 17 35 9 82% 44, 96-112, 127, 208-209, 237 swift.common.constraints 165 3 63 1 98% 90-91, 374 swift.common.container_sync_realms 96 0 26 3 98%
swift.common.daemon 47 4 8 2 89% 49-51, 96 swift.common.db 472 41 150 25 89% 100, 108, 181-182, 291, 349-353, 381-382, 409-412, 447, 570, 577, 579, 599, 605, 612-613, 620-622, 645, 652, 692-694, 714, 787-793, 832-835, 869 swift.common.db_replicator 470 22 169 15 94% 66, 176-182, 295, 498, 552-553, 564-565, 593-595, 663-672, 770 swift.common.direct_client 224 5 103 11 95% 353, 426, 552-558 swift.common.exceptions 148 4 26 3 96% 191, 258, 263, 268 swift.common.header_key_dict 32 0 14 0 100%
swift.common.http 77 0 0 0 100%
swift.common.internal_client 309 4 105 5 98% 798-799, 855-856 swift.common.linkat 34 5 6 3 80% 46-50, 73 swift.common.manager 402 5 171 11 97% 160, 172, 334, 687, 760 swift.common.memcached 256 22 90 18 88% 175-176, 181-182, 190-194, 210, 299-300, 382-383, 409-410, 425-426, 450-453, 467-468 swift.common.middleware 0 0 0 0 100%
swift.common.middleware.account_quotas 56 0 24 0 100%
swift.common.middleware.acl 90 3 55 5 94% 182, 246, 295 swift.common.middleware.bulk 337 20 149 11 94% 384, 386, 452-454, 462-464, 483-485, 533-535, 620, 628-629, 637-639 swift.common.middleware.catch_errors 43 0 4 1 98%
swift.common.middleware.cname_lookup 85 3 36 1 97% 39-41, 92 swift.common.middleware.container_quotas 46 1 26 2 96% 97 swift.common.middleware.container_sync 76 0 33 2 98%
swift.common.middleware.copy 217 3 84 2 98% 291, 297-298 swift.common.middleware.crossdomain 23 6 2 0 76% 86-92 swift.common.middleware.crypto 12 0 0 0 100%
swift.common.middleware.crypto.crypto_utils 125 0 24 0 100%
swift.common.middleware.crypto.decrypter 209 2 60 3 98% 127, 334 swift.common.middleware.crypto.encrypter 174 3 48 4 97% 180, 205, 371 swift.common.middleware.crypto.keymaster 74 3 19 2 95% 154-158 swift.common.middleware.dlo 173 2 74 3 98% 389-390 swift.common.middleware.domain_remap 64 1 30 0 99% 144 swift.common.middleware.formpost 189 9 65 7 94% 267-268, 301, 332-333, 360-361, 398-399 swift.common.middleware.gatekeeper 41 0 10 0 100%
swift.common.middleware.healthcheck 24 0 4 0 100%
swift.common.middleware.keystoneauth 241 2 112 3 99% 186, 304 swift.common.middleware.list_endpoints 99 0 25 1 99%
swift.common.middleware.memcache 59 0 18 0 100%
swift.common.middleware.name_check 49 2 12 2 93% 69, 100 swift.common.middleware.proxy_logging 161 0 60 0 100%
swift.common.middleware.ratelimit 156 0 69 3 99%
swift.common.middleware.recon 270 19 122 19 90% 245, 273-276, 282, 329-331, 335-339, 410-415 swift.common.middleware.slo 455 2 206 7 99% 639, 936 swift.common.middleware.staticweb 216 2 94 4 98% 268, 511 swift.common.middleware.tempauth 373 33 177 15 91% 266-267, 389, 395, 410, 434-441, 484-486, 618, 620-625, 639-641, 646-647, 679-681, 713, 730-732, 738 swift.common.middleware.tempurl 207 4 98 0 99% 233-234, 249-250 swift.common.middleware.versioned_writes 286 14 130 12 94% 268, 270, 344-347, 350, 366, 382, 604, 641, 650, 774, 802-803 swift.common.middleware.x_profile 0 0 0 0 100%
swift.common.middleware.x_profile.exceptions 16 0 0 0 100%
swift.common.middleware.x_profile.html_viewer 244 46 78 16 81% 34-35, 301, 316-317, 391-392, 398-433, 474, 477, 483-484, 506, 521, 531-532 swift.common.middleware.x_profile.profile_model 201 86 50 14 60% 30-31, 80-81, 91-92, 129-229 swift.common.middleware.xprofile 118 19 19 6 82% 174-175, 200-201, 213, 224-234, 251-257 swift.common.request_helpers 225 2 87 3 98% 415, 533 swift.common.ring 3 0 0 0 100%
swift.common.ring.builder 802 29 455 15 96% 52-56, 179-191, 659, 725-726, 829, 1116, 1128, 1239, 1387, 1430, 1497, 1533, 1670 swift.common.ring.ring 246 1 124 9 97% 101 swift.common.ring.utils 296 9 156 6 97% 229-230, 449, 458, 481, 492, 497-498, 611 swift.common.splice 80 0 20 0 100%
swift.common.storage_policy 368 10 140 5 97% 197-198, 226, 330, 333, 346, 369, 390, 461-462 swift.common.swob 711 4 275 8 99% 145, 292, 567, 877 swift.common.utils 1976 84 809 71 94% 76, 208-210, 567-569, 681, 707, 727, 909, 938, 966, 1224-1240, 1407, 1417, 1450, 1521, 1553-1554, 1749, 1839, 1881-1893, 1919-1921, 1968-1969, 2135, 2260, 2268, 2303-2304, 2558, 2566, 2574, 2582, 2590, 2809, 3055-3057, 3072, 3384, 3399, 3623-3624, 3671, 3698-3699, 3732-3735, 3740, 3878, 3993, 4000-4001, 4007, 4031-4034 swift.common.wsgi 523 61 176 45 85% 54-55, 93, 153, 201, 233-235, 272, 368, 430, 446-448, 922-984, 998-999, 1007, 1015, 1133 swift.container 0 0 0 0 100%
swift.container.auditor 72 0 10 2 98%
swift.container.backend 401 32 169 29 89% 219, 243, 246, 295, 402, 521, 529, 545, 558-564, 580, 599-603, 636, 722, 741, 766-767, 831, 894, 906, 919, 948-951, 1000, 1009 swift.container.reconciler 300 18 69 1 95% 89, 146-147, 601-612, 642-643, 705-706, 751-755 swift.container.replicator 167 17 61 9 89% 96, 110, 115, 125-126, 144-146, 151-155, 183, 219-220, 224-225, 248 swift.container.server 338 14 153 14 94% 231, 233, 248, 255-256, 276-277, 292, 296, 336, 353, 366-367, 527 swift.container.sync 251 3 82 6 97% 238, 388, 589 swift.container.sync_store 67 0 22 1 99%
swift.container.updater 184 57 58 25 66% 103-110, 119-121, 123-142, 148, 162-175, 223, 225, 259, 264-265, 295-311 swift.obj 0 0 0 0 100%
swift.obj.auditor 242 1 68 4 98% 276 swift.obj.diskfile 1290 44 488 39 95% 103, 247-249, 257-258, 278, 288-290, 399, 407, 422, 435-438, 451, 569, 660, 904, 940, 989, 1022, 1026-1028, 1354-1358, 1459, 1505, 1684, 1697, 2080, 2368-2379, 2672 swift.obj.expirer 160 1 62 2 99% 187 swift.obj.mem_diskfile 210 35 59 19 80% 77, 86, 195, 204, 211, 223-224, 227, 231, 236, 248-249, 296, 313-314, 330-331, 334, 340-348, 351-357, 364-366, 368, 381, 453-455, 460, 472 swift.obj.mem_server 14 4 0 0 71% 53, 58-60 swift.obj.reconstructor 435 38 159 16 91% 92, 260, 264, 268, 280-287, 326, 396-400, 409, 419-422, 551-555, 705, 835, 912-914, 930-935, 957-972 swift.obj.replicator 455 62 179 42 84% 90-95, 184-187, 192-197, 202, 251, 355-357, 540-544, 553, 563-566, 593-600, 607, 636, 713, 716, 719-723, 728-732, 734-736, 741-747, 755-763, 801-820 swift.obj.server 618 32 230 18 94% 421-426, 435-438, 444, 586-587, 654, 658-659, 670-671, 687, 755-758, 771, 835, 837, 919, 945, 998, 1163-1165 swift.obj.ssync_receiver 219 3 131 22 93% 235-236, 412 swift.obj.ssync_sender 227 0 116 6 98%
swift.obj.updater 177 47 48 16 72% 76-119, 170, 175, 178-184, 209-216, 275-278 swift.proxy 0 0 0 0 100%
swift.proxy.controllers 6 0 0 0 100%
swift.proxy.controllers.account 93 0 34 0 100%
swift.proxy.controllers.base 850 27 409 26 96% 290, 294, 518-519, 880, 948, 952, 990-992, 1057-1059, 1061-1064, 1080-1082, 1100, 1133-1137, 1276, 1345, 1393, 1503 swift.proxy.controllers.container 142 1 61 4 98% 232 swift.proxy.controllers.info 50 0 16 0 100%
swift.proxy.controllers.obj 1110 26 393 46 95% 102, 441, 590, 652-653, 709-711, 720-721, 882-884, 1234-1235, 1254, 1507, 1520, 1640, 1671, 1674, 1744, 1924, 2393-2394, 2419, 2496 swift.proxy.server 286 17 104 10 93% 167, 172-174, 180-182, 187-188, 194, 239, 292, 316, 347, 349, 356, 358-359, 571

TOTAL 24475 1424 9576 1003 93%

Ran 4814 tests in 199.327s OK (SKIP=14) /home/osddev/swift

And how do I contact you privately? Cause there's no IM service on Github, may I send email to you or in other ways?

Regards, Tommy

[hseipp]

Hi @tommyJin ,

Part One

The "dispatcher node" is the node where the swifthlm.dispatcher process is running, the storage nodes are the nodes where the Swift Object Server is running. In your case this is all on your very single node, but still you have to guarantee keyless ssh for the user running your swift(hlm) processes which is osddev as I now understand. So all you need to do is cp /home/osddev/.ssh/id_dsa.pub /home/osddev/.ssh/authorized_keys && chown osddev:osddev /home/osddev/.ssh/authorized_keys && chmod 600 /home/osddev/.ssh/authorized_keys

When we mention the "dispatcher" we refer to the swifthlm.dispatcher process that you managed to start successfully. /mnt/sdb1 is actually your low-latency media backend storage for the Swift Object server, the "storage node" is the node running the object server processes - again the one SAIO node in your case.

Part two

The swift stat looks as expected when you did not have the appropriate environment variables set for Swift API endpoint and credentials - see step 13 of the SAIO setup guide for an example how to provide the appropriate swiftclient CLI parameters. I do set the OS_... environment variables to keep the CLI commands short - see the Swiftclient documentation.

Part three

Everything looks good there, can you please double-check with swift-init main status that the main Swift processes are running?

Private mail

Please refer to the SwiftHLM README.txt file for my eMail address. Hint: find "Harald" in there :wink:

Regards, Harald

[tommyJin]

Hi @hseipp ,

Jun 9, 2017 Keyless After reading your last reply, I did

cp /home/osddev/.ssh/id_dsa.pub /home/osddev/.ssh/authorized_keys chown osddev:osddev /home/osddev/.ssh/authorized_keys chmod 600 /home/osddev/.ssh/authorized_keys

in order to guarantee keyless ssh for osddev user running swift(hlm) processes.

Status of swift When I run

swift-init main status

the output is like below:

[osddev@localhost swift]$ swift-init main status proxy-server running (48478 - /etc/swift/proxy-server.conf) container-server running (48496 - /etc/swift/container-server/4.conf) container-server running (48493 - /etc/swift/container-server/1.conf) container-server running (48494 - /etc/swift/container-server/2.conf) container-server running (48495 - /etc/swift/container-server/3.conf) account-server running (48552 - /etc/swift/account-server/4.conf) account-server running (48549 - /etc/swift/account-server/1.conf) account-server running (48550 - /etc/swift/account-server/2.conf) account-server running (48551 - /etc/swift/account-server/3.conf) object-server running (48608 - /etc/swift/object-server/2.conf) object-server running (48609 - /etc/swift/object-server/3.conf) object-server running (48610 - /etc/swift/object-server/4.conf) object-server running (48607 - /etc/swift/object-server/1.conf)

Stat of swift When I run

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat

the output is like below:

[osddev@localhost swift]$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
Account: AUTH_test Containers: 3 Objects: 0 Bytes: 0 Containers in policy "gold": 3 Objects in policy "gold": 0 Bytes in policy "gold": 0 Meta Owner: Hello World Accept-Ranges: bytes X-Timestamp: 1480498333.03373 X-Trans-Id: tx60ee4eacdc214a93937ce-00593a4dfa Content-Type: text/plain; charset=utf-8

Account’s containers When I run

curl -v -X GET -H "X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710" http://127.0.0.1:8080/v1/AUTH_test

the output is like below:

[osddev@localhost swift]$ curl -v -X GET -H "X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710" http://127.0.0.1:8080/v1/AUTH_test About to connect() to 127.0.0.1 port 8080 (#0) Trying 127.0.0.1... Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) GET /v1/AUTH_test HTTP/1.1 User-Agent: curl/7.29.0 Host: 127.0.0.1:8080 Accept: / X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710 HTTP/1.1 200 OK X-Account-Storage-Policy-Gold-Bytes-Used: 0 Content-Length: 38 X-Account-Storage-Policy-Gold-Object-Count: 0 Accept-Ranges: bytes X-Account-Object-Count: 0 X-Timestamp: 1480498333.02658 X-Account-Storage-Policy-Gold-Container-Count: 3 X-Account-Bytes-Used: 0 X-Account-Container-Count: 3 Content-Type: text/plain; charset=utf-8 X-Account-Meta-Owner: Hello World X-Trans-Id: tx8aa54136f1f8429e8c9f9-00593a4f67 Date: Fri, 09 Jun 2017 07:33:59 GMT container1 container_test contaniner1 Connection #0 to host 127.0.0.1 left intact

SwiftHLM API tests:

Currently, I have not specified backend connector(use dummy connector) and modified other configurations as the README demonstrates.

The output of 'curl -H "X-Auth-Token: $TOKEN" -X POST "http://zagreb.zurich.ibm.com:8080/hlm/v1/migrate/AUTH_test/cont1':

[osddev@localhost swift]$ curl -v -H "X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710" -X POST "http://127.0.0.1:8080/hlm/v1/migrate/AUTH_test/container_test" About to connect() to 127.0.0.1 port 8080 (#0) Trying 127.0.0.1... Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) POST /hlm/v1/migrate/AUTH_test/container_test HTTP/1.1 User-Agent: curl/7.29.0 Host: 127.0.0.1:8080 Accept: / X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710 HTTP/1.1 500 Internal Error Content-Length: 17 Content-Type: text/plain X-Trans-Id: txace2d026ce854f7a83332-00593a50c6 Date: Fri, 09 Jun 2017 07:39:50 GMT Connection #0 to host 127.0.0.1 left intact

Error log of the migrate request I check out the /var/log/swift/proxy.error, the most recent log shows:

Jun 9 15:39:50 localhost proxy-server: STDERR: (48492) accepted ('127.0.0.1', 38832) Jun 9 15:39:50 localhost proxy-server: Error: An error occurred: #012Traceback (most recent call last):#012 File "/home/osddev/swift/swift/common/middleware/catch_errors.py", line 41, in handle_request#012 resp = self._app_call(env)#012 File "/home/osddev/swift/swift/common/wsgi.py", line 1062, in _app_call#012 resp = self.app(env, self._start_response)#012 File "/home/osddev/swift/swift/common/middleware/gatekeeper.py", line 102, in call#012 return self.app(env, gatekeeper_response)#012 File "/home/osddev/swift/swift/common/middleware/healthcheck.py", line 52, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/proxy_logging.py", line 346, in call#012 six.reraise(exc_type, exc_value, exc_traceback)#012 File "/home/osddev/swift/swift/common/middleware/proxy_logging.py", line 338, in call#012 iterable = self.app(env, my_start_response)#012 File "/home/osddev/swift/swift/common/middleware/memcache.py", line 109, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/swob.py", line 1396, in _wsgify_self#012 return func(self, Request(env))(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/tempurl.py", line 346, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/ratelimit.py", line 299, in call#012 ratelimit_resp = self.handle_ratelimit(req, account, container, obj)#012 File "/home/osddev/swift/swift/common/middleware/ratelimit.py", line 236, in handle_ratelimit#012 swift_source='RL')#012 File "/home/osddev/swift/swift/proxy/controllers/base.py", line 389, in get_account_info#012 resp = req.get_response(app)#012 File "/home/osddev/swift/swift/common/swob.py", line 1008, in get_response#012 status, headers, app_iter = self.call_application(application)#012 File "/home/osddev/swift/swift/common/swob.py", line 994, in call_application#012 app_iter = application(self.environ, start_response)#012 File "/home/osddev/swift/swift/common/middleware/crossdomain.py", line 82, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/swob.py", line 1396, in _wsgify_self#012 return func(self, Request(env))(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/tempauth.py", line 234, in call#012 return self.app(env, start_response)#012 File "build/bdist.linux-x86_64/egg/swifthlm/middleware.py", line 293, in call#012 hlm_req = str.lower(cmd_ifhlm)#012TypeError: descriptor 'lower' requires a 'str' object but received a 'NoneType' (txn: txace2d026ce854f7a83332-00593a50c6) Jun 9 15:39:50 localhost proxy-server: STDERR: 127.0.0.1 - - [09/Jun/2017 07:39:50] "POST /hlm/v1/migrate/AUTH_test/container_test HTTP/1.1" 500 179 0.006708 (txn: txace2d026ce854f7a83332-00593a50c6)

Jun 9, 2017 Reply from @hseipp :

your keyless setup, Swift process status and stat as well as your container listing all look good.

The proxy error log entry is due to an HLM operation parsing error (middleware.py, line 293) which does not really seem to match the request example

$ curl -v -H "X-Auth-Token: AUTH_tkc7e630b2c3074fbe801ed5dc2a8c7710" -X POST "http://127.0.0.1:8080/hlm/v1/migrate/AUTH_test/container_test"

you provided below which does have a correct syntax.

Can you please follow the steps described in the troubleshooting section of the just-published Redpaper http://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/redp5430.html?Open to enable debug logging and retry the migrate operation? For sure you can start with the dummy backend connector. It should actually perform the migrate operation without an error, but do nothing.

This comment is for record only.

Regards, Tommy

[tommyJin]

Hi @hseipp ,

I append text below

[hlm] set log_level = DEBUG

to each .conf file under '/etc/swift/object-server' (there are actually 4 conf files), and each conf file looks like the same, I just post one of them.

[osddev@localhost config]$ cat /etc/swift/object-server/1.conf [DEFAULT] devices = /srv/1/node mount_check = false disable_fallocate = true bind_ip = 127.0.0.1 bind_port = 6010 workers = 1 user = osddev log_facility = LOG_LOCAL2 recon_cache_path = /var/cache/swift eventlet_debug = true

[pipeline:main] pipeline = recon object-server

[app:object-server] use = egg:swift#object

[filter:recon] use = egg:swift#recon

[object-replicator] rsync_module = {replication_ip}::object{replication_port}

[object-reconstructor]

[object-updater]

[object-auditor]

[hlm] set log_level = DEBUG

swifthlmlog.conf Create file '/etc/rsyslog.d/00-swifthlmlog.conf' and append text below to the file:

if $programname contains 'hlm-middleware' then /var/log/swift/hlm.log if $programname contains 'hlm-middleware' and $syslogseverity <= 3 then /var/log/swift/hlm.error if $programname contains 'hlm-middleware' then ~ if $programname contains 'dispatcher' then /var/log/swift/hlm.log if $programname contains 'dispatcher' and $syslogseverity <= 3 then /var/log/swift/hlm.error if $programname contains 'dispatcher' then ~ if $programname contains 'handler' then /var/log/swift/hlm.log if $programname contains 'handler' and $syslogseverity <= 3 then /var/log/swift/hlm.error if $programname contains 'handler' then ~ if $programname contains 'connector' then /var/log/swift/hlm.log if $programname contains 'connector' and $syslogseverity <= 3 then /var/log/swift/hlm.error if $programname contains 'connector' then ~

Then I restart rsyslog

systemctl restart rsyslog

The objects of container:

[osddev@localhost swift]$ curl -H "X-Auth-Token: AUTH_tkc09a344b4d5c470b8fac6285ae3321a6" -X GET "http://127.0.0.1:8080/v1/AUTH_test/container1" obj testobject1/testresource/bulk-test testobject1/testresource/subfolder1/subfile1 testobject1/testresource/subfolder2/subfile2

Then I retry the 'migrate', the content of hlm.log is like below:

osddev@localhost swift]$ curl -H "X-Auth-Token: AUTH_tkc09a344b4d5c470b8fac6285ae3321a6" -X POST "http://127.0.0.1:8080/hlm/v1/migrate/AUTH_test/container1/obj" An error occurred[osddev@lsudo vi hlm.log [osddev@localhost swift]$ sudo cat hlm.log Jun 12 18:18:05 localhost nm-dispatcher: Dispatching action 'dhcp4-change' for eno16777736

The error shown in proxy.error is like below:

Jun 12 18:19:35 localhost proxy-server: STDERR: (55901) accepted ('127.0.0.1', 45559) Jun 12 18:19:35 localhost proxy-server: Error: An error occurred: #012Traceback (most recent call last):#012 File "/home/osddev/swift/swift/common/middleware/catch_errors.py", line 41, in handle_request#012 resp = self._app_call(env)#012 File "/home/osddev/swift/swift/common/wsgi.py", line 1062, in _app_call#012 resp = self.app(env, self._start_response)#012 File "/home/osddev/swift/swift/common/middleware/gatekeeper.py", line 102, in call#012 return self.app(env, gatekeeper_response)#012 File "/home/osddev/swift/swift/common/middleware/healthcheck.py", line 52, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/proxy_logging.py", line 346, in call#012 six.reraise(exc_type, exc_value, exc_traceback)#012 File "/home/osddev/swift/swift/common/middleware/proxy_logging.py", line 338, in call#012 iterable = self.app(env, my_start_response)#012 File "/home/osddev/swift/swift/common/middleware/memcache.py", line 109, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/swob.py", line 1396, in _wsgify_self#012 return func(self, Request(env))(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/tempurl.py", line 346, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/ratelimit.py", line 299, in call#012 ratelimit_resp = self.handle_ratelimit(req, account, container, obj)#012 File "/home/osddev/swift/swift/common/middleware/ratelimit.py", line 236, in handle_ratelimit#012 swift_source='RL')#012 File "/home/osddev/swift/swift/proxy/controllers/base.py", line 389, in get_account_info#012 resp = req.get_response(app)#012 File "/home/osddev/swift/swift/common/swob.py", line 1008, in get_response#012 status, headers, app_iter = self.call_application(application)#012 File "/home/osddev/swift/swift/common/swob.py", line 994, in call_application#012 app_iter = application(self.environ, start_response)#012 File "/home/osddev/swift/swift/common/middleware/crossdomain.py", line 82, in call#012 return self.app(env, start_response)#012 File "/home/osddev/swift/swift/common/swob.py", line 1396, in _wsgify_self#012 return func(self, Request(env))(env, start_response)#012 File "/home/osddev/swift/swift/common/middleware/tempauth.py", line 234, in call#012 return self.app(env, start_response)#012 File "build/bdist.linux-x86_64/egg/swifthlm/middleware.py", line 293, in call#012 hlm_req = str.lower(cmd_ifhlm)#012TypeError: descriptor 'lower' requires a 'str' object but received a 'NoneType' (txn: tx90117500c6c94b6eb7c4c-00593e6ab7) Jun 12 18:19:35 localhost proxy-server: STDERR: 127.0.0.1 - - [12/Jun/2017 10:19:35] "POST /hlm/v1/migrate/AUTH_test/container1/obj HTTP/1.1" 500 179 0.002960 (txn: tx90117500c6c94b6eb7c4c-00593e6ab7)

BTW, I found that when the code is executed below, ` req = Request(env) self.req = req

if not self.swift:

    #    self.create_internal_swift_client()

    # Split request path to determine version, account, container, object
    try:
        (namespace, ver_ifhlm, cmd_ifhlm, acc_ifhlm, con_ifhlm, obj_ifhlm)\
                = req.split_path(2, 6, True)
    except ValueError:
        self.logger.debug('split_path exception')
        return self.app(env, start_response)
    self.logger.debug(':%s:%s:%s:%s:%s:%s:', namespace, ver_ifhlm,
            cmd_ifhlm, acc_ifhlm, con_ifhlm, obj_ifhlm) 

` it seems that req is a ver_ifhlm, cmd_ifhlm are NoneType, maybe others are the same after I try to write them to a file.

Before it happens, I did not modify any source code and just

python setup.py install

Regards, Tommy

[tommyJin]

Hi @hseipp ,

Today I try to reinstall the older version Liberty 2.5.0 of openstack Swift, and it works as expected.

This issue is ok for now, but the remaining problem is that errors shown in previous comments would occur when version 0.22 of SwiftHLM cooperates with the Swift Ocata.

Temporarily Solution(2017/06/13):

1. Install older version of swift(Liberty see above)
2. Reconfig swifthlm

I'll keep updating this issue if necessary.

Thanks for the help from @hseipp . :)

Regards, Tommy

[tommyJin]

The 2017/06/20 update fixes the swift-version-issue. See the update info.