search

pndaproject / pnda-heat-templates

Resources for launching PNDA on Openstack
Other
2 stars 26 forks source link

Swift Auth Error #64

Open sheetaljoshi opened 7 years ago

sheetaljoshi commented 7 years ago

I am trying to setup pnda standard on openstack 3.0.0 version. Pnda_cluster deploy fails with below error. Your help is greatly appreciated. I am stuck at this point and can’t proceed further.

I have created containers in admin tenant. And below are my pnda_env.yaml settings.

keystone_user: 'admin'

#

keystone_user: Password for the openstack clients to use

# keystone_password: 'admin1'

#

keystone_tenant: Name of the tenant / project in the openstack environment. The

PNDA stack will be created in this project.

# keystone_tenant: 'service'

#

keystone_auth_url: Keystone authentication URL. The Openstack console provides this

under the Access & Security section.

I am trying to setup pnda standard on openstack 3.0.0 version. Pnda_cluster deploy fails with below error. Your help is greatly appreciated. I am stuck at this point and can’t proceed further.

I have created containers in admin tenant. And below are my pnda_env.yaml settings.

keystone_user: 'admin'

#

keystone_user: Password for the openstack clients to use

# keystone_password: 'admin1'

#

keystone_tenant: Name of the tenant / project in the openstack environment. The

PNDA stack will be created in this project.

# keystone_tenant: 'admin'

#

keystone_auth_url: Keystone authentication URL. The Openstack console provides this

under the Access & Security section.

Resource Create Failed: Remoteerror: Resources.Deploy Install: Remote Error: Clientexception Object Head Failed: Http://147.75.99.215:8080/V1/Auth 88798ee06ebd4409be26e437cd07ec35/A4913856-C4e6-4a3c-9b5d-647e3767ccdd/Pnda-Cluster-Deploy Install-4hzazk5l5obh 401 Unauthorized [U'Traceback (Most Recent Call Last):\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Oslo Messaging/Rpc/Server.Py", Line 133, In Process Incoming\N Res = Self.Dispatcher.Dispatch(Message)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Oslo Messaging/Rpc/Dispatcher.Py", Line 150, In Dispatch\N Return Self. Do Dispatch(Endpoint, Method, Ctxt, Args)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Oslo Messaging/Rpc/Dispatcher.Py", Line 121, In Do Dispatch\N Result = Func(Ctxt, New Args)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Osprofiler/Profiler.Py", Line 154, In Wrapper\N Return F(*Args, *Kwargs)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Heat/Common/Context.Py", Line 424, In Wrapped\N Return Func(Self, Ctx, Args, Kwargs)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Heat/Engine/Service.Py", Line 2215, In Show Software Deployment\N Cnxt, Deployment Id)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Heat/Engine/Service Software Config.Py", Line 239, In Show Software Deployment\N Sd = Self. Show Software Deployment(Cnxt, Deployment Id)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Heat/Engine/Service Software Config.Py", Line 232, In Show Software Deployment\N Cnxt, Sd, Input Values.Get(\'Deploy Signal Id\'))\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Heat/Engine/Service Software Config.Py", Line 136, In Refresh Swift Software Deployment\N Headers = Swift.Head Object(Container, Object Name)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Swiftclient/Client.Py", Line 1744, In Head Object\N Return Self. Retry(None, Head Object, Container, Obj, Headers=Headers)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Swiftclient/Client.Py", Line 1647, In Retry\N Service Token=Self.Service Token, **Kwargs)\N', U' File "/Var/Lib/Kolla/Venv/Local/Lib/Python2.7/Site-Packages/Swiftclient/Client.Py", Line 1186, In Head Object\N Raise Clientexception.From Response(Resp, \'Object Head Failed\', Body)\N', U'Clientexception: Object Head Failed: Http://147.75.99.215:8080/V1/Auth 88798ee06ebd4409be26e437cd07ec35/A4913856-C4e6-4a3c-9b5d-647e3767ccdd/Pnda-Cluster-Deploy Install-4hzazk5l5obh 401 Unauthorized\N'].

qianyi3210 commented 7 years ago

i encounter same issue with openstack mitaka, pnda 3.4 ,any suggestion?Your help is greatly appreciated.

klyr commented 7 years ago

Hi,

Are you sure about the credentials you are using ? You need to source the Openstack script downloaded from the Openstack UI in the security section, before launching deployment.

If you're using Openstack 3 (Havan), we don't support it. We only support Kilo and later.

The 'keystoneuser*' parameters in the pnda_env.yaml file are used for archiving in Openstack swift at PNDA run time, not PNDA deployment time.

Maybe we should make it clearer :)

qianyi3210 commented 7 years ago

thanks very much, i use below command, is it ok?
"./heat_cli.py -e cation -f pico -s pnda create"

qianyi3210 commented 7 years ago

I have used the downloaded credentials file,still have this issue. credential file as below:

export OS_AUTH_URL=http://controller:5000/v3 export OS_PROJECT_ID=4f279bfeb5e04bc7b59ec8ed7fd5dedf export OS_PROJECT_NAME="demo" export OS_USER_DOMAIN_NAME="default" if [ -z "$OS_USER_DOMAIN_NAME" ]; then unset OS_USER_DOMAIN_NAME; fi unset OS_TENANT_ID unset OS_TENANT_NAME export OS_USERNAME="demo" echo "Please enter your OpenStack Password: " read -sr OS_PASSWORD_INPUT export OS_PASSWORD=$OS_PASSWORD_INPUT export OS_REGION_NAME="RegionOne" if [ -z "$OS_REGION_NAME" ]; then unset OS_REGION_NAME; fi

failed log: 017-03-30 06:16:33 [cation]: CREATE_IN_PROGRESS Stack CREATE started 2017-03-30 06:16:33 [private_net]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:33 [Key]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:34 [PndaSecGroup]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:34 [router]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [highstate_config]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [install_config]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [expand_config]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [saltmaster_sec_group]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [orchestrate_config]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:35 [Key]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [install_config]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [PndaSecGroup]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [expand_config]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [saltmaster_sec_group]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [orchestrate_config]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [private_net]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [router]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [highstate_config]: CREATE_COMPLETE state changed 2017-03-30 06:16:35 [private_subnet]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:37 [private_subnet]: CREATE_COMPLETE state changed 2017-03-30 06:16:37 [saltmaster_port]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:37 [router_interface]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:39 [saltmaster_port]: CREATE_COMPLETE state changed 2017-03-30 06:16:39 [router_interface]: CREATE_COMPLETE state changed 2017-03-30 06:16:40 [saltmaster_server]: CREATE_IN_PROGRESS state changed 2017-03-30 06:16:48 [saltmaster_server]: CREATE_COMPLETE state changed 2017-03-30 06:16:48 [deploy_install]: CREATE_IN_PROGRESS state changed 2017-03-30 07:16:30 [deploy_install]: CREATE_FAILED RemoteError: resources.deploy_install: Remote error: ClientException Object HEAD failed: http://controller:8080/v1/AUTH_4f279bfeb5e04bc7b59ec8ed7fd5dedf/a12b0764-ecb1-4b06-9e32-a2a5dd9fdfc1/cation-deploy_install-7rvrb6oneahl 401 Unauthorized [u'Traceback 2017-03-30 07:16:31 [cation]: CREATE_FAILED Resource CREATE failed: RemoteError: resources.deploy_install: Remote error: ClientException Object HEAD failed: http://controller:8080/v1/AUTH_4f279bfeb5e04bc7b59ec8ed7fd5dedf/a12b0764-ecb1-4b06-9e32-a2a5dd9fdfc1/cation-deploy_install-7rvrb6oneahl 401 Un

Stack cation CREATE_FAILED

klyr commented 7 years ago

Do you have swift available and configured ? Does swift list output something ?

qianyi3210 commented 7 years ago

yes, below is swift list's output:

 [root@client pnda-heat-templates]# swift list
03d9bc04-72e9-452f-8f2d-316f2699f9ed
18b1b9c8-e3d0-4ec3-9138-02e42e393dbc
cation-saltmaster_server-4cvwyzlre5c2
cation-saltmaster_server-b7hvreyqw7mf
cation-saltmaster_server-c7utkuff3tai
cation-saltmaster_server-cznzlcifw6g3
cation-saltmaster_server-gdfgjse4nqlm
cation-saltmaster_server-los3py2gnpkb
cation-saltmaster_server-nqvla6mgxkhy
cation-saltmaster_server-nudanxvf4vwo
cation-saltmaster_server-rmq4ovq6ocis
cation-saltmaster_server-x75c75pe5twg
mk
plm
pnda_apps_container
pnda_archives
polikm
test
[root@client pnda-heat-templates]#

this problem still exist.

qianyi3210 commented 7 years ago

this problem was resolved now. :), Because of too long deploying time which out of token's expiration time. I reset from 3600 to 86400. it work now. but still has other problem. my script always stay in [deploy_instll] step and repeat these log too long time :

RESP: [200] Content-Type: application/json; charset=UTF-8 Content-Length: 2542 X-Openstack-Request-Id: req-818a6e8e-7713-448b-99c3-90ea7b73fdb3 Date: Fri, 31 Mar 2017 11:05:09 GMT Connection: keep-alive
RESP BODY: {"events": [{"resource_name": "router_interface", "event_time": "2017-03-31T11:05:05", "links": [{"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/router_interface/events/779b502d-bc9b-446a-afa9-42fa9628c103", "rel": "self"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/router_interface", "rel": "resource"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab", "rel": "stack"}], "logical_resource_id": "router_interface", "resource_status": "CREATE_COMPLETE", "resource_status_reason": "state changed", "physical_resource_id": "075f065f-45b3-4e6b-89d6-4144f8ea2fdb:subnet_id=799fcc7a-19e1-4743-85fd-44c155ecdbd0", "id": "779b502d-bc9b-446a-afa9-42fa9628c103"}, {"resource_name": "saltmaster_port", "event_time": "2017-03-31T11:05:06", "links": [{"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/saltmaster_port/events/638ace77-513a-41cd-ae27-4c78a0f71a95", "rel": "self"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/saltmaster_port", "rel": "resource"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab", "rel": "stack"}], "logical_resource_id": "saltmaster_port", "resource_status": "CREATE_COMPLETE", "resource_status_reason": "state changed", "physical_resource_id": "e1d7d179-424a-49a0-9f80-645d0dbbe6c8", "id": "638ace77-513a-41cd-ae27-4c78a0f71a95"}, {"resource_name": "saltmaster_server", "event_time": "2017-03-31T11:05:06", "links": [{"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/saltmaster_server/events/18e484a8-c68a-4424-b72a-a562afb92c9a", "rel": "self"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/resources/saltmaster_server", "rel": "resource"}, {"href": "http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab", "rel": "stack"}], "logical_resource_id": "saltmaster_server", "resource_status": "CREATE_IN_PROGRESS", "resource_status_reason": "state changed", "physical_resource_id": null, "id": "18e484a8-c68a-4424-b72a-a562afb92c9a"}]}

GET call to None for http://controller:8004/v1/88cec6588c4545a3b3f15395f9a0c186/stacks/cation/1526cc7a-0309-47ce-884a-f57dd3889dab/events?marker=aae7e212-d6fe-459c-9b10-6f5b06f5426b&sort_dir=asc used request id req-818a6e8e-7713-448b-99c3-90ea7b73fdb3

and get below error log

2017-03-31 12:16:06 [cation]: CREATE_IN_PROGRESS  Stack CREATE started
2017-03-31 12:16:06 [install_config]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:06 [Key]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:06 [PndaSecGroup]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:06 [orchestrate_config]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:06 [private_net]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:07 [saltmaster_sec_group]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:07 [highstate_config]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:07 [expand_config]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:07 [router]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:08 [Key]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [saltmaster_sec_group]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [PndaSecGroup]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [private_net]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [highstate_config]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [orchestrate_config]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [install_config]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [expand_config]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [router]: CREATE_COMPLETE  state changed
2017-03-31 12:16:08 [private_subnet]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:09 [private_subnet]: CREATE_COMPLETE  state changed
2017-03-31 12:16:09 [saltmaster_port]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:10 [router_interface]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:12 [saltmaster_port]: CREATE_COMPLETE  state changed
2017-03-31 12:16:12 [router_interface]: CREATE_COMPLETE  state changed
2017-03-31 12:16:12 [saltmaster_server]: CREATE_IN_PROGRESS  state changed
2017-03-31 12:16:27 [saltmaster_server]: CREATE_COMPLETE  state changed
2017-03-31 12:16:27 [deploy_install]: CREATE_IN_PROGRESS  state changed
2017-03-31 15:16:06 [deploy_install]: CREATE_FAILED  CREATE aborted
2017-03-31 15:16:06 [cation]: CREATE_FAILED  Create timed out
qianyi3210 commented 7 years ago

seems stuck in swift object‘s creation for example: cation-deploy_install-hnrsz5lmch7v

IterZebra commented 7 years ago

I also meet the problem, then I also changed the expiration time, then same Timeout Error happened:

2017-06-20 09:36:04Z [cation-test.deploy_install]: CREATE_FAILED CREATE aborted (Task create from SoftwareDeployment "deploy_install" Stack "cation-test" [5f6684ff-5a07-409b-977f-d8b83fa859a5] Timed out) 2017-06-20 09:36:04Z [cation-test]: CREATE_FAILED Timed out

And I found error message in /var/log/syslog . I open another issue for the error https://github.com/pndaproject/pnda-heat-templates/issues/104 I don't know the relationship between them.

IterZebra commented 7 years ago

Now my error is :

2017-06-20 16:04:44Z [cation-new.deploy_install]: CREATE_FAILED MessagingTimeout: resources.deploy_install: Timed out waiting for a reply to message ID a59635437880435aa37ef1081fedab78 2017-06-20 16:04:44Z [cation-new]: CREATE_FAILED Resource CREATE failed: MessagingTimeout: resources.deploy_install: Timed out waiting for a reply to message ID a59635437880435aa37ef1081fedab78

nfss83 commented 7 years ago

Does someone figure out the solution? I also meet the problem with the timeout.