Closed Kami closed 5 years ago
I noticed that Ubuntu Bionic tests fail every now and then on run bootstrap script task.
It turns out it fails because st2auth service is not started.
st2auth
... 0190522T104326+0000 Adding password for user st2admin 20190522T104337+0000 Key written to /etc/st2/keys/datastore_key.json. Secure the permissions so only StackStorm API process and StackStorm admin access the file. 20190522T104348+0000 st2 3.1dev (09d5948a9), on Python 3.6.5 20190522T104348+0000 usage: st2 [-h] [--version] [--url BASE_URL] [--auth-url AUTH_URL] 20190522T104348+0000 [--api-url API_URL] [--stream-url STREAM_URL] 20190522T104348+0000 [--api-version API_VERSION] [--cacert CACERT] 20190522T104348+0000 [--config-file CONFIG_FILE] [--print-config] [--skip-config] 20190522T104348+0000 [--debug] 20190522T104348+0000 {run,action,action-alias,auth,login,whoami,apikey,execution,inquiry,key,pack,policy,policy-type,rule,webhook,timer,runner,sensor,trace,trigger,trigger-instance,rule-enforcement,workflow,service-registry,role,role-assignment} 20190522T104348+0000 ... 20190522T104348+0000 20190522T104348+0000 CLI for StackStorm event-driven automation platform. https://stackstorm.com 20190522T104348+0000 20190522T104348+0000 positional arguments: 20190522T104348+0000 {run,action,action-alias,auth,login,whoami,apikey,execution,inquiry,key,pack,policy,policy-type,rule,webhook,timer,runner,sensor,trace,trigger,trigger-instance,rule-enforcement,workflow,service-registry,role,role-assignment} ... 20190522T104349+0000 ERROR: HTTPConnectionPool(host='127.0.0.1', port=9100): Max retries exceeded with url: /tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7feba811e240>: Failed to establish a new connection: [Errno 111] Connection refused',)) 20190522T104349+0000 20190522T104349+0000 ############### ERROR ############### 20190522T104349+0000 # Failed on Verify st2 # 20190522T104349+0000 #####################################"
After digging in, it turns out there is a race between the time when we configure authentication in st2.conf and staring / restarting st2 services.
If st2auth is started before auth is configured in st2.conf, it won't work correctly.
I will take the liberty to merge this into master and v3.0 because I'm blocked on v3.0.1 release and there are tons of other changes and fixes I need to make, to make progress on the release.
I noticed that Ubuntu Bionic tests fail every now and then on run bootstrap script task.
It turns out it fails because
st2auth
service is not started.After digging in, it turns out there is a race between the time when we configure authentication in st2.conf and staring / restarting st2 services.
If st2auth is started before auth is configured in st2.conf, it won't work correctly.