search

openstf / stf

Control and manage Android devices from your browser.
https://openstf.io
Other
13.38k stars 2.79k forks source link

TypeError: Key must be a buffer #1104

Open LightAutumnMelancholy opened 5 years ago

LightAutumnMelancholy commented 5 years ago

stf:latest

500s when trying to use OAUTH2 from MS in azure. I have removed and isolated the provider and adb stuff in a distributed fashion, so we typically run our rethink, triproxies, auth, nginx, reaper and processor app on a separate hosts in a distributed fashion, and allow individual sites to run provider and adb, since they have connections up and down from the host running the aforementioned, We have stable implementation when using mock authorization. However, this is not an acceptable long term solution for us, and I would like to use oauth2 for this.

Encountered error:



app_1                   | TypeError: Key must be a buffer
app_1                   |     at TypeError (native)
app_1                   |     at new Hmac (crypto.js:92:16)
app_1                   |     at Object.Hmac (crypto.js:90:12)
app_1                   |     at sign (/app/node_modules/keygrip/index.js:22:8)
app_1                   |     at Keygrip.index (/app/node_modules/keygrip/index.js:37:39)
app_1                   |     at Cookies.get (/app/node_modules/cookies/index.js:73:21)
app_1                   |     at tryGetSession (/app/node_modules/cookie-session/index.js:317:21)
app_1                   |     at IncomingMessage.getSession [as session] (/app/node_modules/cookie-session/index.js:91:22)
app_1                   |     at /app/lib/units/app/middleware/auth.js:30:17
app_1                   |     at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
app_1                   |     at trim_prefix (/app/node_modules/express/lib/router/index.js:317:13)
app_1                   |     at /app/node_modules/express/lib/router/index.js:284:7
app_1                   |     at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
app_1                   |     at next (/app/node_modules/express/lib/router/index.js:275:10)
app_1                   |     at _cookieSession (/app/node_modules/cookie-session/index.js:129:5)
app_1                   |     at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
app_1                   |     at trim_prefix (/app/node_modules/express/lib/router/index.js:317:13)
app_1                   |     at /app/node_modules/express/lib/router/index.js:284:7
app_1                   |     at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
app_1                   |     at next (/app/node_modules/express/lib/router/index.js:275:10)
app_1                   |     at favicon (/app/node_modules/serve-favicon/index.js:68:7)
app_1                   |     at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
nginx_1                 | 10.18.73.112 - - [24/Sep/2019:14:45:34 +0000] "GET / HTTP/1.1" 500 517 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36 OPR/62.0.3331.99"

Environment variables:

# GENERAL
PUBLIC_URL=openstf.subdomain.domain.com

# OAUTH2 ( I have randomized the actual secret and tokens, they are not real)
STF_AUTH_OAUTH2_OAUTH_AUTHORIZATION_URL="https://login.microsoftonline.com/as79e14d-5f58-416a-bc71-w28597ks018l/oauth2/v2.0/authorize"
STF_AUTH_OAUTH2_OAUTH_TOKEN_URL="https://login.microsoftonline.com/as79e14d-5f58-416a-bc71-w28597ks018l/oauth2/v2.0/token"
STF_AUTH_OAUTH2_OAUTH_USERINFO_URL="https://login.windows.net/common/openid/userinfo"
# False token for example
STF_AUTH_OAUTH2_OAUTH_CLIENT_ID="0743f413-30b3-4ab1-96d1-d947ad8e1a54"
# False secret for example
STF_AUTH_OAUTH2_OAUTH_CLIENT_SECRET='qwG#;rKv985:KSc=pcFG2kLeb-LkB_.'
STF_AUTH_OAUTH2_OAUTH_CALLBACK_URL="https://openstf.subdomain.domain.com/auth/oauth/callback"
STF_AUTH_OAUTH2_OAUTH_SCOPE="openid email"

# RethinkDB 
RETHINKDB_PORT_28015_TCP=tcp://rethinkdb:28015
STATION_NAME=openstf.subdomain.domain.com

STF is being run in a container and composed. Here is the applicable section:

  app:
    image: openstf/stf:latest
    restart: unless-stopped
    environment:
      - RETHINKDB_PORT_28015_TCP
      - PUBLIC_URL
    command: stf app --auth-url http://${PUBLIC_URL}/auth/oauth/callback/ --websocket-url ws://${PUBLIC_URL}/ --port 3000
    depends_on:
      - rethinkdb
      - auth
      - websocket
  auth:
    image: openstf/stf:latest
    restart: unless-stopped
    environment:
      - STF_AUTH_OAUTH2_OAUTH_AUTHORIZATION_URL
      - STF_AUTH_OAUTH2_OAUTH_TOKEN_URL
      - STF_AUTH_OAUTH2_OAUTH_USERINFO_URL
      - STF_AUTH_OAUTH2_OAUTH_CLIENT_ID
      - STF_AUTH_OAUTH2_OAUTH_CLIENT_SECRET
      - STF_AUTH_OAUTH2_OAUTH_CALLBACK_URL
      - STF_AUTH_OAUTH2_OAUTH_SCOPE
      - PUBLIC_URL

    command: stf auth-oauth2 --app-url http://${PUBLIC_URL}/ --port 3000
  processor:
    image: openstf/stf:latest
    restart: unless-stopped
    environment:
      - RETHINKDB_PORT_28015_TCP
    command: stf processor --connect-app-dealer tcp://triproxy:7160 --connect-dev-dealer tcp://dev-triproxy:7260
    depends_on:
      - rethinkdb
      - triproxy
      - dev-triproxy
  triproxy:

Full docker startup routine: 

Creating processor_node_rethinkdb_1    ... done
Creating processor_node_triproxy_1     ... done
Creating processor_node_dev-triproxy_1 ... done
Creating processor_node_auth_1         ... done
Creating processor_node_storage-temp_1 ... done
Creating processor_node_storage-plugin-apk_1   ... done
Creating processor_node_storage-plugin-image_1 ... done
Creating processor_node_migrate_1              ... done
Creating processor_node_processor_1            ... done
Creating processor_node_reaper_1               ... done
Creating processor_node_api_1                  ... done
Creating processor_node_websocket_1            ... done
Creating processor_node_app_1                  ... done
Creating processor_node_nginx_1                ... done
Attaching to processor_node_storage-temp_1, processor_node_rethinkdb_1, processor_node_dev-triproxy_1, processor_node_auth_1, processor_node_triproxy_1, processor_node_storage-plugin-image_1, processor_node_migrate_1, processor_node_storage-plugin-apk_1, processor_node_processor_1, processor_node_reaper_1, processor_node_websocket_1, processor_node_api_1, processor_node_app_1, processor_node_nginx_1
rethinkdb_1             | Running rethinkdb 2.3.6~0jessie (GCC 4.9.2)...
dev-triproxy_1          | 2019-09-24T14:45:07.689Z INF/triproxy 1 [dev] PUB socket bound on tcp://*:7250
storage-temp_1          | 2019-09-24T14:45:07.913Z INF/storage:temp 1 [*] Listening on port 3000
rethinkdb_1             | Running on Linux 5.1.19-300.fc30.x86_64 x86_64
rethinkdb_1             | Loading data from directory /data/rethinkdb_data
dev-triproxy_1          | 2019-09-24T14:45:07.691Z INF/triproxy 1 [dev] DEALER socket bound on tcp://*:7260
rethinkdb_1             | Listening for intracluster connections on port 29015
triproxy_1              | 2019-09-24T14:45:07.770Z INF/triproxy 1 [app] PUB socket bound on tcp://*:7150
dev-triproxy_1          | 2019-09-24T14:45:07.692Z INF/triproxy 1 [dev] PULL socket bound on tcp://*:7270
rethinkdb_1             | Listening for client driver connections on port 28015
triproxy_1              | 2019-09-24T14:45:07.772Z INF/triproxy 1 [app] DEALER socket bound on tcp://*:7160
auth_1                  | 2019-09-24T14:45:07.935Z INF/auth-oauth2 1 [*] Listening on port 3000
migrate_1               | 2019-09-24T14:45:08.503Z INF/db 1 [*] Connecting to rethinkdb:28015
storage-plugin-image_1  | 2019-09-24T14:45:08.554Z INF/storage:plugins:image 1 [*] Listening on port 3000
triproxy_1              | 2019-09-24T14:45:07.774Z INF/triproxy 1 [app] PULL socket bound on tcp://*:7170
rethinkdb_1             | Listening for administrative HTTP connections on port 8080
rethinkdb_1             | Listening on cluster addresses: 127.0.0.1, 172.19.0.2, ::1, fe80::42:acff:fe13:2%173
storage-plugin-apk_1    | 2019-09-24T14:45:08.748Z INF/storage:plugins:apk 1 [*] Listening on port 3000
rethinkdb_1             | Listening on driver addresses: 127.0.0.1, 172.19.0.2, ::1, fe80::42:acff:fe13:2%173
processor_1             | 2019-09-24T14:45:08.661Z INF/db 1 [*] Connecting to rethinkdb:28015
rethinkdb_1             | Listening on http addresses: 127.0.0.1, 172.19.0.2, ::1, fe80::42:acff:fe13:2%173
migrate_1               | 2019-09-24T14:45:08.527Z INF/db:setup 1 [*] Database "stf" already exists
websocket_1             | 2019-09-24T14:45:09.606Z INF/websocket 1 [*] Subscribing to permanent channel "*ALL"
processor_1             | 2019-09-24T14:45:08.678Z INF/processor 1 [0ef890736e69] App dealer connected to "tcp://triproxy:7160"
migrate_1               | 2019-09-24T14:45:08.548Z INF/db:setup 1 [*] Table "users" already exists
rethinkdb_1             | Server ready, "3d73a3501080_dfe" 0743f413-30b3-4ab1-96d1-d947ad8e1a54
reaper_1                | 2019-09-24T14:45:09.131Z INF/reaper 1 [dev] Subscribing to permanent channel "*ALL"
websocket_1             | 2019-09-24T14:45:09.611Z INF/websocket 1 [*] Listening on port 3000
processor_1             | 2019-09-24T14:45:08.679Z INF/processor 1 [0ef890736e69] Device dealer connected to "tcp://dev-triproxy:7260"
websocket_1             | 2019-09-24T14:45:09.614Z INF/db 1 [*] Connecting to rethinkdb:28015
websocket_1             | 2019-09-24T14:45:09.619Z INF/websocket 1 [*] Sending output to "tcp://triproxy:7170"
migrate_1               | 2019-09-24T14:45:08.549Z INF/db:setup 1 [*] Table "accessTokens" already exists
websocket_1             | 2019-09-24T14:45:09.620Z INF/websocket 1 [*] Receiving input from "tcp://triproxy:7150"
migrate_1               | 2019-09-24T14:45:08.549Z INF/db:setup 1 [*] Table "vncauth" already exists
reaper_1                | 2019-09-24T14:45:09.133Z INF/reaper 1 [dev] Reaping devices with no heartbeat
migrate_1               | 2019-09-24T14:45:08.549Z INF/db:setup 1 [*] Table "devices" already exists
migrate_1               | 2019-09-24T14:45:08.549Z INF/db:setup 1 [*] Table "logs" already exists
nginx_1                 | Got nameserver 127.0.0.11 from resolv.conf
nginx_1                 | Writing include file at /etc/nginx/conf.d/resolver.conf
migrate_1               | 2019-09-24T14:45:08.561Z INF/db:setup 1 [*] Index "vncauth"."response" already exists
reaper_1                | 2019-09-24T14:45:09.136Z INF/db 1 [dev] Connecting to rethinkdb:28015
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Index "vncauth"."responsePerDevice" already exists
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Index "devices"."owner" already exists
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Index "devices"."present" already exists
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Waiting for index "vncauth"."response"
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Waiting for index "vncauth"."responsePerDevice"
migrate_1               | 2019-09-24T14:45:08.562Z INF/db:setup 1 [*] Waiting for index "devices"."owner"
migrate_1               | 2019-09-24T14:45:08.564Z INF/db:setup 1 [*] Waiting for index "devices"."present"
reaper_1                | 2019-09-24T14:45:09.140Z INF/reaper 1 [dev] Receiving input from "tcp://triproxy:7150"
migrate_1               | 2019-09-24T14:45:08.565Z INF/db:setup 1 [*] Index "devices"."providerChannel" already exists
migrate_1               | 2019-09-24T14:45:08.566Z INF/db:setup 1 [*] Waiting for index "devices"."providerChannel"
migrate_1               | 2019-09-24T14:45:08.578Z INF/db:setup 1 [*] Index "devices"."owner" is ready
migrate_1               | 2019-09-24T14:45:08.578Z INF/db:setup 1 [*] Index "devices"."present" is ready
migrate_1               | 2019-09-24T14:45:08.578Z INF/db:setup 1 [*] Index "vncauth"."response" is ready
migrate_1               | 2019-09-24T14:45:08.578Z INF/db:setup 1 [*] Index "vncauth"."responsePerDevice" is ready
migrate_1               | 2019-09-24T14:45:08.578Z INF/db:setup 1 [*] Index "devices"."providerChannel" is ready
migrate_1               | 2019-09-24T14:45:09.304Z INF/db:setup 1 [*] Index "users"."adbKeys" already exists
migrate_1               | 2019-09-24T14:45:09.304Z INF/db:setup 1 [*] Waiting for index "users"."adbKeys"
reaper_1                | 2019-09-24T14:45:09.141Z INF/reaper 1 [dev] Sending output to "tcp://dev-triproxy:7270"
migrate_1               | 2019-09-24T14:45:09.306Z INF/db:setup 1 [*] Index "accessTokens"."email" already exists
migrate_1               | 2019-09-24T14:45:09.306Z INF/db:setup 1 [*] Waiting for index "accessTokens"."email"
migrate_1               | 2019-09-24T14:45:09.307Z INF/db:setup 1 [*] Index "users"."adbKeys" is ready
migrate_1               | 2019-09-24T14:45:09.308Z INF/db:setup 1 [*] Index "accessTokens"."email" is ready
processor_node_migrate_1 exited with code 0
api_1                   | 2019-09-24T14:45:09.838Z INF/api 1 [*] Subscribing to permanent channel "*ALL"
api_1                   | 2019-09-24T14:45:09.881Z INF/api 1 [*] Listening on port 3000
api_1                   | 2019-09-24T14:45:09.887Z INF/api 1 [*] Sending output to "tcp://triproxy:7170"
api_1                   | 2019-09-24T14:45:09.888Z INF/api 1 [*] Receiving input from "tcp://triproxy:7150"
app_1                   | 2019-09-24T14:45:09.919Z INF/app 1 [*] Using pre-built resources
app_1                   | 2019-09-24T14:45:09.930Z INF/app 1 [*] Listening on port 3000
app_1                   | 2019-09-24T14:45:09.932Z INF/db 1 [*] Connecting to rethinkdb:28015
api_1                   | 2019-09-24T14:45:10.066Z INF/db 1 [*] Connecting to rethinkdb:28015

I'm trying to figure out what I am doing wrong here. I feel that one of my variable must be in question here.
issue-label-bot[bot] commented 5 years ago

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.67. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

LightAutumnMelancholy commented 5 years ago

Hmm, So it looks like my auth-url for app was incorrect. However after correcting this, I now have about 18 x 302 redirects and a 502 finally, using the reference docker image for nginx. Again, this is using OAUTH2 from Azure.

image