API creation failure: Unable to configure the API Gateway: "Invalid URI \"undefined/tenants\""

[Dalfons0]

I'm trying to deploy a service with this plugin to a local minikube cluster, if I deploy the service without any kind of event binded to the handler the deploy works perfectly, but If I try to specify an http event to the handler I get this error:

But the thing is that I can create a functional api endpoint with the CLI without problems:

[wellermiranda]

Same here.

[jthomas]

Oh dear this looks bad! The API Gateway integration is a bit complex, I have an idea what's wrong but need to check.

I'm not a k8s person, can you provide simple instructions for me to get OW running on minikube so I can test this out?

[Dalfons0]

I cloned this repository and followed the instrucctions in the README to set up OpenWhisk. You can also try to make the set up with the travis build script, in the same repo, but that didn't work for me 😢 .

[jthomas]

:wave: everyone.

I've managed to set up minikube locally from the repo and get OpenWhisk deployed. Unfortunately, I can't replicate that error with my setup. It all works fine when creating new APIs, which is good but doesn't really help resolve this issue!

Here is the example I'm using...

service: temp

provider:
  name: openwhisk
  ignore_certs: true

functions:
  hello:
    handler: handler.hello
    events:
      - http: GET /api/greeting

plugins:
  - serverless-openwhisk

'use strict';

function hello(params) {
  const name = params.name || 'World';
  return { payload: `Hello, ${name}!` };
}

exports.hello = hello;

Can you run the deploy command again with the debug env param (DEBUG=needle) and paste me the log?

$ DEBUG=needle serverless deploy

I also could use the logs from the API Gateway container. If you minikube ssh and look for the container using the openwhisk/apigateway image. Use docker logs to retrieve these logs. It should be like this example.

==> /var/log/api-gateway/access.log <==
LOGS
==> /var/log/api-gateway/management.log <==
LOGS
==> /var/log/api-gateway/access.log <==
LOGS

[jthomas]

I'd also had another random thought, can you verify the framework is working against another OpenWhisk instance? I want to check this is a problem with the minikube deployment and not your environment and all platforms.

If you sign up for a free account with IBM Cloud, you use test the framework with IBM Cloud Functions, which is a managed OpenWhisk service. I've got some instructions on doing this here: https://github.com/jthomas/openwhisk-workshops/tree/master/bootcamp/ex0%20-%20setting%20up%20development%20environment

[Dalfons0]

I removed the minikube instance because right now I have been reasigned to a quite critical project, so I will be faraway for the moment of my serverless research (sadly). But I have put the state of my investigations in a repo, and I'll conctact @dario-rodriguez and see if he can lend a hand.

[timonback]

I had a similar issue. It works in the IBM Cloud, but as it turns out, the apigateway image was not running in my local vagrant box from the original OpenWhisk project (might be related to a reboot?). Running this helped:

cd ${ANSIBLE_HOME}
ansible-playbook -i environments/local apigateway.yml

Short note about the output of DEBUG=needle serverless deploy: The requests after Serverless: Deploying API Gateway definitions... start throwing HTTP 502 errors.

[dario-rodriguez]

I got the same error: Failed to deploy API Gateway route (hello) due to error: POST https://10.68.14.217:31615/api/v1/web/whisk.system/apimgmt/createApi.http Returned HTTP 502 (Bad Gateway) --> "API creation failure: Unable to configure the API Gateway: "Invalid URI \"undefined/tenants\"""

Serverless DEBUG=needle output:

needle Making request #1 { rejectUnauthorized: false,
  agent: false,
  protocol: 'https:',
  host: '10.68.14.217',
  port: '31615',
  path: '/api/v1/namespaces/_/actions/OpenWhiskTest-dev-hello?overwrite=true',
  method: 'put',
  headers:
   { accept: 'application/json',
     'user-agent': 'Needle/2.2.0 (Node.js v9.5.0; win32 x64)',
     authorization: 'Basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=',
     'content-length': 3368225,
     'content-type': 'application/json; charset=utf-8',
     host: '10.68.14.217:31615' } } +0ms
  needle Got response 200 { server: 'nginx/1.11.13',
  date: 'Tue, 03 Apr 2018 10:42:36 GMT',
  'content-type': 'application/json',
  'content-length': '3368364',
  connection: 'close',
  'access-control-allow-origin': '*',
  'access-control-allow-headers': 'Authorization, Content-Type' } +1s
Serverless: Deploying API Gateway definitions...
  needle Making request #1 { rejectUnauthorized: false,
  agent: false,
  protocol: 'https:',
  host: '10.68.14.217',
  port: '31615',
  path: '/api/v1/web/whisk.system/apimgmt/deleteApi.http?basepath=%2FOpenWhiskTest&force=true',
  method: 'delete',
  headers:
   { accept: 'application/json',
     'user-agent': 'Needle/2.2.0 (Node.js v9.5.0; win32 x64)',
     authorization: 'Basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=',
     host: '10.68.14.217:31615' } } +97ms
  needle Got response 502 { server: 'nginx/1.11.13',
  date: 'Tue, 03 Apr 2018 10:42:37 GMT',
  'content-type': 'application/json',
  'content-length': '121',
  connection: 'close',
  'access-control-allow-origin': '*',
  'access-control-allow-methods': 'OPTIONS, GET, DELETE, POST, PUT, HEAD, PATCH',
  'access-control-allow-headers': 'Authorization, Content-Type' } +764ms
  needle Making request #1 { rejectUnauthorized: false,
  agent: false,
  protocol: 'https:',
  host: '10.68.14.217',
  port: '31615',
  path: '/api/v1/web/whisk.system/apimgmt/createApi.http',
  method: 'post',
  headers:
   { accept: 'application/json',
     'user-agent': 'Needle/2.2.0 (Node.js v9.5.0; win32 x64)',
     authorization: 'Basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A=',
     'content-length': 424,
     'content-type': 'application/json; charset=utf-8',
     host: '10.68.14.217:31615' } } +20ms
  needle Got response 502 { server: 'nginx/1.11.13',
  date: 'Tue, 03 Apr 2018 10:42:38 GMT',
  'content-type': 'application/json',
  'content-length': '112',
  connection: 'close',
  'access-control-allow-origin': '*',
  'access-control-allow-methods': 'OPTIONS, GET, DELETE, POST, PUT, HEAD, PATCH',
  'access-control-allow-headers': 'Authorization, Content-Type' } +802ms

Apigateway logs:

Starting api-gateway ... nginx version: openresty/1.13.6.1 built by gcc 6.4.0 (Alpine 6.4.0) built with OpenSSL 1.0.2n 7 Dec 2017 TLS SNI support enabled configure arguments: --prefix=/usr/local/api-gateway/nginx --with-cc-opt=-O2 --add-module=../ngx_devel_kit-0.3.0 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.05 --add-module=../ngx_coolkit-0.2rc3 --add-module=../set-misc-nginx-module-0.31 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.07 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.11 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.18 --add-module=../redis2-nginx-module-0.14 --add-module=../redis-nginx-module-0.3.7 --add-module=../rds-json-nginx-module-0.15 --add-module=../rds-csv-nginx-module-0.08 --add-module=../ngx_stream_lua-0.0.3 --with-ld-opt=-Wl,-rpath,/usr/local/api-gateway/luajit/lib --sbin-path=/usr/local/sbin/api-gateway --conf-path=/etc/api-gateway/api-gateway.conf --error-log-path=/var/log/api-gateway/error.log --http-log-path=/var/log/api-gateway/access.log --pid-path=/var/run/api-gateway.pid --lock-path=/var/run/api-gateway.lock --add-module=/tmp/api-gateway/openresty-1.13.6.1/../naxsi-0.53-2/naxsi_src --with-pcre=/tmp/api-gateway/openresty-1.13.6.1/../pcre-8.37 --with-pcre-jit --with-stream --with-stream_ssl_module --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_degradation_module --with-http_auth_request_module --with-http_v2_module --without-http_ssi_module --without-http_userid_module --without-http_uwsgi_module --without-http_scgi_module --with-stream --with-stream_ssl_module ... with dns resolver 10.96.0.10; ... testing configuration nginx: the configuration file /etc/api-gateway/api-gateway.conf syntax is ok nginx: configuration file /etc/api-gateway/api-gateway.conf test is successful ... using log level: 'warn'. Override it with -e 'LOG_LEVEL=' ==> /var/log/api-gateway/access.log <==

==> /var/log/api-gateway/error.log <==

==> /var/log/api-gateway/gateway_error.log <==

==> /var/log/api-gateway/management.log <==

[jthomas]

I haven't had time to look into this but I have just seen a similar error using the API Gateway when I had a missing APIGW_ACCESS_TOKEN from the authentication credentials.

Have you got the valid value for this token set in your credentials?

[kurtschwanda]

I'm getting the same error message:

Failed to deploy API Gateway route (/api/endpointName1) due to error: POST https://openwhisk.ng.bluemix.net/api/v1/web/whisk.system/apimgmt/createApi.http Returned HTTP 502 (Bad Gateway) --> "API creation failure: Unable to configure the API Gateway: "Invalid URI \"undefined/tenants\"""

Deploying functions with Serverless works just fine. But I can't seem to connect them to an API through my serverless.yaml. If I leave events out of it, things work fine.

I am a little confused about the APIGW_ACCESS_TOKEN. I definitely don't have that is my .wskprops file. I just have APIHOST and AUTH set in .wskprops. I found the values for APIHOST and AUTH here https://console.bluemix.net/openwhisk/learn/api-key but I'm not sure where to find, or create, the value that belongs in APIGW_ACCESS_TOKEN.

It sounds like a value for APIGW_ACCESS_TOKEN might fix my problems, any directions on where to find or generate that token?

[jthomas]

@kurtschwanda If you are using IBM Cloud Functions, the APIGW_ACCESS_TOKEN token should be auto-generated by any calls to the IBM Cloud CLI, e.g the bx wsk api list.

Can you try that and see if that fixes it?

If not, can you provide the following..

• serverless.yml used
• How you installed the CLI and configured authentication

[jthomas]

Closing this ticket for the time being due to lack of activity.

[pheuberger]

I'm experiencing this problem with the IBM Cloud DevOps pipeline. I'm trying to get CI/CD for my serverless project to work and the access token is seemingly not auto-generated when calling an IBM Cloud CLI command in the shell script. Perhaps due to file access restrictions in the Ubuntu instances they provision for the deployments. Do you have an idea, how to go about it @jthomas?

[jthomas]

@pheuberger hey - do you want to ping me on the openwhisk slack or DM me on Twitter (thomasj) and we can investigate?

[pheuberger]

Thanks @jthomas! I'm pinging you on Twitter 😊

[huksley]

Hi! I am trying OpenWhisk locally on docker and have the same error:

Unable to configure the API Gateway: "Invalid URI \"undefined/tenants\"""

Is there any way to fix this? I just added simple HTTP event to template project:

functions:
  hello:
    handler: handler.hello
    events:
      - http: GET /hello

Docker compose log

apigateway_1       | 10.6.3.137 - 23bc46b1-71f6-4ed5-8c54-816aa4f8c502 [20/Feb/2019:09:08:43 +0000] "POST /api/v1/web/whisk.system/apimgmt/createApi.http HTTP/1.1" 502 117 "-" "openwhisk-client-js

Docker log openwhisk_db_1

[info] 2019-02-20T09:12:23.304616Z nonode@nohost <0.2108.0> -------- OS Process #Port<0.8498> Log :: function raised exception (new TypeError("doc.binding is not an object", "undefined", 3)) with doc._id guest/my_service-dev-hello

[andreialecu]

Having the same issue referencing undefined/tenants on a local openwhisk installation on kubernetes with helm on Docker Desktop Windows.

Any progress on this?

[andreialecu]

Note that even without any http handlers being defined in serverless.yml, the error still appears in serverless info:

endpoints (api-gw):
**failed to fetch routes**

  Open Whisk Error ---------------------------------------

  GET https://localhost:31001/api/v1/web/whisk.system/apimgmt/getApi.http Returned HTTP 502 (Bad Gateway) --> "Unable to obtain Tenant from the API Gateway: "Invalid URI \"undefined/tenants\"""

[bin-wang]

@jthomas I think this issue should be reopened since the problem is not solved with self-hosted OpenWhisk.

[jthomas]

@Jerry-BinWang That makes sense. I'm happy to re-open the issue but don't have time to look into this at the moment.

If other people have ideas as to what is going wrong or where the issue is, please update the issue.

[nrdobie]

It seems like you have to provide at least some token now.

After running debug on on the wsk CLI tool to list API GW function I noticed the URL being called had a dummy token being sent even though I never created one or had one in my env. After setting APIGW_ACCESS_TOKEN to DUMMY_TOKEN it works.

REQUEST:
[GET]   https://192.168.86.96:31001/api/v1/web/whisk.system/apimgmt/getApi.http?accesstoken=DUMMY+TOKEN&limit=30&skip=0&spaceguid=23bc46b1-71f6-4ed5-8c54-816aa4f8c502
Req Headers
{
  "Authorization": [
    "Basic MjNiYzQ2YjEtNzFmNi00ZWQ1LThjNTQtODE2YWE0ZjhjNTAyOjEyM3pPM3haQ0xyTU42djJCS0sxZFhZRnBYbFBrY2NPRnFtMTJDZEFzTWdSVTRWck5aOWx5R1ZDR3VNREdJd1A="
  ],
  "User-Agent": [
    "OpenWhisk-CLI/1.0 (2019-03-25T16:21:36.596+0000) windows amd64"
  ]
}
RESPONSE:Got response with code 200
Resp Headers
{
  "Access-Control-Allow-Headers": [
    "Authorization, Origin, X-Requested-With, Content-Type, Accept, User-Agent"
  ],
  "Access-Control-Allow-Methods": [
    "OPTIONS, GET, DELETE, POST, PUT, HEAD, PATCH"
  ],
  "Access-Control-Allow-Origin": [
    "*"
  ],
  "Connection": [
    "keep-alive"
  ],
  "Content-Length": [
    "16"
  ],
  "Content-Type": [
    "application/json"
  ],
  "Date": [
    "Sun, 05 May 2019 01:19:04 GMT"
  ],
  "Server": [
    "nginx/1.15.12"
  ],
  "X-Openwhisk-Activation-Id": [
    "9b5af671ba1e487e9af671ba1ee87edb"
  ],
  "X-Request-Id": [
    "R9JK3TNaPicFRyaNTD6cZuavvWhk9DpP"
  ]
}
Response body size is 16 bytes
Response body received:
{
  "apis": []
}

[rakeshyoga]

Exporting an environment variable OW_APIGW_ACCESS_TOKEN with the value of APIGW_ACCESS_TOKEN found in .wskprops property file fixes the problem.

[b-n]

Funnily enough even just adding the following to .wskprops works: APIGW_ACCESS_TOKEN=<anything except nothing>

[salmanprk]

Quick questions, what is the home directory for .wskprops in Windows 10?