METAFLOW_SERVICE_URL always return "Missing Authentication Token" when accessed in browser

[jaysonpryde]

I’m currently experimenting on Metaflow. I followed the documentation and was able to deploy an aws setup with the given cloud formation template. My question is why is that I’m always getting a:

              message: "Missing Authentication Token"

when I access METAFLOW_SERVICE_URL in the browser, even if I made sure that the APIBasicAuth was set to false during the creation of cfn? Shouldn’t this setting make the metadata service accessible without the authentication/api key? How can I resolve this? Or is this expected? That is, I cannot really view the metadata/metaflow service url via browser? Thanks in advance

[ferras]

you still need to set the x-api-key header if you are trying to access the service url via the browser. To get the api-key you can go to the aws console Api Gateway -> Api Keys -> show api key

Alternatively you can use the metaflow client in the sagemaker notebook which should be automatically setup for you via the template.

[ferras]

also worth mentioning that there are two sets of endpoints. The one provided by the api gateway (which you seem to be hitting) and the one provided by the service itself. The api gateway forwards the requests the the service endpoints but needs the x-api-key to be set in the header. You can probably try hitting the service endpoints directly since you disabled auth.

[jaysonpryde]

@ferras thanks a lot. how can I access the service endpoints? I meant are they accessible via browser? where can I find them? as for the x-api-key to be specified, on my first setup, I actually enabled the auth. I used postman to specify the x-api-key in header, but I got this return:

"message": "No method found matching route / for http method GET."

So i'm not sure why. all i want is for me to see my flows, runs. steps, tasks in the browser, which i'm expecting the metaflow service URL is to provide.

thanks again

[ferras]

Is there a reason you don't want to use the python client that was provided to access this data? It will make your life a lot simpler.

What is the endpoint path that you are hitting?

The paths will vary slightly depending on whether you hit them from the api-gateway or directly via the service endpoints.

The hostname for both should be available via the aws console and should be exported via the METAFLOW_SERVICE_URL env variable

[jaysonpryde]

Is there a reason you don't want to use the python client that was provided to access this data? It will make your life a lot simpler.

I don't want to use the python client because I want to access a Web UI where in I can view past flows, runs, and tasks easily, instead of coding them, which I'm expecting to see when I access the METAFLOW_SERVICE_URL. Like the ones available in airflow or in python celery. Is my expectation correct?

What is the endpoint path that you are hitting?

Snapshot above contains the METAFLOW_SERVICE_URL path outputted in the cloud formation setup. So again, when I access this path in Chrome, I'm getting the message: "Missing Authentication Token".

The api gateway forwards the requests the the service endpoints but needs the x-api-key to be set in the header. You can probably try hitting the service endpoints directly since you disabled auth

Regarding this, I assumed that the api gateway is equivalent to the METAFLOW_SERVICE_URL. Is this correct? Or is the METAFLOW_SERVICE_URL the service endpoint you?

[jaysonpryde]

by the way @ferras , you also developed the metaflow metadata service. Is there a way to utilize this instead? It says there to set the following variables:

• MF_METADATA_DB_HOST [defaults to localhost]
• MF_METADATA_DB_PORT [defaults to 5432]
• MF_METADATA_DB_USER [defaults to postgres]
• MF_METADATA_DB_PSWD [defaults to postgres]
• MF_METADATA_DB_NAME [defaults to postgres]

But I just don't know how to utilize the RDS created in the CloudFormation setup. I am not even sure if I can utilize it.

I'll also open this topic as an issue in your actual project link.

[jaysonpryde]

@ferras I double checked via aws console that the RDS created during the Cloud Formation setup is not accessible to public:

So I guess, I cannot use this RDS.

[jaysonpryde]

@ferras and @savingoyal we may now close this issue. apparently, I had to create a stage where the api will be deployed. Thanks for your help

[ferras]

@jaysonpryde glad it was resolved. Yes the RDS was intentionally made inaccessible to the public.

The cloudformation template actually deploys the metaflow metadata service you mentioned.

Ill close the issue for now but the cloud formation template should have already had a stage that deploys and makes the api accessible