Keeps redirecting to http instead of https

[dimm0]

Describe the bug The pages keep redirecting me to http://my_url:443/... I found this: https://stackoverflow.com/questions/14810795/flask-url-for-generating-http-url-instead-of-https

Am I on the right path?

I have traefik ingress serving the requests to pods, and deployed your helm chart.

Affected Component(s)

• [ ] Back-End (Database, Microservices, Containers, etc)
• [X] Front-End (Flask, API, etc)
• [ ] Docs
• [ ] Build System

Expected behavior Return https:// scheme in redirects

[vkotronis]

@dimm0 Is this related to the kubernetes deployment? When using docker-compose for "native" ARTEMIS I cannot replicate this bug (redirection works fine, as configured in nginx). If it is only in k8s let us know to narrow down the search.

Have you made any custom modifications to nginx.conf yourself?

[dimm0]

Yes, this is in kubernetes No changes in nginx.conf. I tried removing the non-ssl path after, with the same result.

[dimm0]

Again, I have another proxy (traefik ingress) which does SSL termination, and proxies the requests to nginx

[vkotronis]

ok, will coordinate with @pgigis to make the redirection from within flask according to the fix you mention in the initial bug description.

[alistairking]

@vkotronis is there anything i can do to help with this? I have access to the K8 cluster that @dimm0 deployed artemis on, so might be able to help with debugging/testing.

[vkotronis]

@vkotronis is there anything i can do to help with this? I have access to the K8 cluster that @dimm0 deployed artemis on, so might be able to help with debugging/testing.

@alistairking we will try to first replicate it locally (you can also check https://github.com/FORTH-ICS-INSPIRE/artemis/wiki/Kubernetes-Deployment , we use the microk8s and helm tools). @pgigis are there any updates with this? Would the solution be easy to apply in our frontend container, or does it require changes also in the k8s management interface (or configuration)?

[vkotronis]

btw besides changing flask internally, which I do not know if it is a good idea (nginx should handle the redirection), there is this solution: https://medium.com/store2be-tech/kubernetes-redirect-http-to-https-with-elb-and-the-nginx-ingress-controller-911c3f4f605

@dimm0 @alistairking please also take a look at this and in the meanwhile we will check if we should provide also some extra configuration for this (the approach proposed in the article should probably work). It also mentions the traefik ingress problem (no auto-redirection of http).

[dimm0]

I the URL I provided in the first message they mention the url_for function, which returns the full URL for redirection - it seems like the problem is there, not in nginx. Our k8s ingress is working fine for multiple projects in the cluster, and artemis is the only one having issues. Also it's traefik, not nginx.

[vkotronis]

@dimm0 thanks for this clarification, it helps to attack the problem correctly; we will focus on the flask part then. With Petros we will update the comments here after testing (I could not test on my Linux laptop due to resource limitations with microk8s; we will try on MAC and a stronger server to deploy the pods).

[dimm0]

If you could build a test container for us, we could test it here and let you know whether it worked. I just wasn't sure what exactly I should build..

[vkotronis]

yes, this will be done when the pull request (PR) is ready to check. It will be accompanied with a PR-specific helm package which can be installed and tested before we merge into master.

[slowr]

@dimm0 the problem that you are facing is because you don't actually need the nginx deployment and service that we provide for SSL termination and some endpoint rewriting. Instead, you need to define an ingress resource for artemis that will mimic this nginx configuration. So the steps are:

• Remove nginx-deployment.yaml file
• Remove from artemis-chart/values.yaml these lines:

  nginx:
  ports:
  - 80
  - 443

  and provide the new yaml file when creating the artemis resources with helm pointing to this file and your secrets file:

  helm install -f other/example_values.yaml -f new_yaml_without_nginx.yaml artemis-chart

• Implement an ingress that redirects for /api/graphql to service graphql port 8080 endpoint /v1alpha1/graphql (with websocket support) and for / (anything else) to service frontend port 8000. Pretty much mimc the nginx configuration for an ingress. Can you work on that and give us feedback? I had this working for the NGINX Ingress Controller but not for the one you are using.

[slowr]

ingress for nginx controller example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: artemis-ingress
  annotations:
    nginx.org/websocket-services: "graphql"
    nginx.org/rewrites: "serviceName=graphql rewrite=/v1alpha1/graphql"
spec:
  rules:
  - host: artemis.com
    http:
      paths:
      - path: /
        backend:
          serviceName: frontend
          servicePort: 8000
      - path: /api/graphql
        backend:
          serviceName: graphql
          servicePort: 8080

[dimm0]

I did that, and portal is working properly now. How can I check if websockets are working? No errors in web console Still no modules in System Status

[slowr]

If you don't see any module then the websocket connection is not correctly established. You can debug it in inspect mode of your browser to see what happens to the query on the /api/graphql endpoint (or filtering wss protocol)

[dimm0]

I'm seeing tons of:

{"timestamp":"2019-07-21T00:29:17.461+0000","level":"error","type":"event-trigger","detail":{"internal":{"statement":"\n      UPDATE hdb_catalog.event_log\n      SET locked = 't'\n      WHERE id IN ( SELECT l.id\n                    FROM hdb_catalog.event_log l\n                    JOIN hdb_catalog.event_triggers e\n                    ON (l.trigger_name = e.name)\n                    WHERE l.delivered ='f' and l.error = 'f' and l.locked = 'f'\n                          and (l.next_retry_at is NULL or l.next_retry_at <= now())\n                    FOR UPDATE SKIP LOCKED\n                    LIMIT 100 )\n      RETURNING id, schema_name, table_name, trigger_name, payload::json, tries, created_at\n      ","prepared":true,"error":{"exec_status":"FatalError","hint":null,"message":"relation \"hdb_catalog.event_log\" does not exist","status_code":"42P01","description":null},"arguments":[]},"path":"$","error":"postgres query error","code":"unexpected"}}

errors in the graphql pod

[slowr]

This is an error from the graphql engine (hasura). Some searching suggests that there might be a versioning error of some sort. Could you try to run v1.0.0alpha45 version of it instead of v1.0.0-beta.3 just to be sure this is not the issue?

You can change it here https://github.com/FORTH-ICS-INSPIRE/artemis/blob/master/artemis-chart/templates/graphql-deployment.yaml#L48 and then patching the already running deployment you have or restarting whole helm chart

[vkotronis]

@dimm0 let us know please if a clean install (with removal of the db) of the latest ARTEMIS version worked for you. Remember to re-check the nginx fix that @slowr proposed. The graphql error you see might have been due to a problematic version of the graphql engine, so let's rule this out first (and see if the problem persists). Feel free to comment your results. Cheers!

[dimm0]

I'll try to get to this today, otherwise I'll be on vacation for 2 weeks, and will try after

[dimm0]

I'm running the hasura/graphql-engine:v1.0.0-alpha45 already. Did you want me to run the v1.0.0-beta.3?

[dimm0]

Looking better with beta, now getting in logs:

{"timestamp":"2019-07-25T23:13:19.391+0000","level":"info","type":"websocket-log","detail":{"i_user_vars":null,"i_event":{"type":"closed"},"i_connection_info":{"websocket_id":"533bcddb-2108-40b2-aeee-9cffed09fd9b","jwt_expiry":null,"msg":null}}}
{"timestamp":"2019-07-25T23:13:31.809+0000","level":"info","type":"websocket-log","detail":{"i_user_vars":null,"i_event":{"type":"accepted"},"i_connection_info":{"websocket_id":"d2bcadeb-e3a6-466d-b618-af4ac9847079","jwt_expiry":null,"msg":null}}}
{"timestamp":"2019-07-25T23:13:31.809+0000","level":"error","type":"websocket-log","detail":{"i_user_vars":null,"i_event":{"type":"conn_err","detail":"Could not verify JWT: JWSError JWSInvalidSignature"},"i_connection_info":{"websocket_id":"d2bcadeb-e3a6-466d-b618-af4ac9847079","jwt_expiry":null,"msg":null}}}

[vkotronis]

So does it work now? I did not get if you get proper data now (e.g., you see the modules, BGP updates incoming, etc.)

[dimm0]

Nope, no modules

[slowr]

@dimm0 you are getting a JWT failure. Can you see if /jwt/auth endpoint returns the token correctly? This endpoint is on frontend served by flask.

[vkotronis]

Since we will receive feedback on this after around ~2 weeks, moving to release afterwards (of course the issue will stay open until also the /jwt/auth problem is checked and resolved in cooperation with @dimm0).

[slowr]

closing this issue as it seems that the problem was the ingress resourse. if the issue continue feel free to re-open