Open yossisht9876 opened 2 years ago
Same thing for me. I have a similar setup where I use ingress to expose Nifi Web UI and it shows exactly the same error you have.
I even tried kubectl port-forward service/nifi 8443:8443
and navigated to https://localhost:8443 only to get the following error:
An error occurred during a connection to localhost:8443. PR_END_OF_FILE_ERROR
It used to work when we had HTTP endpoints though.
right i have the same errors , do we have the option to add white listed hosts to nifi? i saw users that use NIFI_WEB_PROXY_HOST in order to white list hosts to nifi but it not working for me , i can add it to the pods via env vars but its not adding the hosts to the white list.
any other ideas ? thanks
Does this comment help?
https://github.com/cetic/helm-nifi/pull/169#discussion_r716947725
Same here, installing the chart on Openshift, getting same error page.
System Error The request contained an invalid host header [nifi-external-OPENSHIFT-ROUTE-URL] in the request [/]. Check for request manipulation or third-party intercept. Valid host headers are [empty] or: 127.0.0.1 127.0.0.1:8443 localhost localhost:8443 ::1
nifi-1.nifi-headless.nifikop.svc.cluster.local nifi-1.nifi-headless.nifikop.svc.cluster.local:8443 10.129.3.209 10.129.3.209:8443 0.0.0.0 0.0.0.0:8443
Does this comment help?
After adding the following, I was able to get the UI working with kubectl port-forward
command. However, when I try to expose it via an ingress controller, I still get the same error. Any help, folks?
the web UI works for me after adding this:
nginx.ingress.kubernetes.io/upstream-vhost: "localhost:8443" nginx.ingress.kubernetes.io/proxy-redirect-from: "https://localhost:8443" nginx.ingress.kubernetes.io/proxy-redirect-to: "https://nifi-domain.com"
to the ingress configuration.
the web UI works for me after adding this:
nginx.ingress.kubernetes.io/upstream-vhost: "localhost:8443" nginx.ingress.kubernetes.io/proxy-redirect-from: "https://localhost:8443" nginx.ingress.kubernetes.io/proxy-redirect-to: "https://nifi-domain.com"
to the ingress configuration.
Can you output the Nifi ingress rule in YAML here?:
kubectl get ingress <ingress-name> -n <namespace-name> -o yaml
Note: You may hide the Nifi URL if it's confidential.
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx meta.helm.sh/release-name: nifi-helm meta.helm.sh/release-namespace: nifi-helm nginx.ingress.kubernetes.io/affinity: cookie nginx.ingress.kubernetes.io/backend-protocol: HTTPS nginx.ingress.kubernetes.io/configuration-snippet: | proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-Port 443; proxy_set_header X-ProxyHost https://nifi-xxxx.com; nginx.ingress.kubernetes.io/proxy-redirect-from: https://localhost:8443 nginx.ingress.kubernetes.io/proxy-redirect-to: https://nifi-xxxx.co nginx.ingress.kubernetes.io/ssl-redirect: "true" nginx.ingress.kubernetes.io/upstream-vhost: localhost:8443 creationTimestamp: "2021-11-22T09:54:42Z" generation: 1 labels: app: nifi app.kubernetes.io/managed-by: Helm chart: nifi-1.0.2 heritage: Helm release: nifi-helm managedFields:
the ui works now but every time that I try to click on something on the UI, it send me away to the login page with the error:
Unable to communicate with NiFi
Please ensure the application is running and check the logs for any errors.```
its happen when trying to configure a new processor just "playing" with the menu options.
the pods are up and running and there are no errors in the app log or on any other pod's logs
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx kubernetes.io/tls-acme: "true" meta.helm.sh/release-name: nifi-helm meta.helm.sh/release-namespace: nifi-helm nginx.ingress.kubernetes.io/affinity: cookie nginx.ingress.kubernetes.io/backend-protocol: HTTPS nginx.ingress.kubernetes.io/configuration-snippet: | proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-Port 443; proxy_set_header Origin https://nifi.example.com; proxy_set_header Referrer https://nifi.example.com; nginx.ingress.kubernetes.io/proxy-redirect-from: https://localhost:8443 nginx.ingress.kubernetes.io/proxy-redirect-to: https://nifi.example.com nginx.ingress.kubernetes.io/upstream-vhost: localhost:8443 creationTimestamp: "2021-11-16T09:47:20Z" generation: 3 labels: app: nifi app.kubernetes.io/managed-by: Helm chart: nifi-1.0.1 heritage: Helm release: nifi-helm managedFields:
* apiVersion: networking.k8s.io/v1beta1 fieldsType: FieldsV1 fieldsV1: f:status: f:loadBalancer: f:ingress: {} manager: nginx-ingress-controller operation: Update time: "2021-11-16T09:48:06Z" * apiVersion: extensions/v1beta1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:kubernetes.io/ingress.class: {} f:kubernetes.io/tls-acme: {} f:meta.helm.sh/release-name: {} f:meta.helm.sh/release-namespace: {} f:nginx.ingress.kubernetes.io/affinity: {} f:nginx.ingress.kubernetes.io/backend-protocol: {} f:nginx.ingress.kubernetes.io/configuration-snippet: {} f:nginx.ingress.kubernetes.io/proxy-redirect-from: {} f:nginx.ingress.kubernetes.io/proxy-redirect-to: {} f:nginx.ingress.kubernetes.io/upstream-vhost: {} f:labels: .: {} f:app: {} f:app.kubernetes.io/managed-by: {} f💹 {} f:heritage: {} f:release: {} f:spec: f:rules: {} manager: Go-http-client operation: Update time: "2021-11-17T13:49:24Z" name: nifi-helm-ingress namespace: nifi-helm resourceVersion: "44153253" uid: gdgdgd-dkdkmd-ddmdxxxxx spec: rules: * host: nifi.example.com http: paths: * backend: service: name: nifi-helm port: number: 8443 path: / pathType: ImplementationSpecific status: loadBalancer: ingress: * hostname: internal-xxxxxxxxxxxxxxxx.xxxxxx.xxxxxx2.us-east-1.elb.amazonaws.com
@yossisht9876 this workaround first actually routes to https://nifi.example.com/nifi.example.com and then automatically redirects to https://nifi.example.com/nifi. I don't think this is the right workaround.
Can some look at this issue on priority?
Hi @banzo, I am installing the Nifi cluster using the latest release as a LoadBalancer service (by changing the type as LoadBalancer in values.yml) , but I am getting the same error as above when trying to access the UI. Can you please help what need to be done. I have also added the properties as mentioned in the comments in safetyValve properties.
Is it possible to run as load balancer and is http still supported?
Also I am not sure what need to be set in webproxyhost. Any advise to make this working is appreciated. Thanks.
@leshibily i edit my ingress output - please try if it works for you
@leshibily i edit my ingress output - please try if it works for you
That did not work as well. I got the error.
Unable to validate the access token.
Hi, I was facing the same errors and couldn't tell if they were on the helm chart or at NiFi.
I've deployed a simple nifi using the base docker image from dockerhub, https://hub.docker.com/r/apache/nifi/ .
Then, I've setup a minimal ingress for testing and faced the first problem:
System Error
The request contained an invalid host header [nifixxx.xxx.xx.co] in the request [/]. Check for request manipulation or third-party intercept.
Valid host headers are [empty] or:
127.0.0.1
127.0.0.1:8443
localhost
localhost:8443
[::1]
[::1]:8443
nifi.xxx.xxx..xx
nifi.xxx.xxx..xx:8443
0.0.0.0
0.0.0.0:8443
I fixed this first problem, with the following nginx.ingress.kubernetes.io/backend-protocol
annotation:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nifi-ingress-service-internal
namespace: nifi-test
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: nifi.xxx.xx
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nifi-cluster-ip-service
port:
number: 8443
This happens, because the NiFi is running with HTTPS inside the cluster, so, the reverse proxy must be aware of this, and this annotation tell him this. After this setup, I was able to load the web UI and login normally.
After login, I faced the second problem:
Whenever I click on something on the UI I was redirected to a page with the following message:
Unable to communicate with NiFi
Please ensure the application is running and check the logs for any errors.
With some research, I found how to fix this in the oficial NiFi documentation for systems administrators: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration
So, I updated the ingress definition to the following:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nifi-ingress-service-internal
namespace: nifi
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/upstream-vhost: "localhost:8443"
nginx.ingress.kubernetes.io/proxy-redirect-from: "https://localhost:8443"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header 'X-ProxyScheme' 'https';
proxy_set_header 'X-ProxyPort' '443';
spec:
rules:
- host: nifi.xxx.xx
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nifi-cluster-ip-service
port:
number: 8443
I hope this can help! M
@murilolobato hi, did it fix the ```Unable to communicate with NiFi Please ensure the application is running and check the logs for any errors.
because i got the same issue
Hi, I was facing the same errors and couldn't tell if they were on the helm chart or at NiFi.
I've deployed a simple nifi using the base docker image from dockerhub, https://hub.docker.com/r/apache/nifi/ .
Then, I've setup a minimal ingress for testing and faced the first problem:
System Error The request contained an invalid host header [nifixxx.xxx.xx.co] in the request [/]. Check for request manipulation or third-party intercept. Valid host headers are [empty] or: 127.0.0.1 127.0.0.1:8443 localhost localhost:8443 [::1] [::1]:8443 nifi.xxx.xxx..xx nifi.xxx.xxx..xx:8443 0.0.0.0 0.0.0.0:8443
I fixed this first problem, with the following
nginx.ingress.kubernetes.io/backend-protocol
annotation:apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nifi-ingress-service-internal namespace: nifi-test annotations: kubernetes.io/ingress.class: 'nginx' nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" spec: rules: - host: nifi.xxx.xx http: paths: - path: / pathType: Prefix backend: service: name: nifi-cluster-ip-service port: number: 8443
This happens, because the NiFi is running with HTTPS inside the cluster, so, the reverse proxy must be aware of this, and this annotation tell him this. After this setup, I was able to load the web UI and login normally.
After login, I faced the second problem:
Whenever I click on something on the UI I was redirected to a page with the following message:
Unable to communicate with NiFi Please ensure the application is running and check the logs for any errors.
With some research, I found how to fix this in the oficial NiFi documentation for systems administrators: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#proxy_configuration
So, I updated the ingress definition to the following:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: nifi-ingress-service-internal namespace: nifi annotations: kubernetes.io/ingress.class: 'nginx' nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" nginx.ingress.kubernetes.io/upstream-vhost: "localhost:8443" nginx.ingress.kubernetes.io/proxy-redirect-from: "https://localhost:8443" nginx.ingress.kubernetes.io/configuration-snippet: | proxy_set_header 'X-ProxyScheme' 'https'; proxy_set_header 'X-ProxyPort' '443'; spec: rules: - host: nifi.xxx.xx http: paths: - path: / pathType: Prefix backend: service: name: nifi-cluster-ip-service port: number: 8443
I hope this can help! M
@murilolobato the nifi URL is redirecting to https://localhost:8443. Do you have any idea? Any help would be appreciated.
Hi @leshibily ,
I think you are facing the second problem I mentioned. You should check the annotations section of your ingress definition, and ensure you set-up the correct settings according to my example and most important, according to the NiFi System Administrators guide.
If you have already set the same annotations, ensure that the ingress controller you are using does support the annotations. In my example, I'm using the https://kubernetes.github.io/ingress-nginx/ controller, and the example annotations I have provided are compatible with it.
M
Hi @leshibily ,
I think you are facing the second problem I mentioned. You should check the annotations section of your ingress definition, and ensure you set-up the correct settings according to my example and most important, according to the NiFi System Administrators guide.
If you have already set the same annotations, ensure that the ingress controller you are using does support the annotations. In my example, I'm using the https://kubernetes.github.io/ingress-nginx/ controller, and the example annotations I have provided are compatible with it.
M
Please find my nifi ingress rule configuration below.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/configuration-snippet: |-
proxy_set_header X-ProxyScheme 'https';
proxy_set_header X-ProxyPort '443';
nginx.ingress.kubernetes.io/proxy-redirect-from: https://localhost:8443
nginx.ingress.kubernetes.io/upstream-vhost: localhost:8443
name: nifi
namespace: nifi
spec:
rules:
- host: nifi-0099.example.com
http:
paths:
- backend:
service:
name: nifi
port:
number: 8443
path: /
pathType: Prefix
The login page loads but once I login to Nifi, it redirects to https://localhost:8443. Did you try logging in? And the ingress controller I use is ingress-nginx (https://kubernetes.github.io/ingress-nginx)
@leshibily hi, were you able to fix this?
The main problem is that Nifi did not support the NIFI_WEB_PROXY_HOST (webProxyHost in values.yaml file) environement variable in 1.14.0 version.
Could you please try by using this pull request: #206 .
The ingress has also been updated.
@zakaria2905 I tried adding the NIFI_WEB_PROXY_HOST using the following but I am still getting the invalid host header
error in the values.yaml
env:
- name: NIFI_WEB_PROXY_HOST
value: "nifi.test.example.com"
error:
The request contained an invalid host header [nifi.test.example.com] in the request [/nifi]. Check for request manipulation or third-party intercept.
@leshibily , after pulling PR #206 , I only modify the following lines in values.yaml
file:
webProxyHost: nifi.test.local
---
ingress:
enabled: true
hosts:
- nifi.test.local
path: /
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
In addition, I enable the minikube ingress addon minikube addons enable ingress
I also set my /etc/hosts
file by adding the minikube IP address and the domain name (nifi.test.local)
And it works
HI ALL, I am installing the Nifi cluster as a LoadBalancer service but getting below error.
Any update on the issue.
System Error The request contained an invalid host header [IP:8443] in the request [/nifi]. Check for request manipulation or third-party intercept. Valid host headers are [empty] or: 127.0.0.1 127.0.0.1:8443 localhost localhost:8443 ::1
nifilb-0.nifilb-headless.namespace.svc.cluster.local nifilb-0.nifilb-headless.namespace.svc.cluster.local:8443 10.7.1.113 10.7.1.113:8443 0.0.0.0 0.0.0.0:8443
HI All, There is workaround for this issue. Once yaml deployed, you have to edit statefulset and add env value
@arunbabumm NIFI_WEB_PROXY_HOST is ignored in 1.14.0 , what we did instead is to change it directly in the properties section and we added also some annotation in ingress, the final values.yaml will be:
...
properties:
webProxyHost: xxx.net
....
ingress:
enabled: true
annotations:
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
...
k8s version: v1.20.13 chart version: 1.0.5 nifi version: 1.14.0
Does this comment help?
no not working.
This ticket helped me lot to solved.
Here you go my working igress
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header 'X-ProxyScheme' 'https';
proxy_set_header 'X-ProxyPort' '443';
hosts:
- host: "*.dmd.so"
paths:
- path: /()(.*)
backend:
serviceName: nifi
servicePort: 8443
key Observation Kindly note once you deploy NiFI give it 3-5 minutes to come up completely. Initially, I was trying immediately after deploying but it was not working then I was trying another setting where I was doing a mistake.
I got this to work with AWS Load Balancer Controller
and default nifi docker image from Nifi's Dockerhub. In order to stop Nifi complaining about the hostname header, I had to add a routable domain name in Route53 and set it as nifi.web.proxy.host
in NiFi properties via env variable.
nifi-deployment.yml looks like this:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nifi-deployment
namespace: nifi
labels:
app: my-nifi
type: nifi
spec:
replicas: 1
selector:
matchLabels:
app: my-nifi
template:
metadata:
labels:
app: my-nifi
spec:
containers:
- name: my-nifi
image: apache/nifi:latest
env:
- name: NIFI_WEB_PROXY_HOST
value: "nifi.mydomain.com"
- name: NIFI_WEB_HTTPS_PORT
value: "8443"
Nodeport service yaml:
apiVersion: v1
kind: Service
metadata:
name: my-nifi-nodeport
namespace: nifi
labels:
app: my-nifi
spec:
type: NodePort
ports:
- port: 8443
targetPort: 8443
nodePort: 30443
protocol: TCP
name: https
selector:
app: my-nifi
ingress service YAML:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-nifi-ingress
namespace: nifi
labels:
app: my-nifi
annotations:
alb.ingress.kubernetes.io/backend-protocol: HTTPS
alb.ingress.kubernetes.io/certificate-arn: ${enter your cert ARN here}
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-group-attributes: stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=60
alb.ingress.kubernetes.io/target-type: instance
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-nifi-nodeport
port:
number: 8443
I solved the problem by adding the following annotations:
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/cors-allow-headers: 'Content-Type, Authorization'
nginx.ingress.kubernetes.io/cors-allow-methods: 'PUT, GET, POST, OPTIONS'
nginx.ingress.kubernetes.io/cors-allow-origin: 'https://your-nifi-host:443'
nginx.ingress.kubernetes.io/enable-cors: 'true'
nginx.ingress.kubernetes.io/session-cookie-name: nifi-session-cookie
nginx.ingress.kubernetes.io/ssl-passthrough: 'true'
nginx.ingress.kubernetes.io/upstream-vhost: 'your-nifi-host:443'
I have this issue with nifi UI which just actions through PUT methods (like changing a processor properties) facing error "Unable to communicate with NiFi". The actual reason is "Blocked loading mixed active content" because for just PUT methods instead of HTTPS schema the action calls HTTP schema of the domain.
This is my ingress config:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nifi
namespace: nifi
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header 'X-ProxyScheme' 'https';
proxy_set_header 'X-ProxyPort' '443';
add_header 'Content-Security-Policy' 'upgrade-insecure-requests';
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/upstream-vhost: "localhost:8443"
# nginx.ingress.kubernetes.io/proxy-redirect-from: "https://localhost:8443"
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "hello-cookie"
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
ingressClassName: nginx
rules:
- host: "nifi.xxxxxxxx.xxx"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: nifi
port:
number: 8443
Describe the bug trying to run nifi on eks version 1.19 all the pods are running and i can see in the logs that the server is up and running. im using NGINX with aws internal load balancer. web UI is under HTTPS so the url will be https://nifi.xxx.xx.com
Version of Helm and Kubernetes: helm 3 eks version 1.19
What happened: the web UI is not loading and i cant override the nifi.properteis file via the values.yaml file
on the web UI we get :
System Error The request contained an invalid host header [nifixxx.xxx.xx.co] in the request [/]. Check for request manipulation or third-party intercept. Valid host headers are [empty] or:
What you expected to happen: im expecting to load the web UI
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know: my value.yaml
what im missing here ? thanks