search

i2group / analyze-deployment-tooling

Develop i2 Analyze configurations by using the configuration development environment. Review reference architectures for containerised deployments of i2 Analyze.
https://i2group.github.io/analyze-deployment-tooling/
Other
1 stars 1 forks source link

Connector Designer. ERROR: Load Balancer service is NOT live #32

Closed lisandrosUnitech closed 1 week ago

lisandrosUnitech commented 2 weeks ago

Hello,

I am trying to set up Connector Designer on a Red Hat Linux server. I followed the steps from the following link: https://docs.i2group.com/analyze/4.4.4/deploy_connector_designer.html

The i2 server I am trying to associate with Connector Designer is deployed on a Windows Server and only contains i2 Connect, so I had to set the root path for Connector Designer to opaldaod instead of opal.

Here are my topology, command-access-control, user.registry, and server.extensions.xml files from my i2 Analyze server:

The problem I am encountering is that when I run the command ./scripts/deploy, I get the following error at a certain point:

" [user@msm-i2anbp-desa-db scripts]$ ./deploy ok -- validation done

----------------------------------------------------------------------

Deployment Information:

----------------------------------------------------------------------

ANALYZE_CONTAINERS_ROOT_DIR: /opt/analyze-deployment-tooling CONFIG_NAME: con-des-default DEPLOYMENT_PATTERN: i2c DB_DIALECT: postgres I2A_DEPENDENCIES_IMAGES_TAG: 4.4.4.0

----------------------------------------------------------------------

Checking Licenses Accepted

----------------------------------------------------------------------

----------------------------------------------------------------------

Stopping containers for other configs

----------------------------------------------------------------------

----------------------------------------------------------------------

Stopping connector containers for other configs

----------------------------------------------------------------------

WARN: File /opt/analyze-deployment-tooling/path-configuration.json does not contain shared configurations. Skipping.

----------------------------------------------------------------------

Restarting containers for config: con-des-default

----------------------------------------------------------------------

load_balancer.con-des-default_4.4.4

----------------------------------------------------------------------

Previous deployment did not complete - retrying

----------------------------------------------------------------------

----------------------------------------------------------------------

Clearing down configuration

----------------------------------------------------------------------

----------------------------------------------------------------------

Copying all-patterns Configuration to /opt/analyze-deployment-tooling/templates/config-development/configuration

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuring form based authentication

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuring extensions

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuring prometheus and grafana

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuring load balancer

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuring connector designer

----------------------------------------------------------------------

----------------------------------------------------------------------

Creating solr configuration

----------------------------------------------------------------------

----------------------------------------------------------------------

Configuration has been successfully created

----------------------------------------------------------------------

----------------------------------------------------------------------

Stopping connector_designer.con-des-default_4.4.4 container

----------------------------------------------------------------------

connector_designer.con-des-default_4.4.4

----------------------------------------------------------------------

Deleting connector_designer.con-des-default_4.4.4 container

----------------------------------------------------------------------

connector_designer.con-des-default_4.4.4

----------------------------------------------------------------------

Connector Designer container connector_designer.con-des-default_4.4.4 is starting

----------------------------------------------------------------------

1: Pulling from i2group/i2eng-connector-designer Digest: sha256:9bedf9e493c348fafa810e8d0f709e11f7a19c7d0f0a0ee0e24d6172c223f531 Status: Image is up to date for i2group/i2eng-connector-designer:1 1eaf9cf08f5477bc44e15127a1f279332325188ebb559e45af87c3314d998c14

----------------------------------------------------------------------

Waiting for Connector Designer to be live on https://connectordesigner.eia:3000/api/configuration/connectors

----------------------------------------------------------------------

Could not connect to https://connectordesigner.eia:3000/api/configuration/connectors Connector Designer is NOT live (attempt: 1/40). Waiting... Connector Designer is live

----------------------------------------------------------------------

Stopping load_balancer.con-des-default_4.4.4 container

----------------------------------------------------------------------

load_balancer.con-des-default_4.4.4

----------------------------------------------------------------------

Deleting load_balancer.con-des-default_4.4.4 container

----------------------------------------------------------------------

load_balancer.con-des-default_4.4.4

----------------------------------------------------------------------

Load balancer container load_balancer.con-des-default_4.4.4 is starting

----------------------------------------------------------------------

2.9: Pulling from i2group/i2eng-haproxy Digest: sha256:2fb652a258ff612194eedcbd32b03234e43ec63162582283b6ca9027b9df084b Status: Image is up to date for i2group/i2eng-haproxy:2.9 fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3

----------------------------------------------------------------------

Waiting for Load Balancer service to be live

----------------------------------------------------------------------

Load Balancer service is NOT ready to receive connections (attempt: 1/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 2/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 3/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 4/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 5/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 6/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 7/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 8/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 9/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 10/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 11/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 12/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 13/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 14/15). Waiting... Error response from daemon: container fe02bf0a51b896b2f0bb6a1e9ad6db218ed76d218120f179e8b7988c865feeb3 is not running Load Balancer service is NOT ready to receive connections (attempt: 15/15). Waiting... Enter PEM pass phrase: Enter PEM pass phrase: [NOTICE] (1) : haproxy version is 2.9.9-ad

75c48 [NOTICE] (1) : path to executable is /usr/local/sbin/haproxy [ALERT] (1) : config : parsing [/usr/local/etc/haproxy/haproxy.cfg:52] : 'bind *:9046' in section 'frontend' : No Private Key found in '/tmp/i2acerts/i2Analyze.pem.key'. [ALERT] (1) : config : Error(s) found in configuration file : /usr/local/etc/haproxy/haproxy.cfg [ALERT] (1) : config : Fatal errors found in configuration.

ERROR: Load Balancer service is NOT live

"

image image image image

What is causing this error, and how can I resolve it?

Sometimes the command also throw me this message error but when I execut it again, it shows me the first error message I described.

image

I hope I have been as clear as possible.

Kind regards.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

From the logs you are providing, it seems that the load balancer certificates are not configured correctly. Could you please check that the certificates provided in step 10 and 11 of the instructions are in PEM format and the user that executes the installation has permission to the files? (https://docs.i2group.com/analyze/4.4.4/deploy_connector_designer.html)

I suggest to delete the environment-secrets directory, copy the certificates (ensure permissions and format is correct) and rerun the installer script. This should reset it if the permissions got corrupted along the way.

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Thank you for your response.

I was able to configure correctly Connector Designer and I could execute the ./deploy command without any issue.

image

But now I have a new issue related with the i2 Analyze server.

When I execute the setup -t start command it throw me this message error:

image

I attached the ffdc_24.08.27_11.57.15.0.log file to see if you can help me with this issue.

ffdc_24.08.27_11.57.15.0.log

I can create a new issue if you prefer or we can continue through this same channel.

We really apreciate all the help and support you gave us.

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

Happy to help.

It seems to be a certificate error again. In this case it is the configuration in i2 Analyze. Can you confirm you have added the externalCA.cer to the application trust-store set in the topology.xml? See configuring Liberty for TLS for more information.

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

In the topology.xml we already had configured a liberty-keystore

image

image

We generated the CA.cer using the liberty-keystore and that file we sent to the Connector Designer Server and located on the path analyze-deployment-tooling-main/environment-secrets/generated-secrets/certificates/externalCA

I noticed in the link you send me that there is another file that we aren't using, the liberty-trustore.

Should we create and configure this file too?

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

Yes. The liberty server needs to "trust" the external certificate so it can connect to the connector designer machine. Please create another jks file, add the external certificate and set it in the topology.xml as the application truststore.

This document explains the network and security for the deployment https://docs.i2group.com/analyze/4.4.4/understand_connector_designer.html

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Sorry for the delay in answer. I've configuring the connector designer correctly and finally I did it.

When I execute the setup -t start command it doesn't show me any message error.

image

Also the Connector Designer deploy sucessfully

image

But when I'm trying to access to the Connector Designer from a browser like Google Chrome it shows me the following:

image

Why can't I access the connector designer panel? Or what could this be due to?

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

I'm glad you got the certificate issue sorted. The connector designer is hosted under /connector-designer but it seems that the load balancer cannot connect to the i2analyze server and this will prevent you to log in. Can you execute the command docker logs load_balancer.con-des-default_4.4.4.0 and send the output? And please send the contents of the .cfg file at the root of the connector designer installation.

I also noticed your browser is showing the connection as insecured. To fix this you need to install the external certificate by following the instructions in this link https://i2group.github.io/analyze-deployment-tooling/content/deploy_config_dev.html#installing-the-certificate

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

I send you the log and the .cfg file you asked me.

load_balancer.con-des-default_4.4.4.0 log.txt

cfg file.txt

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

Thanks for the files.

From the logs it seems the connector designer machine doesn't have access to the i2 Analyze machine. See the line 'server liberty/liberty1' : could not resolve address 'unitech-i2eia-win', disabling server.

Are you able to ping unitech-i2eia-win and get packages? Can you ensure the DNS is resolvable from the connector designer machine and viceversa (from i2 Analyze to connector designer) and rerun the installer?

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Yes I'm able to do a ping from the Linux Server to the i2 Analyze server.

image

Ping on Windows server image

In fact, I add the IP address and the hostname in the hosts file on each server

Linux Server (Connector Designer) image

Windows Server (I2 Analyze) image

Also I enabled connections on port 9443 on Linux Server.

Kind regards, Lisandro

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

Given your DNS is not public you will need to make another change in your deployment for connector designer to have access since it doesn't automatically inherit the hosts file.

Please add a file called docker.env at the root of your connector designer installation, with the line DOCKER_EXTRA_ARGS=--add-host unitech-i2eia-win:10.1.11.101 then rerun the install script. This will ensure the container has the correct IP to resolve the DNS.

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

I did what you told me but I still can't access the Connector Designer panel.

image

This is the docker.env file that I creat.

image

image

This is the log I got after the creation of docker.env file.

image

docker log.txt

I don't know if it is related to the error but I still wanted to tell you the following. In step 10 I generated the liberty server CA certificate using the keystore explorer tool (on my personal PC) and then moved the CA.cer file to its corresponding folder in Connector Designer. I don't know if the way I created this Liberty server certificate is the right way to do it.

Then in step 11 the server.key and server.cer certificates that I created for the server where I installed connector-designer I created them with an internal CA and then I copied those files to the /environment-secrets/generated-secrets/certificates/i2analyze and replaced the existing ones as the documentation indicated. I don't know if this is also the correct way to create these files.

I hope I have been as clear as possible.

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

I'm sorry this hasn't solved your issue.

I believe the certificate error in the log alert certificate unknown is related to the browser not having the certificate. If you can please follow the instructions in this link https://i2group.github.io/analyze-deployment-tooling/content/deploy_config_dev.html#installing-the-certificate we can confirm if that error goes away.

For the DNS, can you please run docker exec -it load_balancer.con-des-default_4.4.4.0 cat /etc/hosts to confirm if the IP map has been added correctly to the container?

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

I execute the command you told me and this is what I've got.

image

I installed the CA certificate in my personal computer but I still can't access the Connector Designer panel.

Maybe there is something wrong with the certificates that I created.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

The command shows that the fix I mentioned does work but now there is an unspecified DNS error.

From the logs it seems that 10.1.11.101 (i2 Analyze) is successfully connecting to the connector designer but the load balancer is not able to connect to the i2 Analyze server. It also shows that the IP 10.1.0.2 is using an unknown certificate. Is this your personal computer's IP?

On the certificates question, I haven't used the keystore explorer, instead we use keytool and openssl commands. The certificate in the connector designer machine needs to be signed with the same external CA that was imported into the liberty truststore.

These commands will generate the required files (you can modify key length and expiration time as required):

# Generate CA.key
openssl req -new -nodes -newkey rsa:4096 -keyout CA.key -subj "/CN=unitech-i2eia-win" -out CA.csr

# Generate CA.cer
openssl x509 -req -sha256 -extfile x509.ext -extensions ca -in CA.csr -signkey CA.key -days 365 -out CA.cer

# Generate key
openssl genrsa -out server.key 4096

# Generate certificate signing request
openssl req -new -key server.key -subj "/CN=unitech-connector-designer" -out server-key.csr

# Generate certificate
openssl x509 -req -sha256 -CA CA.cer -CAkey CA.key -days 365 -CAcreateserial -CAserial CA.csr -extfile x509.ext -extensions server -in server-key.csr -out server.cer

The content of the x509.ext file is the following:

[ ca ]
# X509 extensions for a ca
keyUsage                = critical, cRLSign, keyCertSign
basicConstraints        = CA:TRUE, pathlen:0
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid:always,issuer:always

[ server ]
# X509 extensions for a server
keyUsage                = critical,digitalSignature,keyEncipherment
extendedKeyUsage        = serverAuth, clientAuth
basicConstraints        = critical,CA:FALSE
subjectKeyIdentifier    = hash
authorityKeyIdentifier  = keyid,issuer:always
subjectAltName          = @subject_alt_names

[ subject_alt_names ]
DNS.1 = unitech-connector-designer

To import the CA.cer into your truststore, delete your current truststore file and run:

keytool -import -trustcacerts -keystore "i2-liberty-truststore.p12" -file "CA.cer" -alias ca -storetype PKCS12

I hope this is useful.

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Sorry for the delay in answer.

I was talking with members of my team about the issue we had in configurin connector designer.

We just noticed that in the Connector Designer architecture the possible clients are: i2 ANB and i2 Notebook.

image

Since I'm configurin an i2 Analyze opaldaod server, this kind of deployment doesn't have i2 Notebook so I'm not sure If I will we able to configure correctly Connector Designer in this server.

Can you confirm if this is possible?

I hope I have been clear as possible.

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

The clients are optional. You can use one/both/either.

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Thank you for your response.

I still can't connect to the Connector Designer panel but this time I did the following.

I re install Connector Designer but this time I assigned this name: unitech-i2eia-win to the FQDN.

From the keystore explorer tool, using the Liberty keystore that I already had configured on the i2 Analyze server, I exported the CA.cer and CA.key files and took them to the server where Connector Designer is deployed.

Then I generated the server.key and server.csr files. I generated the server.cer file signed by CA.cer. I took the server.cer and server.key files to their corresponding folder. I also took the CA.cer file to its corresponding folder. Do I also have to take the CA.key file to the same folder as the CA.cer file?

I then made the other configurations and ran the ./deploy command in Connector Designer and it deployed successfully. When reviewing the logs this is what I get.

image

From i2Analyze I am getting the following error when starting the application.

image

So what I did to summarize was use the same CA that I already had on the Liberty server to sign the certificates for the Connector Designer.

Why docker is changing the IP address?

I hope I have been as clear as possible.

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

I'm confused why you've changed the FQDN. It seems that this change, possibly influenced by the docker.env file, is causing a conflict with the certificate configuration.

Please verify the following:

If you've already tried these steps and are still encountering issues, please consider raising a support ticket so our team can provide more tailored assistance. We'll be able to delve deeper into your specific configuration and environment to identify the root cause.

Regards, Ariana

lisandrosUnitech commented 2 weeks ago

Hi Ariana,

Thank you for your answer.

We decided to change the FQDN to the common name of the Liberty certificate because the official i2 documentation indicates that.

image

On the other hand, reading the last thing you told us, we wanted to clarify the following:

The common name of the Liberty certificate must match the FQDN of the Connector Designer and in turn this must match the Common Name of the server.cer and server.key files?

For example:

Liberty Certificate Common Name: unitech-i2eia-win

FQDN Connector Designer: unitech-i2eia-win

Common Name server.cer and server.key: unitech-i2eia-win

Are we right?

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 2 weeks ago

Hi Lisandro,

I understand how the instructions could be misleading.

As per the architecture diagram, we recommend the clients to be configured to access the application through the load balancer instead of directly through liberty.

You must have a Certificate Authority (CA.cer + CA.key) which should be used to sign both the liberty server certificates and the connector designer certificates (see my previous message for instructions on how to create these files).

The common name of the liberty server should not be the same as the connector designer if the are deployed in different machines. The common names should match the host names of each machine.

I hope this answers your question.

Regards, Ariana

lisandrosUnitech commented 1 week ago

Hi Ariana,

I changed the hostname of the Connector Designer to the hostname of the server (msm-i2anbp-desa-db.unitech.com.ar). Then I was able to configure the certificate for Connector Designer with the CA of Liberty Server and use the ext file x509 to configure the subject alt name with DNS.1= msm-i2anbp-desa-db.unitech.com.ar. For this certificates I didn't use the command you tell me before, instead I use the CA of the Liberty server.

After that, I installed the server.cer certificate on my computer and I tried to access the Connecto Designer panel but I still can't do it. The only difference is that I can access through a secure connection.

image

This are the logs docker logs.txt

I don't know what could be the problem this time, but I think we were able to make some progress.

Also I configured the docker.env file.

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 1 week ago

Hi Lisandro,

Without knowing more details about how the Liberty certificate was created I would guess this is a self-sign certificate for a server and not a CA. The system requires a CA to work which can be created with the commands I've provided or you can use a trusted CA. This should then be used to sign the server certificates for both machines.

Please raise a support ticket to better assist you on setting up the certificates for your deployment.

Regards, Ariana

lisandrosUnitech commented 1 week ago

Hi Ariana,

Good news, we were able to configure Connector Designer!!

image

Thank you for all your help and support you gave us during all these days. We really apreciate it.

Last I want to ask you if there is any documentation, besides the official one, of how can we use, configure, create a service, etc in Connector Designer?

Kind regards, Lisandro.

Ariana-Hlavaty-i2 commented 1 week ago

Hi Lisandro,

Glad that it is all sorted and you are able to use Connector Designer.

For completion, do you mind explaining here what the underlying issue was and how you solved it?

For Connector Designer enablement, we only have the i2 docs at the moment but the team is working on releasing some videos soon.

Regards, Ariana

lisandrosUnitech commented 1 week ago

Hi Ariana,

Yes of course, here is what we do.

The issue I was having was that the load balancer (deployed on Linux) couldn't connect to i2 Analyze (Windows). So, what we did was deploy i2 Analyze v4.4.4 with i2 Connect on the same server where the load balancer and Connector Designer are located. We created a keystore (using Keystore Explorer) to secure Liberty. From that same keystore, we extracted the Liberty certificate and private key to generate and sign the certificates for Connector Designer. Both Liberty and Connector Designer certificates were configured using the server's IP address as the Common Name and Subject Alternative Name. Then, we created a truststore in which we imported the generated certificate for Connector Designer, configured it in i2 Analyze, and successfully started Connector Designer.

Thank you again for all your help and support in these days we really apreciate it.

Kind regards, Lisandro.