BASE_URL_HTTPS contains wrong IP address

[shimizuf]

Under certain circumstances the BASE_URL_HTTPS of the deployed CORE contains the wrong IP address. This causes the hyperlinks in email notifications to be broken and may also affect communication between ESCM components.

This may be a regression introduced by servicecatalog/oscm-dockerbuild/issues/100.

The current theory follows.

Normal first deployment

• Usually on the first Chef run the IP address and several other variables are empty
• Because of this, deploy-escmserver aborts
• On the next run these values are filled with correct values and deploy-escmserver proceeds to deploy the application

Error happens

• An old deployment exists with an IP address, e.g. a.b.c.10
• Operator deletes old deployment stacks and applies the Barclamp again to deploy a new ESCM
• The docker VM instance is created with a new floating IP, e.g. a.b.c.20
• On the first Chef run the variables which are usually empty are now filled with the old values; in the case of IP: a.b.c.10
• deploy-escmserver does not abort because the values are filled (but incorrect) and proceeds to deploy ESCM with the old IP address

Theory of why this did not happen before servicecatalog/oscm-dockerbuild/issues/100:

• Before the change, configsettings in the DB were overwritten by initdb
• First deployment was still with the wrong IP address
• But the second and subsequent Chef runs had the new, correct values and initdb overwrote these in the DB

To verify this theory, one can verify the following: 1) The error does not happen if there was no prior docker VM instance with a floating IP. It only happens after deleting an existing deployment and then applying the Barclamp again 2) The incorrect IP address in the BASE_URL_HTTPS setting is in fact the old IP of the previous instance and not some other IP 3) Usually the very first run of deploy-escmserver aborts because of some empty variables. This can be seen at the beginning of the log file /docker/logs/setup-machine-. On a deployment where the error happens and the IP is wrong, the first run of deploy-escmserver did not abort. There is no message "Mandatory variables unset - aborting" very close to the beginning of the log file

[GoebelL]

@kowalczyka Please check. Can you confirm @shimizuf's assumption?

If so, I wonder if this is suitable to consider as an installer bug. If the cause is that settings from previous installations retain, the issue may probably be that it was not entirely or properly cleaned-up before reinstalling. Or, from where else does the Chef run get the old IP address? I guess there should be an operational way to get rid of it. Naively speaking I'd expect something like clear cache, initialize crowbar, or similar to complete the clean-up procedure. This would be maximum a documentation request.

In any case, what are the options to resolve it without barclamp update?

1. Solution or workaround, e.g. as mentioned above
2. Avoid IPs. As far as I remember, the URLs in configuration settings, such as BASE_URL_HTTPS, are supposed to contain the host FQDN rather than its IP.
3. Revert the fix of servicecatalog/oscm-dockerbuild#100

The later would open again the usability discussion, while the second could be risky (if applicable at all). I would go for 1. Any else?

[shimizuf]

Regarding 2: Unfortunately in SOC it's entirely up to the operator and their environment to set up a host name and FQDN for the floating IP. The only thing we can be sure of is the floating IP of the instance, so that's what we use. The operator may choose to run ESCM entirely without host name and FQDN. Thus I think we can only use the IP.

As for 3: If the current assumption is correct, this would work around the problem.

For 1: The first of our scripts, heat-config, receives the API endpoint of Heat. But the second script deploy-escmserver and the Deployer do not. If the Deployer knew the endpoint address, it might be possible to query Heat directly for the instance's floating IP address and verify whether the variable is correct. But we would have to find a way to pass the endpoint address to the second script or to the Deployer and then implement the check without changing Barclamp code.

[kowalczyka]

I can confirm the issue anf @shimizuf's assumption.

As for 1: I did not find any offitcial information about how to clear chef so that when reinstalling all its attributes are clear. There is a var/chef/cache dir on the admin node - I will check if cleaning it can resolve the issue.

As for 2: I agree with @shimizuf. The HOST_FQDN which we can see in configuration settings is the env variable and its value is propagated from chef script.

As for 3: Reverting the fix is one option, the other one which I came up with, would be a workaround with adding the simple sql script with updating those few configsettings which includes HOST_FQDN variable - this script should be always executed at the end of initdb container run.

[shimizuf]

adding the simple sql script with updating those few configsettings which includes HOST_FQDN variable - this script should be always executed at the end of initdb container run

I think this is a very good idea. It would work with the current Barclamp code and the effort should be reasonable. And it can be implemented entirely in the Docker code.

[kowalczyka]

Issue resolved within docker containers area according to what has been agreed above.

For the details please refer to: https://github.com/servicecatalog/oscm-dockerbuild/commit/a370b19469338c59bcf4c4c130ff725cd4ef0297