Drop URL validation on ElasticSearch host in proxy settings

[shaliko]

For docker-compose and OpenShift/Kubernetes deploy we have a hack for able use Elasticsearch URL.

For OpenShift/Kubernetes (or cloud services) Elasticsearch can be available by host name (not valid URL) like "elasticsearch".

Suggestion: Drop URL validation and allow user use url safe string as hostname.

Examples:

"elasticsearch" "elasticsearch-2-4" "elastic-search"

[shaliko]

After finish that story we can drop hack from docker file https://github.com/apinf/platform/blob/develop/docker-compose.yml#L14-L15.

[brylie]

Some quick notes/questions,

1. can't the internal elasticsearch url be structured to pass regex validation? E.g. http://elasticsearch.local?
   • in other words, is it the protocol in the string that is breaking things, or something else?
2. we are currently refactoring our codebase, so that the Elasticsearch URL will be irrelevant ASAP (#2036)

[shaliko]

1. Sometimes you can't use dot symbol (OpenShift).
2. If that in near feature, my request is not relevant.

[bajiat]

@shaliko #2036 will not be implemented this sprint (36), I have only asked for a research on the changes required. Depending on the amount of changes or whether any current functitonality is lost, I am considering sprint 37.

[brylie]

It is also worth considering how the API Umbrella URL field will be affected by OpenShift deployment.

• How are we deploying the API Umbrella instance related to OpenShift?
• What kind of URL will we expect from the API Umbrella instance when deployed on OpenShift?

Issue #2036 is opened in order to remove the Elasticsearch url, which will mean we communicate directly with API Umbrella for analytics data. Regardless of when #2036 is resolved, it is our planned trajectory.

[brylie]

We can easily remove the RegEx validation from any URL field, but I just want to understand the issue and explore alternatives (in context of our development roadmap).

[brylie]

E.g. it may be possible to specify a different OpenShift hostname, which would possible resolve this issue:

• OpenShift Cookbook: Specifying a different OpenShift server hostname
• OpenShift Core Concepts: Route Host Names

[shaliko]

"API Umbrella URL" because we still need access to API-Umbrella after setup for getting admin API-keys, we will need to assign regular domain name for API-Umbrella.

The issue can happen for internal services (ElasticSearch) because they should be accessible only from a private network by hostname instead of domains.

In future, if we will able get Admin API keys from "API Umbrella" without visit it from browser and do it only from Platform UI, we will have the same issue. Because "API Umbrella" will be only in private network by name like "api-umbrella".

[shaliko]

@brylie Your link for the settings of setup OpenShift hostname (admin panel), not for services run on openshift (docker images like platform, api-umbrella). That not related to our issue.

[brylie]

In order for services to be exposed externally, an OpenShift Container Platform route allows you to associate a service with an externally-reachable host name.

OpenShift Core Concepts: Route Host Names

The first sentence of that section seems to be talking about services run on OpenShift.

[brylie]

I just think this issue is not urgent, although I can understand why it seems to be a high priority. Rather than take up the issue right away, I am simply recommending we wait until #2036 progresses.

[bajiat]

Can I place this issue in icebox until we find out what will happen with #2036? The current understanding is that the data available through the admin API is really limited and would not serve our dashboard. Either we would need to enhance the admin API or we need to continue using ElasticSearch.

Or will placing this in icebox and waiting mean problems with any OpenShift experiments?

[shaliko]

I am OK with a move that issue to the icebox, fix #2036 will make not relevant that issue.

[marla-singer]

@bajiat Interested

[bajiat]

@brylie wanted have a discussion about this need in the Github issue first

[bajiat]

@marla-singer Can you make the change?

[marla-singer]

@bajiat Yes, I can

[marla-singer]

@frenchbread Now emq scheme has the regex for ElasticSearch field as well. Should it be removed?

[frenchbread]

@marla-singer Yep, similarly to api-umbrella