Microservice configuration (MP26)

[lauracowen]

Introduce MP Config, and why it is important to be able to configure microservices without having to update the app code (can move them between environments eg test, production). Give examples of what kinds of things you can do with config in MP (eg change connections to databases depending on environment, inc Kubernetes (see guide); ....see other MP features for other examples).

Include that you can put your microservice config in various places (though some of this info might be more appropriate in the server config docs and linked to instead?).

Relevant guides (for background research and to link to):

• https://www.openliberty.io/guides/microprofile-config-intro.html
• https://www.openliberty.io/guides/microprofile-config.html
• https://www.openliberty.io/guides/kubernetes-microprofile-config.html

A blog post Emily wrote:

• https://www.eclipse.org/community/eclipse_newsletter/2017/september/article3.php

Contact:

• Emily Jiang? Tom Evans?

[ManasiGandhi]

// Copyright (c) 2018 IBM Corporation and others. // Licensed under Creative Commons Attribution-NoDerivatives // 4.0 International (CC BY-ND 4.0) // https://creativecommons.org/licenses/by-nd/4.0/ // // Contributors: // IBM Corporation // :page-description: MicroProfile Config is an API that externalizes configuration from microservices, thus keeping it separate from the source code. Microprofile Config can be used by applications as a single API that can retrieve configuration information from different sources. :seo-description: MicroProfile Config is an API that externalizes configuration from microservices, thus keeping it separate from the source code. Microprofile Config can be used by applications as a single API that can retrieve configuration information from different sources. :page-layout: general-reference :page-type: general

= Microservice configuration

MicroProfile Config is an API that externalizes configuration from microservices, thus keeping it separate from the source code. Microprofile Config can be used by applications as a single API that can retrieve configuration information from different sources. When your applications are deployed as microservices, having a single place to update the configuration data of multiple microservices is useful for several reasons. • Building systems as collections of independent microservices increases the number of services that need to be managed. • Different stages in the pipeline often use different configuration sources for a single piece of configuration data and a given component needs updating as it moves through the pipeline. • Containerisation of services means that even while a component is ‘live’ in production, the environment it is running in might be more dynamic and the components need configuration updates without being restarted. The Microprofile Config API allows for a consistent means to access configuration data from pluggable configuration sources. Configuration properties can be injected using Java CDI in a form that is easy to consume. Data values and sources can vary across the devOps pipeline without code change or repackaging, and dynamic and fully typed data is catered for.

== Why it is important to configure microservices without having to update the app code?

A Micro Service can be deployed hundreds of times, using multiple different versions and across multiple different environments. The micro service needs to continue to work in all those environments without the need to modify the code. It is important to avoid embedding configuration in code, particularly if that config contains sensitive information such as passwords.

Configuration data can come from different locations and in different formats – system properties, system environment variables, properties files or resources, XML files or resources, and even data sources. MicroProfile Config calls these sources of data ConfigSources. Since the same configuration property could be defined in multiple ConfigSources, a prioritization can be used to determine which ConfigSource is used for the property value. Sometimes configuration values can change dynamically. Applications need to be able to access the latest values without restarting. This is particularly important for microservices running in a cloud environment. MicroProfile Config supports dynamically updating configuration values.

== What you can do with MP Config?

Here are some things you can do with a Microprofile Config API:

• Amalgamate multiple configuration sources into a single configuration and access with one API. • Override configuration property values with values from configuration sources that are designated as having a higher precedence. • Store values in named properties files, system environment variables, or Java™ System properties. • Load ConfigSource resources by using a Java ClassLoader, either the application’s current context ClassLoader or a user supplied ClassLoader. • Provide values by registering a user implementation of a ConfigSource interface. • Retrieve values as Strings or as typed objects of a particular Java class by using built in or custom type Converters. • Discover ConfigSource and Converter implementations by using the Java ServiceLoader pattern. • Directly inject configuration property values, whether for primitives, standard types, or user supplied types, by using Java CDI (Context and Dependency Injection).

==== See also:

[lauracowen]

Sorry, I meant can you create it as a .adoc file like this one https://github.com/OpenLiberty/docs/blob/draft/ref/general/contexts_dependency_injection.adoc (not in this issue)?

[lauracowen]

Some points I'm coming across that maybe should be included in this:

• using MP Config property is the best practice as the microservices will be portable (rather than server.xml config, in cases where there's the option to use either).
• if you put same configuration properties in MP Config and in server.xml, the server.xml one overrides the others. See also https://openliberty.io/guides/microprofile-config.html#ordering-multiple-configuration-sources for priorities of different MP Config locations.
• only some of the configuration properties are available to MP Config (others are only available in server.xml still).
• We're collating MP Config properties for each MP spec together into an issue so that we can create a reference topic of it (or something): #505

[Charlotte-Holt]

https://github.com/OpenLiberty/docs/blob/draft/ref/general/Microservice%20configuration

[lauracowen]

I think the first draft of this topic is a reasonable start but I think it needs more work. It needs to be clearer and more concise (it currently repeats some of the points several times, or sometimes only makes half a point so it's not clear what it's saying).

Structure:

• I think the focus on the topic needs to be on externalising configuration in microservices and why that's important. Then go on to discuss how that can be done using MP Config. Don't start with MP Config as that's not what the reader cares about - MP Config is the solution, not the problem they're trying to solve. In a similar way to the REST clients topic.
• The first paragraph needs to be concise and to the point about externalising configuration, what that is and why (at a high level; eg microservices move environments a lot and resources change) it's important and (no mention of MP Config at this point unless the last sentence of the intro maybe say something like "MP Config can make this easy to do...").
• Then have a section giving more detail about why externalising config is important (as you have now but without going into the details of the different files at this stage). Stick to the scenarios of when it's likely to matter. It needs to be possible to change configuration of microservices without having to update the code and recompile it because [could even refer explicitly to the 12 factor best practices that these following three points relate to - see factors 3, 7, 4 respectively and see what you think]:
  • during lifecycle, move through dev, test, prod environments in which resources such as port numbers, and the addresses and credentials to access resources (backing services) will change.
  • in the cloud (Kubernetes and containerization), microservices need to be able to talk to each other so port binding (knowing which port to connect to) is important. Ports need to change (eg [I think] as new instances of microservices are created and destroyed to scale the app up and down) so it's important that Kubernetes can assign new port numbers to microservices and that information is injected into all the microservices that need to know it as the microservices start up. see guide: https://openliberty.io/guides/kubernetes-microprofile-config.html
  • it needs to be possible to change the backing services (databases, messaging services, SMTP services, API-accessible consumer services [see Factor 4 for these examples and re-word or give examples as necessary]). These services could be managed by the same developer/company or by an external service - it shouldn't matter.
• Then introduce MP Config: briefly what it's for, and then how it works re the three default files that can be defined as ConfigSources, which have priority by default, and that the relative priorities can be changed. That there can be multiple configuration sources. Don't go into detail about how to use MP Config at a code level here (bullet points 4-8 in "What you can do with MP Config?" - remove the whole section anyway).
• Then briefly explain how MP Config is used to externalize configuration of keystores when securing microservices (MP JWT), configuration of channel attributes (MP Reactive Messaging), overriding values in fault tolerance (MP FT). There's an issue to document all the MP properties that can be configured externally and I'm currently collating them all. The topic won't be ready before this one but you can refer to it to see what I mean about how MP Config is used by other parts of MP: #505.

Link to (as they come up in the text, or at the end if not):

• relevant MP Config guides: https://openliberty.io/guides/microprofile-config-intro.html and https://openliberty.io/guides/microprofile-config-intro.html and https://openliberty.io/guides/kubernetes-microprofile-config.html
• Emily's blog post (maybe)
• the 12factor.net site
• any other of our existing doc topics that are relevant
• feature doc for config instructions: https://www.openliberty.io/docs/ref/feature/#mpConfig.html

For background research:

• the spec (the intro bit might, or might not, be useful): https://download.eclipse.org/microprofile/microprofile-config-1.3-RC4/microprofile-config-spec.html

Minor thoughts:

• I don't think it helps to keep saying "is an API" or "as a single API" - just say what it does eg "MP Config externalizes the configuration…" or "MP Config can retrieve information…". The fact that it's an API is not really that remarkable - everything in MP is an API.

[lauracowen]

Questions from Slack conversation: 1) If we mention Kubernetes (with respect to port binding) should we also provide a context for it? 2) Can you please provide more details or links for MP JWT, MP Reactive Messaging, and MP FT?

1) Kubernetes - yes...you're right, it's a little bit  of a surprise mention at the moment. How about change the earlier sentences slightly: "In the cloud, microservices deployed to Kubernetes clusters need to communicate..." and "As Kubernetes creates and destroys instances of microservices to scale the app up an down, port numbers change." Or something like that? Then your existing Kubernetes sentence makes more sense. Do you think that gives enough context to it? Should also link from this topic to https://openliberty.io/guides/kubernetes-microprofile-config.html (maybe from this section is best, rather than just sticking a list of links out of context at the end of the topic).

2) I think just give some examples of how they're used for each.

MP JWT - You could maybe just say that public key location and verification can be set in properties files so that if, say, the location of the key repository changes, it can easily be updated?? (not sure of the exact wording but check that with the reviewer). I would say to link to this guide: https://openliberty.io/guides/microprofile-jwt.html but I think this was written before MP Config came about and I don't think it's demonstrating the use of MP Config :( And we don't currently have anything else to link to, so maybe not. 

MP Reactive Messaging - eg can set the connection details of messaging channels and connectors, and the topic names that are used by the app to consume and produce messages. (see the descriptions in here https://github.com/OpenLiberty/docs/issues/505 )

MP FT - eg can set the value of an annotation like the maximum number of retries for the @Retry annotation - - you can easily change the value if (I guess!) the tolerance is found to be too low or too high (get that explanation checked as I'm speculating!) (again, see the description here https://github.com/OpenLiberty/docs/issues/505 )

At some point, when we have done the #505 issue, we can link to that topic from here but we've not got that far yet.

[chirp1]

Hi Manasi,

• Your micoservices config topic here still isn't displaying right: https://draft-openlibertyio.mybluemix.net/docs/ref/general/ . I don't see the title for it in the Docs > Refefence > General breadcrumb trail and I don't seem to see it in the Nav.
• For "thus keeping it separate " change it to a noun phrase to make clear what "it" is.
• For "Building systems as collections..." and "Different stages in the pipeline" I don't see a description of the benefit for either of them. Add a benefit to each.
• Remove the quotes from "'live'" for accessibility reasons.
• Change "without code change" to "without code changes".
• Fix the following titles. You have question marks after them but neither is a question:
  • Why it is important to configure microservices without having to update the app code?
  • What you can do with MP Config?
• Change "times, by using" to "times by using".
• for " ((e.g. development, test, production)",
  • as mentioned previously, don't use parentheses. Reword.
  • also do not use e.g. or .etc or i.e. in documentation.
• Change "It is important to avoid embedding configuration in code, particularly if that config" to "Avoid embedding a configuration in code, particularly if that configuration".
• Change "properties files or resources, XML files or resources," to "properties files, properties resources, XML files, XML resources,".
• Reword "There are many benefits of using MP Config:" so that you are not using "there are". When you write, do not use "there are" "there is".
• Reword "application’s " so that you are not using an apostrophe.
• Change "ClassLoader" to the "class loader", which is the English wording, OR put the appropfiate noun after ClassLoader.
• Should Strings and Converters be capitalized? If not, make lowercase.
• Change "supplied types, by using Java CDI (Context and Dependency Injection)." to "supplied types by using Java Context and Dependency Injection (CDI).". In general, on first occurrence, spell out an acronym and then put the acronym in parentheses.

[chirp1]

Hi Manasi, Here are my latest comments. "Microservices move among different environments and resource connection details can change." are two different thoughts. What is the relationship that you are trying to draw between the two? Also, how does this sentence relate to the second sentence in the short description? Would you you rewrite this short description so that the information is cohesive, following the 2 sentence short description guidelines that we studied in our ID workshops?

In various places a definite or indefinite article is missing in front of "configuration". For example "Externalizing configuration" is missing the. Throughout the article, add in the correct article in front of "configuration". Change " multiple environments- development, test, or production." to "development, test, and production environments." As with parentheses, don't use hyphens. They are both shorthand. Instead, write out what you have to say.

Would you rewrite this sentence as I find it a bit confusing: "Each backing service must be attached and detached as a resource to your application, as required, without a code change, or a distinction between a local and external service." It's the ", or a distinction between a local and external service." that is confusing. You can write it as multiple sentences if necessary. I think that you have too many commas, "or's" and "and's" in one sentence.

Use "application" instead of "app". Do so whenever you write.

Change "Applications can use MicroProfile Config as a single API to retrieve configuration information from different sources. Configuration data can come from different locations and in different formats – system properties, system environment variables, properties files, XML files, or data sources." to "Applications can use MicroProfile Config as a single API to retrieve configuration information from different sources such as system properties, system environment variables, properties files, XML files, or data sources." <= scrubs out unnecessary words.

Speak to the customer by using "you" and active voice instead of using passive voice. So, for example, change "A prioritization can be used to determine which ConfigSource is used for the property value. " to "You can prioritize which ConfigSource to use for the property value.". Chanage the wording in other places, too.

Generally for titles, don't have titles that are sentences. For "MicroProfile Config has three default configsources", change it to something like "Default configsources".

For "MicroProfile Config helps in overriding values in fault tolerance.", scrub out non-essential words. So, you could change the wording to "MicroProfile Config helps override values in fault tolerance.".

• Karen

[ManasiGandhi]

Hi @lauracowen, I moved the MP Config issue into ready to publish after talking with Karen. Here is a link for your reference. https://draft-openlibertyio.mybluemix.net/docs/ref/general/#mp-config.html

Can you please review?

[lauracowen]

Some last review comments:

• "without a change in code" reads a bit oddly. Try "without needing to change the code."
• "They need different configuration data" -> "Different environments often need different configuration data…"
• "or credentials that change regularly" -> "or credentials that can change regularly"
• "without a change in code" -> "without changing the code" (in multiple places)
• "Your application must swap among local and external services " -> "Your application must be able to swap between different local and external services"
• I think the first and section paragraphs in the "Separating…" section should be collapsed together and remove the sentence "Microservices must run in these conditions without a change in code." as the two paragraphs currently repeat each other to some extent.
• Karen's point about putting "the" before "configuration" is not relevant in all contexts here. The article is talking about the general concepts of configuration and code, not a specific configuration. So the main heading should be "Externalizing configuration in microservices"; in the intro paragraph "Externalizing configuration with MicroProfile…"; first sub-heading "Separating configuration from code" (I think it should be "from" here, btw). And so on.
• heading "MicroProfile Config" could maybe be more explicit/meaningful as "Externalizing configuration with MicroProfile Config". Then remove the first sentence under that heading because it just says the same thing.
• "into services in the containers without repackaging them." -> maybe "into applications in containers without recompiling the appplication code or repackaging the applications."
• "configsources" isn't a real word so you need to capitalise it correctly if you use it like that (ie ConfigSources). Otherwise, write it out as "configuration sources".
• the last para of the topic needs its own heading (eg "Externalizing configuration from other MicroProfile features" or something).
• the last para is a summary of things that you can use MP Config to do. So say "can" and, as Karen mentioned, use the active voice rather than passive. eg "You can use MicroProfile Config to configure the keystores when securing microservices. For example, in MIcroProfile JWT…"; "You can also use MIcroProfile Config to configure channel attributes. For example, in MicroProfile Reactive Messaging, you can set…"; "You can override values of fault tolerance annotations with MicroProfile Config. For example…". MP Config doesn't necessarily do any of these things but the developer can use it to do them if they want (that's the point I'm trying to make.)
• "JWT" doesn't stand for "Java Web Token". The name of the MP feature is "MicroProfile JSON Web Token" or can be shortened to "MicroProfile JWT".

Thanks

[ManasiGandhi]

Published link of the topic on the live OL site https://openliberty.io/docs/ref/general/#mp-config.html