Is "-Dtangosol.coherence.override" not used in Coherence CE? Still talking about the old v14 sorry for that.... :-(

[javafanboy]

According to the documentation shared between Oracle Coherence and Coherence CE one should be able to specify an "override file" with -Dtangosol.coherence.override but as far as we can tell (we just noticed it and is still investigating/finding work arounds), at least in V14 that we still use, this parameter is ignored by CE and only the file "jar:file:/opt/oas/cache/lib/cache-coherence-2.7941.0.jar!/tangosol-coherence-override-prod.xml" seem to be read?! Not sure if this is the intended behaviour in CE and if so is it documented somewhere?

[mgamanho]

We're phasing out the 'tangosol.' part from configs. Try using 'coherence.override' and see if it helps. If not let us know we'll investigate. I'll verify and check whether we document this.

[chpatel3]

Hi @javafanboy, we need couple of clarification

1) What do you set -Dtangosol.coherence.override to? 2) What is the content of the "jar:file:/opt/oas/cache/lib/cache-coherence-2.7941.0.jar!/tangosol-coherence-override-prod.xml" which is packaged inside your cache-coherence-2.7941.0.jar?

[javafanboy]

1. We set it to the name of a config file that is located in the root of the same JAR file as and side by side with the tangosol-coherence-override-prod.xml (that is loaded instead). We have a large number of different override files there for different configurations and we need to select one for each environment using with the "-Dtangosol.coherence.override" property but as mentioned for some reason we don't manage to get this working...
2. That is a "default" config that we use if no override is specified but we plan to always use one of the override files for real environments so it is just a fall back for test on a laptop or something... A real environment will not work as expected if it is used and that was how we noticed that the tangosol.coherence.override did not take effect. We also looked in the log that did not mention loading our desired override and instead stated that tangosol-coherence-override-prod.xml was loaded.

I also have a question about a parameter called "-Dcoherence.mode" - is this used for Coherence CE or is it just a licensing thing for the enterprise edition? If it is used in CE what exactly does it do - the description in the doc is "not very complete" to say the least... We have specified it to the value "prod" as the documentation seemed to hint that this had some "benefits"... not sure if this can have anything to do with the problem we are seeing?! Please advice if we should try not setting it....

[thegridman]

Maybe we should explain how the Operational config is loaded.

1. Coherence loads it's root Operational config from coherence.jar - this is the tangosol-coherence.xml file and you should never replace this with a file of the same name.
2. In the root <coherence> element of the tangosol-coherence.xml file is this attribute xml-override="/tangosol-coherence-override-{mode}.xml" which specifies that the overrides to apply should come from a file named tangosol-coherence-override-{mode}.xml where {mode} get replaced with the value from the coherence.mode System property. Coherence includes a tangosol-coherence-override-dev.xml tangosol-coherence-override-prod.xml and tangosol-coherence-override-eval.xml files in coherence.jar
3. The root <coherence> element of the tangosol-coherence-override-{mode}.xml files contains xml-override="{coherence.override /tangosol-coherence-override.xml}" which means after loading the correct tangosol-coherence-override-{mode}.xml file, more overrides will be applied by loading the tangosol-coherence-override.xml file (or whichever file is specified by the coherence.override System property).

So the question asked by Chinmay above should maybe have been clearer - is the file tangosol-coherence-override-prod.xml in your cache-coherence-2.7941.0.jar a custom override file you have written yourself, and not a copy of the tangosol-coherence-override-prod.xml file from coherence.jar? If it is your own xml file, and it does not have the xml-override="{coherence.override /tangosol-coherence-override.xml}" attribute in the root <coherence> element, then whatever is specified by -Dcoherence.override will not be loaded.

Regarding the coherence.mode property; many versions ago Coherence used to recommend some slightly different configurations between environments like Dev and Prod, and the method of loading overrides allowed for this. In more recent versions, the need for these differences have gradually been removed until now I believe the only difference is the log buffer limit.

[javafanboy]

Thanks for the full explanation - I can now confirm that we with a minimal example (using the DefaultCacheServer) managed to get the override file to load so this is NOT an problem with Coherence CE as we first speculated.

But are still struggling with our real environment deploy where we neither see any log saying that the override is read or ignored/not-found etc but do see the following warning and error messages where we would have EXPECTED to see this type of log entries (i.,e. where other logging is emitted about loaded and not provided override files):

To guard against XXE vulnerabilities, ensure provided XML parser is secure. Validator: org.apache.xerces.jaxp.validation.ValidatorImpl Error: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.

Right now we are working with the theory that loading the override somehow triggers this error resulting in it geting ignored. Unfortunately we do not have any "XML Guru" in our team so we are a little bit unsure how to proceed debugging the error... The search goes on - we are suspicious if we in some way have specified some XML parser version in the classpath (that is like a mile long as we also use wildfly and a lot of other stuff) that causes the problem (i.e. we suffers from some "JAR hell" thing) but that is more of a guess than built on any facts right now...

The override file we TRY to load is, as far as we can tell, syntactically correct and is attached (renamed to extension txt as xml was not allowed).

master-coherence-Staging-override.txt

[thegridman]

The message "To guard against XXE vulnerabilities..." is a warning because you have Xerces on your class path and version of Xerces being used does not support JAXP version 1.5. The just means you have no protection against XXE and can be ignored, it is not the cause of any issues loading the overrides files.

Coherence logs out the Operational configurations it is loading when it starts, for example:

2023-03-08 20:20:59.936/0.449 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Loaded operational configuration from "jar:file:/Users/jonathanknight/dev/Coherence/dev/main/prj/coherence/target/coherence-15.1.1-0-0-SNAPSHOT.jar!/tangosol-coherence.xml"
2023-03-08 20:21:00.006/0.506 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Loaded operational overrides from "jar:file:/Users/jonathanknight/dev/Coherence/dev/main/prj/coherence/target/coherence-15.1.1-0-0-SNAPSHOT.jar!/com/oracle/coherence/defaults/tangosol-coherence-override-dev.xml"
2023-03-08 20:21:00.008/0.507 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified

The log above shows Coherence first loads coherence-15.1.1-0-0-SNAPSHOT.jar!/tangosol-coherence.xml, then it loads tangosol-coherence-override-dev.xml and finally it says there is no /tangosol-coherence-override.xml file on the class path.

If I specify -Dcoherence.override=foo.xml where foo.xml does not exist, then it los this:

2023-03-08 20:21:53.920/0.450 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Loaded operational configuration from "jar:file:/Users/jonathanknight/dev/Coherence/dev/main/prj/coherence/target/coherence-15.1.1-0-0-SNAPSHOT.jar!/tangosol-coherence.xml"
2023-03-08 20:21:53.990/0.507 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Loaded operational overrides from "jar:file:/Users/jonathanknight/dev/Coherence/dev/main/prj/coherence/target/coherence-15.1.1-0-0-SNAPSHOT.jar!/com/oracle/coherence/defaults/tangosol-coherence-override-dev.xml"
2023-03-08 20:21:53.992/0.509 Oracle Coherence 15.1.1.0.0 (dev-jonathanknight) <Info> (thread=Coherence, member=n/a): Optional configuration override "foo.xml" is not specified

Where the final line says "foo.xml" is not specified so it is ignored and Coherence continues to start.

So the log should be displaying exactly what Coherence is loading (or not able to find).

[javafanboy]

Thanks for all the information and helpful advice - we found no less than 3 errors in our configuration that together gave the problem we observed that we now have managed to fix - the way config files work in Coherence is powerful but a bit complex...