WARNING: An illegal reflective access operation has occurred

DJSmy commented 6 years ago

F:\Program Files\Landscape Editor 2D>java -cp lib/xstream.jar;lib/xpp3.jar;mapeditor.jar com.GUI WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/F:/Program%20Files/Landscape%20Editor%202D/lib/xstream.jar) to field java.util.Properties.defaults WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release

joehni commented 6 years ago

Yes, XStream can either serialize only half of the object or avoid the illegal access.

AlanBateman commented 6 years ago

The ReflectionFactory API was updated in JDK 8 and JDK 9 to help custom serialization libraries work with modules that want to encapsulate their internals. Look for the methods that return a method handle to the common readXXX/writeXXX methods.

joehni commented 6 years ago

@Alan: So, the application can open its class types ... that will not help to access the default properties of a Properties instance, right?

AlanBateman commented 6 years ago

@joejni - Properties does not define a method to return the defaults in bulk. If there is a strong case to add such a method then it should be brought to core-libs-dev for discussion.

DJSmy commented 6 years ago

Tell me, what is the work around for this? With the release of Java 9, is XStreem going to become an invalid library?

joehni commented 6 years ago

The current workaround is to permit illegal access.

You will always be able to serialize your own classes, since it is up to you to open that module for XStream. In case of 3rd party classes this might be different and you will have to write your own converters in future instead of using XStream's reflection-based converters assuming you are able to implement those converters so, that you can recreate the object.

For some types in the JDK this might be no longer possible or only with limitations (e.g. currently no default entries for Property objects).

This reflects at least my current knowledge of the restriction with the new module system.

Katharsas commented 6 years ago

I want to refactor my code, so that XStream does not need to use reflection as much. I get
Illegal reflective access by com.thoughtworks.xstream.converters.collections.TreeMapConverter.

I guess it's caused by reflection-based deserialization of Java collections like HashSet, could this be right (i don't think i serialize any TreeMaps directly)? Is there a way to exatcly determine the class that i need to write custom converters for, or to log where exactly xstream uses such "illegal" reflection?

joehni commented 6 years ago

You may overwrite the setupConverters method and register only the converters you're going to use. Note, there's no problem using reflection-based converters for your own classes, if you open your module for XStream.

ulfjack commented 6 years ago

I think this is probably what you meant:

XStream xstream = new XStream(new StaxDriver()) {
      @Override
      protected void setupConverters() {
      }
    };
    xstream.registerConverter(new ReflectionConverter(xstream.getMapper(), xstream.getReflectionProvider()), XStream.PRIORITY_VERY_LOW);
    xstream.registerConverter(new IntConverter(), XStream.PRIORITY_NORMAL);
    xstream.registerConverter(new StringConverter(), XStream.PRIORITY_NORMAL);
    xstream.registerConverter(new CollectionConverter(xstream.getMapper()), XStream.PRIORITY_NORMAL);

GedMarc commented 6 years ago

XStream reads package org.xmlpull.v1 from both xmlpull and xpp3.min

Error:java: module xpp3.min reads package org.xmlpull.v1 from both xmlpull and xpp3.min Error:java: module xstream reads package org.xmlpull.v1 from both xmlpull and xpp3.min Error:java: module xmlpull reads package org.xmlpull.v1 from both xmlpull and xpp3.min

:'( Sad Panda

davewichers commented 6 years ago

I just ran into this same issue with Java 10. As an FYI, this is still considered a WARNING in Java 10 as well (java version "10.0.1" 2018-04-17), so no difference between Java 9 and 10, yet anyway. [INFO] --- maven-war-plugin:2.2:war (default-war) @ benchmark --- WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:toMy/.m2/repository/com/thoughtworks/xstream/xstream/1.3.1/xstream-1.3.1.jar) to field java.util.Properties.defaults WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release

khmarbaise commented 6 years ago

@davewichers You should use a more recent version of maven-war-plugin cause this version you are using is about six years old...The most recent version 3.2.2 uses the most recent version of xstream 1.4.10 ....

handsimeboy commented 6 years ago

now,i have got the same problem when i use "mvn install",and it says"illegal reflective access by com.thoughtworks.xsteram.core.util.fields to field java.util.properties.defaults". it's very similar to your problem, so how do you solve this problem?i didn't get it from your talking....thanks

handsimeboy commented 6 years ago

i should tell you first that jdk10 and tomcat8.5 were used in my pc.

gschrader commented 6 years ago

I'm getting the same split package issue that @GedMarc is above, are people using --patch-module to address this?

martinm1000 commented 5 years ago

@gschrader

What worked for me was to only use xstream without any other of its dependencies, which by luck skipped those problems. Serialization might be slower but that's not a problem for me.

XStream x = new XStream(new DomDriver());

<!-- https://x-stream.github.io -->
            <dependency>
                <groupId>com.thoughtworks.xstream</groupId>
                <artifactId>xstream</artifactId>
                <version>1.4.10</version>
                <exclusions>
                  <exclusion>
                    <groupId>xpp3</groupId>
                    <artifactId>xpp3_min</artifactId>
                  </exclusion>
                  <exclusion>
                    <groupId>xmlpull</groupId>
                    <artifactId>xmlpull</artifactId>
                  </exclusion>
                </exclusions>
            </dependency>

gschrader commented 5 years ago

Thanks @martinm1000 I will try that as well since performance isn't a concern

joehni commented 5 years ago

XStream is not responsible for problems in depending libraries. If you use the Xpp3Driver directly, you can omit exclude xmlpull alone.

aaime commented 5 years ago

I've tried to follow @joehni suggestion and override the setupConverters method. Ended up having some small difficulties with the approach:

The InternalBlackList seems something "important", but it's not visible to subclasses, I had to copy over its code in my XStream subclass
The registerConverterDynamically is not visible, again, had to make a copy of it
The TreeMapConverter and TreeSetConverter use some reflection that helps with performance, but it's not strictly required, I ended up copying over the class contents for tree and set converter and removing the optimizations. It may be nice to have that split in base class and optimized subclass, so that JDK 11 users do not have to do this operation?
The collection converter does not handle automatically a few type of private classes, such as java.util.Arrays$ArrayList, java.util.Collections$UnmodifiableList, java.util.Collections$UnmodifiableSet, that should be easy to marshal as common lists without having to do deep reflection (which causes warnings). This is likely a bit is specific, as I don't care them to come back in their original wrapped class, but in case you're interested, here

@joehni if you agree with some of the above changes I could try to make a PR (time permitting).

joehni commented 5 years ago

Hi,

let's make some comments about your topics:

the approach to overload setupConverters is used for your optimized scenario, i.e. when you know which converters you're going to use
the InternalBlackList is a bad hack for 1.4.x that does not exists in 1.5, therefore it is not part of any API and it is only used if you stubbornly refuse to initialize the security framework
since you know, what converters you need, there's no need to register them dynamically
the reflection used for the maps and sets is essential and not only an optimization. It is required if you have circular dependencies between your objects and keep them in a hashed or sorted collection. In such a case you have to avoid absolutely that the hashCode or compare methods of those objects are called when you add them to the collection, because they might not have been completely initialized yet. This might break an application possibly in some very subtle ways (JIRA 184). A NPE might be quite obvious, but it might also happen that the hash code of an object changes after it has been added to a hashed which will compromise the functionality of that collection. Same applies to sorted collections, where the elements are no longer sorted after deserialization.
those special collections have special functionality. XStream aims for a 1:1 marshalling

See, it is not, that I don't know what to do to make the compiler happy, I care more for the user's applications.

aaime commented 5 years ago

@joehni thanks for your feedback.

About "knowing which converters I need", it's not actually/fully true, GeoServer is a pluggable application and developers outside of the core ones can create their own extensions and add stuff into the configuration beans (the ones that we serialize with XStream) that was not foreseen. Also, with the advent of custom JDK builds, GeoServer might end up working against reduced side JDKs, it would still be useful to have access to the dynamic registration method imho.

About the sets, I understand. Just one observation, right now using XStream causes the warnings to be emitted whether one actually uses TreeMap/TreeSet classes or not. I know that GeoServer users will start hitting us with complaints about the warnings, so gonna keep the custom converters for now, just to avoid those and the associated panic. We do use TreeSet, so I have no way out, but others might not and still get the warnings, this is going to trigger migrations away from xstream that are imho not fully justified. It would help, imho, if the converters in questions did the static reflective lookups only when needed (e..g, via a helper class), instead of at the beginning... a "pay as you go" model where one gets the warnings only for what is actually needed, and can then decide if open java.util for deep reflection via command line params, remove usage of those classes, or do something else, but without having to clone hundreds of lines from XStream.setupConverters. Just a thought.

joehni commented 5 years ago

Thanks for your feedback.

Java 9 with module path is actually somewhat like the old restricted mode available for Java 1.4 only with all the limits. The static lookups were mainly used to setup the environment once. However, the code has to be updated for the new runtime environment. E.g. java.base will never be open for XStream in a module path and as long as the Java runtime does not provide functionality to overcome the restrictions, users will have to live with it. However, that's the main reason, why xstream is kept currently in the unnamed module.

ipkpjersi commented 5 years ago

Hi,

I am wondering, is this illegal reflective access something that can/will be fixed in a new version of xstream, or is this not really possible? I am aware of a scenario where netty was able to fix illegal reflective access: https://github.com/netty/netty/issues/8318 and I was wondering if something like that may be possible here? I'd love to see xstream continue working in Java 12 and beyond.

Thanks.

twogee commented 5 years ago

WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:xstream-1.4.11.1.jar) to field java.util.TreeMap.comparator Using Java 11

melloware commented 5 years ago

@twogee I have the exact same issue with Java 11 on Treemap.comparator.

ThomasDaheim commented 5 years ago

Same here. I'm pretty sure I don't want to serialize java.util.TreeMap.comparator. So I would happily exclude it if I only knew where to start...

McDime commented 5 years ago

Any update on this? Is it safe to ignore this warning with Java 11? XStream is a sub-dependency of Spring Cloud. Is there a way to enforcing DomDriver not from the code, but somehow via some configuration? Just excluding xpp3_min from the dependency tree does not help really in my case.

melloware commented 5 years ago

I am in the same boat as @McDime this comes as part of Spring Cloud and was hoping to get it ignored or resolved.

ipkpjersi commented 5 years ago

Hello everyone,

I am pretty confident that this warning means that xstream will stop working eventually, whether that be in Java versions beyond 12 or 13, I am pretty sure that eventually it will stop working. So, this issue likely needs to be addressed for the long-term survival of this project.

Cheers.

AlanBateman commented 5 years ago

It sounds like xstream needs to be tested with --illegal-access=deny to shake out issues where it might depend on hacking into JDK internals. As I mentioned in a post in 2017, the ReflectionFactory API was updated in JDK 9 to provide help to custom serialization libraries; IIOP serialization (CORBA) is one example that was updated to make use of that.

Hronom commented 5 years ago

Got updated to spring cloud:

            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>Greenwich.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>

and I'm in trouble with this warning WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields, please do something

mokun commented 5 years ago

I have the following warning :

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/C:/Users/mkhelios/.m2/repository/com/thoughtworks/xstream/xstream/1.4.9/xstream-1.4.9.jar) to field java.util.TreeMap.comparator
WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

What can I do with the java.util.TreeMap.comparator to remove the warning ?

Fortunately, I don't need to serialize any class that uses Xstream in my app.

alwyn commented 5 years ago

Reading a few comments it sounds like the 'official' approach is to 'change your code to avoid the issue'.

This is the wrong way of looking at it.

Your library is a transitive dependency of other dependencies. You can fix it 'upstream' by correcting the behavior in the default case and making the broken case optional via configuration.

bsbodden commented 5 years ago

What @alwyn said. In my case, I have several projects with Java 11 now they are showing this warning, where xstream is a transitive dependency. Here's one with a Drools 7 project:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/Users/bsb/.m2/repository/com/thoughtworks/xstream/xstream/1.4.9/xstream-1.4.9.jar) to field java.util.TreeMap.comparator
WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

pikomdas commented 5 years ago

WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by cucumber.deps.com.thoughtworks.xstream.core.util.Fields (file:/C:/Users/partha.das/.m2/repository/info/cukes/cucumber-jvm-deps/1.0.5/cucumber-jvm-deps-1.0.5.jar) to field java.util.TreeMap.comparator WARNING: Please consider reporting this to the maintainers of cucumber.deps.com.thoughtworks.xstream.core.util.Fields WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release

is any fix available so far?

JiangHongTiao commented 5 years ago

We would like to update dependencies as well to Java 11 (LTS) Thanks for your time spending on this!

jawadnaeem commented 5 years ago

Hi, Any idea when solution of this problem is expected?

Thanks.

marschall commented 5 years ago

We removed XStream from our project because of how this issue was "handled".

jawadnaeem commented 5 years ago

@marschall, alternate you used then?

joehni commented 5 years ago

Please, if you look for help for another software, then this is definitely not the right place.

jawadnaeem commented 5 years ago

Please, if you loog for help for aother software, then this is definitely not the right place.

I am looking for solution of this problem facing due to XStream jar

Hronom commented 5 years ago

@joehni we all looking for a solution for this library, please read task description and all comments.

joehni commented 5 years ago

@jawadnaeem It is one thing to ask for an alternative to XStream, but another to ask for help using this alternative. The alternative has its own issue trackers and forums. It is out of scope here.

@Hronom What do you expect? XStream is based on stuff that is currently only available if the library is on the classpath. The official Java API does not provide alternatives for the used functionality. If XStream restricts itself to the now available functionality, some of you might be very surprised what no longer works. Especially if you rely on the fact, that it still can unmarshal your already existing XML data.

zhaokejin commented 5 years ago

SpringBoot Euraka starts WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.thoughtworks.xstream.core.util.Fields (file:/D:/Java/m2/repository/com/thoughtworks/xstream/xstream/1.4.10/xstream-1.4.10.jar) To field java.util.TreeMap.comparator WARNING: Please consider reporting this to the maintainers of com.thoughtworks.xstream.core.util.Fields WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release How to solve it? What is going on with java 11?

Taynan-Vieira commented 5 years ago

Were you able to solve the problem?

twogee commented 5 years ago

So the current state of affairs is that for Java 9+ xstream does not work at all on module path and behaves like this on classpath?

twogee commented 5 years ago

Why is #120 closed? Will there be a 1.5?

Hronom commented 5 years ago

So I want to ask one question related to the future versions of JDK.

It seems like this library will become unusable at newer versions of JDK, so all dependent libraries should revisit theirs approaches and change related stuff and stop use this library, is this correct?

twogee commented 5 years ago

There may be some hope with 1.5 still, maybe @joehni could clarify if there were some recent developments.

GedMarc commented 5 years ago

@Hronom Yes that is definitely correct. Libraries that break the fundamental rules of Java/Encapsulation are no longer given leniency. You can of course run your apps in "classpath" mode, but that too will be dropped (in no given time frame), but you will feel a very strong performance drop.

It's worth noting though, between Jackson 2.10 (which is jpms/jlink 100%), and JAXB (which eclipse has already moved into the jakaarta space minus javax.activation) are also JLink friendly (AKA 100% Modular)

JAXB was also finalized and has satisfied all enterprise requests for a good more than 5 years, with very minimal changes between each release there-after. Those nasty performance problems, and the serialize/deserialize anything problems have also been solved (JAXBIntrospector).

So the real question is, which I painfully had to admit when coming across this, and the few years of trying to resolve it, was sadly - why was I using this, when all the issues in the standardized libraries are resolved.

x-stream / xstream

WARNING: An illegal reflective access operation has occurred #101