dkpro / dkpro-core

Collection of software components for natural language processing (NLP) based on the Apache UIMA framework.
https://dkpro.github.io/dkpro-core
Other
195 stars 67 forks source link

desc/type/POS.xml not found in SOLR but fine outside SOLR #1348

Closed ggiavelli closed 5 years ago

ggiavelli commented 5 years ago

I think this might be related to classloader or classpath? I checked the segmentation-asl jar and the file is there. It runs fine outside of SOLR from eclipse.

Caused by: org.apache.uima.util.InvalidXMLException: An import could not be resolved. No file with name "desc/type/POS.xml" was found in the class path or data path. (Descriptor: jar:file:/C:/dev/solr-7.5.0/server/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.10.0.jar!/desc/type/LexicalUnits.xml)

I have the jar I produce added to the schema instance lib path, configured to load in the lib, and just for grins created a system classpath and added it to that as well.

It happens when I first try to create a JCas object mJCas = JCasFactory.createJCas();

reckart commented 5 years ago

Are you using a fat JAR?

ggiavelli commented 5 years ago

Yes I am. I have compile with dependencies set to maven putting all the sub jars into one

reckart commented 5 years ago

See: https://uima.apache.org/d/uimafit-current/tools.uimafit.book.html#ugr.tools.uimafit.packaging

ggiavelli commented 5 years ago

thanks will try and get back on it

ggiavelli commented 5 years ago

Just an update. I set up the maven shade plugin and then got a security exception java.lang.SecurityException: Invalid signature file digest for Manifest main attributes at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:330) ~[?:1.8.0_191]

so then, I tried adding an exclude for security files and the unpacking plugin. (that was a recommended solution) But it might not be using it. Here is my POM

 <build>
    <sourceDirectory>src/main/java</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.7.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
           <excludes>
                 <exclude>**/main/</exclude>
         </excludes>  
        </configuration>
      </plugin>
       <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-shade-plugin</artifactId>
      <version>2.2</version>
      <executions>
        <execution>
          <phase>package</phase>
          <goals><goal>shade</goal></goals>
          <configuration>
            <transformers>
              <!-- Set the main class of the executable JAR -->
              <transformer
                implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                <mainClass>MyMainClass</mainClass>
              </transformer>
              <transformer
                implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/org.apache.uima.fit/fsindexes.txt</resource>
              </transformer>
              <transformer
                implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/org.apache.uima.fit/types.txt</resource>
              </transformer>
              <transformer
                implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
                <resource>META-INF/org.apache.uima.fit/typepriorities.txt</resource>
              </transformer>
            </transformers>
            <outputFile>${project.build.directory}/${artifactId}-${version}-standalone.jar</outputFile>       
          </configuration>
        </execution>
      </executions>
    </plugin>

     <plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-dependency-plugin</artifactId>
  <version>2.8</version>
  <executions>
    <execution>
      <id>unpack-dependencies</id>
      <phase>package</phase>
      <goals>
        <goal>unpack-dependencies</goal>
      </goals>
      <configuration>
        <excludes>META-INF/*.SF,META-INF/*.DSA,META-INF/*.RSA</excludes>
      </configuration>
    </execution>
  </executions>
</plugin>
  </plugins>    
    <resources>
        <resource>
            <directory>src/main/resources</directory>
            <excludes>
                <exclude>*.properties</exclude>
            </excludes>
            <filtering>false</filtering>
        </resource>
    </resources>
</build>
reckart commented 5 years ago

I haven't been working with signed JARs, but I doubt this works out of the box with fat JARs. Probably you'll have to tell the shade plugin to drop the signature files. I don't think telling the dependency plugin will do much.

reckart commented 5 years ago

Btw, if your fat JAR has no primary main class, then you should drop the MyMainClass section from the shade configuration.

ggiavelli commented 5 years ago

Thanks, I was thinking the same thing. will keep trying. will let you know if I get it.

ggiavelli commented 5 years ago

I did the individual jars approach and still go the same issue. So its not the combined jar, it must be some kind of classpath error. I checked and what it is looking for is there exactly where it is looking.

No file with name "desc/type/POS.xml" was found in the class path or data path. (Descriptor: jar:file:/C:/dev/solr-7.5.0/server/lib/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl-1.10.0.jar!/desc/type/LexicalUnits.xml)

Do I need to go down this Generate PEAR file path? Seems like a ton of work: https://uima.apache.org/doc-uima-pears.html

reckart commented 5 years ago

The file desc/type/POS.xml would be in the JAR de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.10.0.jar - do you have that on your classpath? Maybe it is a matter of Maven dependency scopes (i.e. compile vs test)?

ggiavelli commented 5 years ago

I think thats in most of the components. Its in the dependency jar as well. Its getting the error totally outside eclipse, when I stage it in SOLR. Runs fine in eclipse. thats the scooby doo mystery. I believe that is the PEAR info for the component if I'm not mistaken, for UimaFit

reckart commented 5 years ago

Its in the dependency jar as well

@ggiavelli The desc/type/POS.xml file should only be in de.tudarmstadt.ukp.dkpro.core.api.lexmorph-asl-1.10.0.jar- what do you mean "Its in the dependency jar as well"?

reckart commented 5 years ago

Since there was no further communication, I assume this could be resolved.