search

dkpro / dkpro-tc

UIMA-based text classification framework built on top of DKPro Core and DKPro Lab.
https://dkpro.github.io/dkpro-tc/
Other
34 stars 19 forks source link

DKPro TC Release 1.0.0 #421

Closed Horsmann closed 6 years ago

Horsmann commented 6 years ago

ToDo list (no release date yet)

DKPro family

General dependencies

Close remaining issues

reckart commented 6 years ago

@Horsmann DKPro Core 1.9.0 is staged here: https://oss.sonatype.org/content/repositories/detudarmstadtukpdkpro-1041

Planning to release it to Maven Central soon.

Horsmann commented 6 years ago

I created a branch core190 where i added this URL as repository and set the version of core-dependencies to 1.9.0. Building this branch crashes with this serialization issue. I am still using 1.8.0 in TC. Its probably not an issue with the RC but a general change between 1.8. and 1.9 ?

reckart commented 6 years ago

Possibly. What is the problematic class?

Horsmann commented 6 years ago

java.lang.NoSuchFieldError: SERIALIZED at org.dkpro.tc.ml.TestFoldUtil.createNoSequenceCas(TestFoldUtil.java:324) at org.dkpro.tc.ml.TestFoldUtil.setUp(TestFoldUtil.java:236)

the call looks at the moment like this: AnalysisEngine xmiWriter = AnalysisEngineFactory.createEngine(BinaryCasWriter.class, BinaryCasWriter.PARAM_TARGET_LOCATION, tmpFoldNoSeq.getRoot(), BinaryCasWriter.PARAM_FORMAT, "6+");

Horsmann commented 6 years ago

I am also a bit confused by the Jenkins log-output:

https://basa.ltl.uni-due.de:33000/job/Core190/7/console

It tries to retrieve the LAB snapshot from the RC repository but for Core it tries to access zoidberg

Downloaded artifact http://basa.ltl.uni-due.de:34001/artifactory/snapshots/org/dkpro/lab/dkpro-lab/0.14.0-SNAPSHOT/maven-metadata.xml
Failed to transfer Could not find metadata org.dkpro.lab:dkpro-lab:0.14.0-SNAPSHOT/maven-metadata.xml in abc (https://oss.sonatype.org/content/repositories/detudarmstadtukpdkpro-1041)
Established TCP socket on 39209
[Core190] $ /usr/lib/jvm/java-8-oracle/bin/java -cp /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-agent-1.12-alpha-1.jar:/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven3/boot/plexus-classworlds-2.5.1.jar:/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven3/conf/logging jenkins.maven3.agent.Maven32Main /var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven3 /var/cache/jenkins/war/WEB-INF/lib/remoting-3.7.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-interceptor-1.12-alpha-1.jar /var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-commons-1.12-alpha-1.jar 39209
<===[JENKINS REMOTING CAPACITY]===>channel started
Executing Maven:  -B -f /var/lib/jenkins/workspace/Core190/pom.xml -U -DfailOnWarning=true -Dmaven.surefire.heap=6g -Dmaven.surefire.argLine=-Duima.check_fs_update_corrupts_index -Dtreetagger.home=/opt/treetagger -Pcheck-license.txt,pmd,findbugs,jacoco,attach-source,attach-javadoc,deps-not-on-maven-central,use-full-resources clean install dependency:analyze-only javadoc:aggregate
[INFO] Scanning for projects...
[INFO] Downloading: http://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-model-releases-local/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom
[INFO] Downloading: http://jcenter.bintray.com/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom
[INFO] Downloading: http://repo.maven.apache.org/maven2/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom

I don't know what the problem is

reckart commented 6 years ago

Maven tries to resolve an artifact from all repositories it has access to. You can configure your Jenkins to roll out a settings.xml file with the builds which contains additional repository declarations for your own local (snapshot) repositories to ensure that Jenkins finds the artifacts you have built and deployed to your own repos. If you do not do that, then Maven will still check in the local .m2 folder for other artifacts. But if say no DKPro Lab build has run for the last few days, then Maven will try to update locally cached SNAPSHOT artifacts from its repos - this is when you run into this kind of problem.

Horsmann commented 6 years ago

I uploaded my own settings.xml to Jenkins and set the file as global settings.xml in the Jenkins project that builds the branch with 1.9.0

In the activated profile it should find the needed dependencies i.e.

<profile>
        <id>ltl</id>
                <repositories>
                <repository>
                        <id>basa-snap</id>
                        <name>basa-snapshots</name>
                        <url>http://basa.ltl.uni-due.de:34001/artifactory/snapshots</url>
                </repository>
                 <repository>
                        <id>abc</id>
                        <name>Core</name>
                        <url>https://oss.sonatype.org/content/repositories/detudarmstadtukpdkpro-1041</url>
                 </repository>
       </repositories>
    </profile>

but the build log says:

[INFO] Scanning for projects...
[INFO] Downloading: http://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-model-releases-local/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom
[INFO] Downloading: http://jcenter.bintray.com/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom
[INFO] Downloading: http://repo.maven.apache.org/maven2/de/tudarmstadt/ukp/dkpro/core/de.tudarmstadt.ukp.dkpro.core-asl/1.9.0/de.tudarmstadt.ukp.dkpro.core-asl-1.9.0.pom
[ERROR] The build could not read 2 projects -> [Help 1]
[ERROR]   
[ERROR]   The project org.dkpro.tc:dkpro-tc-examples:1.0.0-SNAPSHOT (/var/lib/jenkins/workspace/Core190/dkpro-tc-examples/pom.xml) has 7 errors
[ERROR]     Non-resolvable import POM: Could not find artifact de.tudarmstadt.ukp.dkpro.core:de.tudarmstadt.ukp.dkpro.core-asl:pom:1.9.0 in ukp-oss-model-releases (http://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-model-releases-local)
reckart commented 6 years ago

Are you sure the file is recognized and the profile is active? The Jenkins link you provided earlier doesn't work for me.

Horsmann commented 6 years ago

I deleted the branch and removed the Jenkins project.

I am not sure. I saw no log messages which settings.xml is used. I copied my settings.xml into var/lib/jenkins/.m2/ but I couldn't tell if the file was actually loaded. I changed the project of the build-job to use the global one I copied over but it didn't work.

reckart commented 6 years ago

On our Jenkins, I have a settings.xml file set up under Manage Jenkins -> Managed files and then in the builds under Build -> Advanced -> Settings file -> provided settings.xml -> Provided settings, I select this file to be used.

In the logs, I can see this file being used, e.g.


...
Commit message: "Merge branch '1.9.x'"
 > git rev-list 5abe6900d7b80e01eb6eef78da0380b59356fbb0 # timeout=10
Parsing POMs
using settings config with name UKP OSS settings.xml
Replacing all maven server entries not found in credentials list is true
...
<===[JENKINS REMOTING CAPACITY]===>channel started
using settings config with name UKP OSS settings.xml
Replacing all maven server entries not found in credentials list is true
Executing Maven:  -B -f /home/svc_jenkins/workspace/DKPro Core (GitHub)/pom.xml -s /tmp/settings2445850588410097386.xml -U -DfailOnWarning=true -Dmaven.surefire.heap=6g ...
...
Horsmann commented 6 years ago

I think we are getting close to a 1.0.0 release. The windows-bugfix is the last blocking item.

Horsmann commented 6 years ago

@zesch is it still necessary to keep this project: dkpro-tc-features-spelling ? Isn't this moving to the other LREC submission anyway? Can I delete it? This project has an extremely specific use-case. This belongs imo in edu-user-space.

zesch commented 6 years ago

This package can be safely dropped from future releases.

2018-02-08 17:08 GMT+01:00 Tobias Horsmann notifications@github.com:

@zesch https://github.com/zesch is it still necessary to keep this project: dkpro-tc-features-spelling ? Isn't this moving to the other LREC submission anyway? Can I delete it? This project has an extremely specific use-case. This belongs imo in edu-user-space.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dkpro/dkpro-tc/issues/421#issuecomment-364160402, or mute the thread https://github.com/notifications/unsubscribe-auth/ACkQ4GC8TouxiZ5elEyT--PytRpzysrAks5tSxvwgaJpZM4Q6uJn .

Horsmann commented 6 years ago

@reckart

We have a TC 1.0.0 RC candidate available at 

https://oss.sonatype.org/content/repositories/orgdkpro-1036

Please test :)

Horsmann commented 6 years ago

Found a bug. I drop RC.

Horsmann commented 6 years ago

New RC

https://oss.sonatype.org/content/repositories/orgdkpro-1037

@reckart Is it realistic for you to take a look on the RC? I would want to release it before LREC...

reckart commented 6 years ago

Unfortunately, before LREC it is quite unrealistic for me to have a look at it.

Horsmann commented 6 years ago

Ok. I will release TC then and rather prepare a new release if serious problems show up.