Deploy udpipe.jar to Maven Central

[reckart]

It would be great if the Java part of the UDPipe bindings would be available on Maven Central.

It would be even more great if the native libraries would also be available as JARs, e.g. by building and packaging them using JavaCPP? I've seen it being used e.g. in DL4J - not used it myself through. Seems like a nice way of packaging up natively compiled libraries along with their Java interfaces in JARs and to distribute them via Maven.

[foxik]

It would be nice-to-have, but unfortunately I do not have the resources to do so. You are welcome to help, if you like :-)

[reckart]

For the moment, I have deployed the JAR here: https://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/webapp/browserepo.html?4&pathId=public-ext-releases-local:cz/cuni/mff/ufal/udpipe/udpipe/1.0.0/udpipe-1.0.0.jar

But yes - we can probably in time help out deploying this to Maven Central as third-party upload.

[foxik]

Closing -- I do not have resources to do this. Anybody is welcome to help :-)

[betoboullosa]

Hi, @foxik , I can upload the updated JAR from @reckart to Maven Central. But it would be good if there was a new release before that, because issue #10 is still not fixed in the current release. Do you think you could make the new release?

[foxik]

The new release will be ready on March 15, so please wait until then.

BTW, how does it work if the JAR needs a C++ native library? You also upload builds for all possible systems/architectures (we have 5 such variants in the binary builds)? Or you distribute the sources and it gets somehow built (which is how it works for PyPI and CPAN packages at the moment)?

Reopening.

[reckart]

What we have done for the present release is creating a JAR that contains all the different binaries. You can have a look at it here.

[foxik]

It sounds I should just learn how to do this (there is more than 1 person wanting this now :-).

After the 1.1 release, I will try doing this myself and then ask you to check the result...

[reckart]

I'm trying to do a third-party upload of UDPipe 1.1.0 Java Bindings to Maven Central for an upcoming DKPro Core release:

https://issues.sonatype.org/browse/OSSRH-35580

[foxik]

Thanks. I am currently on a 6-month research stay, and am not working on this for the time being.

[reckart]

UDPipe 1.1.0 (the Java JAR) is now available on Maven Central: http://search.maven.org/#search%7Cga%7C1%7Cudpipe

[dedekj]

@reckart The jar you uploaded to the maven central (cz.cuni.mff.ufal.udpipe:udpipe:jar:1.1.0) doesn't contain the native libraries, does it? How could one use such jar without them? I'm really curious because I would love to use udpipe via maven... Maybe you could deploy the native libs directly as they are with different classifiers, e.g. linux64 as:

<dependency>
    <groupId>cz.cuni.mff.ufal.udpipe</groupId>
    <artifactId>udpipe</artifactId>
    <version>1.1.0</version>
    <classifier>linux64</classifier>
    <type>so</type>
</dependency>

[reckart]

For DKPro Core, we have packaged the binaries as

    <dependency>
      <groupId>org.dkpro.core</groupId>
      <artifactId>dkpro-core-udpipe-bin</artifactId>
      <version>20170425.0</version>
    </dependency>

and distribute them via

      <repository>
          <id>ukp-oss-model-releases</id>
          <url>http://zoidberg.ukp.informatik.tu-darmstadt.de/artifactory/public-model-releases-local</url>
          <snapshots>
              <enabled>false</enabled>
          </snapshots>
      </repository>

In order to initialize the UDPipe native binaries at runtime, we use the following code:

https://github.com/dkpro/dkpro-core/blob/master/dkpro-core-udpipe-asl/src/main/java/org/dkpro/core/udpipe/internal/UDPipeUtils.java

The script that we use to package them is here:

https://github.com/dkpro/dkpro-core/blob/master/dkpro-core-udpipe-asl/src/scripts/build.xml

[dedekj]

Nice @reckart , but why do you not deploy the dkpro-core-udpipe-bin archive to the maven central as well?

[reckart]

None of the models or binaries we package in DKPro Core are on Maven Central. If need be, we have more control over them on our own repo. We prefer pushing only our own artifacts to Maven Central and not re-packaged artifacts from third parties.

I believe that @foxik was considering to look into better approaches of packaging UDPipe libraries. As mentioned in the initial comment of this issue, e.g. DL4J seems to have put more thought into packaging native binaries as Maven artifacts. There is probably something to learn from their approach.

[jgrivolla]

Hi all, we are currently looking into updating the UDPipe integration to the latest release (1.2.0) and models (Universal Dependencies 2.5 Models for UDPipe (2019-12-06) from https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-3131 or newer if possible).

@foxik : it would be really good to have some support from your side, at least with regard to having the UDPipe artifacts available through Maven.

[foxik]

I have to say I do not work much with Java, so I really do not know how to do it, and to be honest, I am not planning to do it in the nearest future. I am currently fully occupied by online teaching (all universities went to online here), and if I find some time, I will rather spend it on UDPipe 2 release (we already have the models in the online service, but not available for official download) or on developing UDPipe 3.

You can build the UDPipe packages by yourself (bindings/java), but that is currently and in near future all Java support in UDPipe.

[reckart]

Hi @jgrivolla, hi @foxik :) I have uploaded the UDpipe version to Maven Central that we currently have there. @jgrivolla maybe you could pick up the baton and publish an update.

Here is the original OSSRH ticket that I used to get the current version published: https://issues.sonatype.org/browse/OSSRH-35580

Here are some guidelines on how to upload stuff to Maven Central: https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+The+Central+Repository

Here for manual deployment: https://central.sonatype.org/pages/manual-staging-bundle-creation-and-deployment.html

[jgrivolla]

Hi @reckart, @foxik, it seems that I have it working (with all artifacts in our local maven repo so far), and I will see how to do the 3rd party upload.

However, I get some test failures in DKpro. At first sight it looks like they might be reasonable changes due to using a different models (such as some words changing from PROPN to NOUN). However, I'm not sure whether to simply change the tests to accept the current results as "correct". @reckart: Should I simply modify the tests so they pass? And @foxik: it might be useful if you could check the current output to see if it is as expected.

And then, @reckart, we should probably continue in an issue on the DKpro tracker for the pull request to integrate the update, etc.

[reckart]

The tests in DKPro Core should be seen more as sanity checks than as quality checks. The do not really test the quality of a model. However, they try to alert of e.g. changes in the tagset used by the model which might require updates to the tag-mapping-tables in DKPro Core or to the model metadata on the DKPro Core side. That said, even with such basic checks, we did at times discover bugs or unwanted changes in upstream models and tools and reporting them back to their authors helped improving the upstream tools.

Unfortunately, DKPro Core so far doesn't come with any easy tooling for checking a pre-trained model against a reference corpus. If you have doubts about the output you are getting, maybe paste it here so @foxik can get a look without having to build DKPro Core.

[foxik]

Closing, as we do not have manpower to upload Maven artifacts; also future development has moved from UDPipe to LinPipe.