axkr / symja_android_library

:coffee: Symja - computer algebra language & symbolic math library. A collection of popular algorithms implemented in pure Java.
https://matheclipse.org/
GNU General Public License v3.0
381 stars 85 forks source link

Cleanup maven structure - use semantic versioning - deploy on Maven central? #193

Open axkr opened 4 years ago

axkr commented 4 years ago

How can we move releases to Maven Central?

Do we have to include all dependencies which are not on Maven Central? External Java sources are already refactored into maven artifact: matheclipse-external with this commit a230537 But what about (Snpshot) JARs from other repositories (currently: jas, apfloat,...)? Are there Maven tools which can help?

Do we have to rename these dependencies to don't get in conflict with other JARs?

Are GitHub Packages a better choice?

How to reduce dependencies? For example hipparchus is used but dependencies use commons-math. antlr is used in logicngbut we always use our own parser. commons-csv vs univocity.

How can we keep this structure more "stable" in future releases (semantic versioning):

The current coredependency tree:

org.matheclipse:matheclipse-core:jar:1.0.0-SNAPSHOT
+- org.matheclipse:matheclipse-external:jar:1.0.0-SNAPSHOT:compile
+- org.matheclipse:matheclipse-frontend:jar:1.0.0-SNAPSHOT:compile
+- com.atlassian.commonmark:commonmark:jar:0.15.0:compile
+- com.atlassian.commonmark:commonmark-ext-gfm-tables:jar:0.15.0:compile
+- de.uni-mannheim.rz.krum:jas:jar:2.6.6017:compile
+- org.apfloat:apfloat:jar:1.10.0-SNAPSHOT:compile
+- com.google.guava:failureaccess:jar:1.0.1:compile
+- com.google.guava:guava:jar:29.0-jre:compile
|  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
|  +- org.checkerframework:checker-qual:jar:2.11.1:compile
|  +- com.google.errorprone:error_prone_annotations:jar:2.3.4:compile
|  \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
+- org.apache.logging.log4j:log4j-core:jar:2.13.2:compile
+- org.apache.logging.log4j:log4j-api:jar:2.13.2:compile
+- org.slf4j:slf4j-simple:jar:1.7.25:test
|  \- org.slf4j:slf4j-api:jar:1.7.25:compile
+- org.slf4j:log4j-over-slf4j:jar:1.7.25:compile
+- org.hipparchus:hipparchus-core:jar:1.7:compile
+- org.hipparchus:hipparchus-clustering:jar:1.7:compile
+- org.hipparchus:hipparchus-fft:jar:1.7:compile
+- org.hipparchus:hipparchus-fitting:jar:1.7:compile
+- org.hipparchus:hipparchus-ode:jar:1.7:compile
+- org.hipparchus:hipparchus-optim:jar:1.7:compile
+- org.hipparchus:hipparchus-stat:jar:1.7:compile
+- commons-codec:commons-codec:jar:1.14:compile
+- org.apache.commons:commons-csv:jar:1.4:compile
+- com.google.code.findbugs:jsr305:jar:3.0.0:compile
+- junit:junit:jar:4.12:test
|  \- org.hamcrest:hamcrest-core:jar:1.3:test
+- com.tngtech.archunit:archunit:jar:0.14.1:test
+- com.tngtech.archunit:archunit-junit4:jar:0.14.1:test
+- org.logicng:logicng:jar:1.6.2:compile
|  \- org.antlr:antlr4-runtime:jar:4.7.1:compile
+- com.itranswarp:compiler:jar:1.0:compile
+- org.jgrapht:jgrapht-core:jar:1.4.0:compile
|  \- org.jheaps:jheaps:jar:0.11:compile
+- org.jgrapht:jgrapht-io:jar:1.4.0:compile
|  \- org.apache.commons:commons-text:jar:1.6:compile
|     \- org.apache.commons:commons-lang3:jar:3.8.1:compile
+- org.gavaghan:geodesy:jar:1.1.3:compile
+- tech.tablesaw:tablesaw-core:jar:0.38.1:compile
|  +- org.apache.commons:commons-math3:jar:3.6.1:compile
|  +- it.unimi.dsi:fastutil:jar:8.3.0:compile
|  +- org.roaringbitmap:RoaringBitmap:jar:0.8.12:compile
|  |  \- org.roaringbitmap:shims:jar:0.8.12:runtime
|  +- com.univocity:univocity-parsers:jar:2.8.4:compile
|  +- com.ibm.icu:icu4j:jar:65.1:compile
|  \- io.github.classgraph:classgraph:jar:4.8.60:compile
+- tech.tablesaw:tablesaw-html:jar:0.38.1:compile
|  \- org.jsoup:jsoup:jar:1.12.1:compile
+- tech.tablesaw:tablesaw-jsplot:jar:0.38.1:compile
|  +- com.fasterxml.jackson.core:jackson-databind:jar:2.10.3:compile
|  |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.10.3:compile
|  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.10.3:compile
|  \- io.pebbletemplates:pebble:jar:3.1.2:compile
|     \- org.unbescape:unbescape:jar:1.1.6.RELEASE:compile
+- tech.tablesaw:tablesaw-json:jar:0.38.1:compile
|  +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.10.3:compile
|  \- com.github.wnameless.json:json-flattener:jar:0.8.1:compile
|     +- com.eclipsesource.minimal-json:minimal-json:jar:0.9.5:compile
|     \- com.github.wnameless.json:json-base:jar:1.0.0:compile
+- tech.tablesaw:tablesaw-excel:jar:0.38.1:compile
|  \- org.apache.poi:poi-ooxml:jar:4.1.1:compile
|     +- org.apache.poi:poi:jar:4.1.1:compile
|     |  \- org.apache.commons:commons-collections4:jar:4.4:compile
|     +- org.apache.poi:poi-ooxml-schemas:jar:4.1.1:compile
|     |  \- org.apache.xmlbeans:xmlbeans:jar:3.1.0:compile
|     +- org.apache.commons:commons-compress:jar:1.19:compile
|     \- com.github.virtuald:curvesapi:jar:1.06:compile
\- org.owasp.encoder:encoder:jar:1.2.2:compile
axkr commented 4 years ago

@rjolly moved script-engine from Maven module matheclipse-core to matheclipse-io.

matheclipse-core should contain only packages which can be used in android development in the future.

rjolly commented 4 years ago

Noted. Looking forward to being able to draw everything from maven central!

axkr commented 3 years ago

Using google-java-format to reformat source codes:

The Eclipse plugin adds a google-java-format formatter implementation that can be configured in Window > Preferences > Java > Code Style > Formatter > Formatter Implementation.

HannesWell commented 3 years ago

I'm also interested in Symja being published on Maven-Central and therefore offer my help in this issue.

But before going for the actual deployment I suggest to improve the following points in the maven structure plus the points you already mentioned in your initial code.

I would like to start with improving and simplifying the structure of the pom.xml files.

In the second step I would improve the dependencies in the following ways.

The jars in the symja_android_library/lib seem to be not necessary for me? The dependencies are defined in Maven's pom.xml. This brings me to the question: Do you use the Eclipse Maven integration m2e? From the many Eclipse files I assume you use Eclipse.

Regarding your questions:

Do we have to include all dependencies which are not on Maven Central? External Java sources are already refactored into maven artifact: matheclipse-external with this commit a230537

You can include any available Maven-style repository in the <repositories> of a pom. So the answer is: no, you don't have to include dependencies not on maven-central, as long as they are available in any Maven-style repository. Since you can specify a version for a dependency you can also stick with an old version without any problem. EDIT: While it is possible to refer to dependencies not on Maven-Central it is not recommended.

But what about (Snpshot) JARs from other repositories (currently: jas, apfloat,...)? Are there Maven tools which can help?

For Snapshots it's basically the same, they are also not a problem. You just have to specify the desired snapshot version (usually with suffix -SNAPSHOT) and have to make sure that the corresponding snapshot repositories are enabled as well. EDIT: I was wrong here. Artifacts that are deployed to Maven Central must not contain SNAPSHOT dependencies. The Maven-Enforcer-Plugin offers a rule to enforce this.

How to reduce dependencies? For example hipparchus is used but dependencies use commons-math. antlr is used in logicngbut we always use our own parser. commons-csv vs univocity.

If certain transitive dependencies are not used by Symja, respectively the code paths that require them, they it can be excluded trough the pom. If dependencies use a similar library (like with hipparchus and commons-math) I fear that it is not easily possible to replace one by the other without porting the code, since the namespaces are different. The only way I see would be to ask the dependency to update from commons-math to hipparchus as well.

As third step I would work towards publishing symja to maven-central.

So if you agree I would start with the points mentioned first and open

axkr commented 3 years ago

Yes please go ahead.

I'm using the JARs in the symja_android_library/lib for faster testing different versions to see the differences and to use a similar structure as in the releases.

If possible the releases should also be generated:

HannesWell commented 3 years ago

@axkr thanks for merging the PR. I've added a brief section to the developer wiki, on how to setup the IDE and how to import the Symja sources from git. https://github.com/axkr/symja_android_library/wiki/Developer-FAQ#how-can-i-prepare-an-ide-to-work-on-symja-source-code

HannesWell commented 3 years ago

As @rjolly pointed out in the discord-chat and one can deduce from this Maven-FAQ (https://maven.apache.org/repository/guide-central-repository-upload.html#faq-and-common-mistakes) patched dependencies can be (re-)published by using your groupid. So for symja we don't have to concern to much about embedded dependencies, they can be simply published using the <groupid>org.matheclipse</groupid>. Furthermore I think we should move embedded dependencies into their own Maven module. I'am about to open a separate issue, to discuss this suggestion.

HannesWell commented 3 years ago

After I have read through the guide at https://central.sonatype.org/publish/ and created the following list of tasks from it in order to prepare the publication of Symja to Maven Central:

Furthermore I think we should automate the deployment process using GitHub actions. So as a first step I can

If this is operational the release deployment should be simple. Furthermore I think we should

Before we perform the actual release I'm in favour to resolve the following issues:

@axkr I can take core of most of the listed tasks. If not yet done you should take care of calming the org.matheclipse namespace and requesting access to the Sonatype OSSRH repo and of generating and distributing the signing keys. Furthermore the access credentials and signing keys have to be uploaded into this GitHub repos secret store in order to automate the deployment via GitHub actions. I can write you the details as soon as I have prepared the workflow (I will use my clone of Symja in order to test it) or maybe we have a video call where we can also review/discuss details if necessary.

And just that you don't wonder: I'm on a one week vacation next week so I cannot continue then.

axkr commented 3 years ago

Claim the org.matheclipse namespace at Sonatype and request access to the repository

* @axkr since you already deploy snapshots manually from time to time to https://oss.sonatype.org/content/repositories/snapshots/org/matheclipse/ I assume you have done that already?

I commented on the old ticket:

HannesWell commented 2 years ago

From now on snapshot builds of Symja are published to the OSSRH on each push to the master branch. The deployed artefacts have sources, javadoc and are gpg signed.

One can get the latest Symja master snapshots with the following coordinates (use the corresponding child-module you need):

<dependency>
  <groupId>org.matheclipse</groupId>
  <artifactId>matheclipse-core</artifactId>
  <version>2.0.0-SNAPSHOT</version>
</dependency>