search

OSGeo / PROJ-JNI

Java Native Interface for PROJ
https://osgeo.github.io/PROJ-JNI/
MIT License
23 stars 15 forks source link

Submit as an OSGeo project? #33

Closed desruisseaux closed 3 years ago

desruisseaux commented 4 years ago

Do we want to submit those JNI bindings as an OSGeo project? If yes, what would be the steps for that?

kbevers commented 4 years ago

Becoming an OSGeo project is quite an elaborate process: https://www.osgeo.org/about/committees/incubation/incubation-process/. Currently it would be impossible. Becoming an OSGeo community project is probably possible but there is also some way to go for that to happen: https://wiki.osgeo.org/wiki/OSGeo_Community_Projects

desruisseaux commented 4 years ago

Community project is fine. Or alternatively is there a process or repository for extensions or plugins to an OSGeo project? Is it a question that could be asked on the PROJ mailing list?

kbevers commented 4 years ago

Formally I don't think it would be a problem to keep PROJ-JNI under the wings of the PROJ project. That would just be another product maintained by the project, similarly to PROJ-data. The main issue is whether the PROJ project is willing to take responsibility for this code or not. None of the current PROJ core committers are interested in (or able to) maintain Java code so a couple of committed volunteers taking on that job is definitely required. The fear is of course that we end up with more or less abandoned code as happened with the JNI bindings that was bundled in the PROJ repo.

Discussing this on the PROJ mailing list is a good start. At some point a formal RFC would have to be written detailing how PROJ-JNI would be handled by the project. But start by raising the topic and get a feel for the interest of the community.

desruisseaux commented 4 years ago

I can volunteer for maintaining the JNI-binding. I didn't before because the fact that it didn't implemented GeoAPI made it less usable to me, to the point that I was maintaining another binding of Proj.4. With this binding under PROJ wing I would delete the other binding, so my focus would be on the PROJ one only.

kbevers commented 4 years ago

That sounds good. Ideally we would have a couple of people involved with PROJ-JNI so that it isn't too dependant on you. Do you know anyone that would be interested in pitching in from time to time?

desruisseaux commented 4 years ago

It is a little bit hard to said. The very first JNI bindings were done by Andrea Antonello from Grass community, but it was many years ago. Even Rouault has already provided a patch for a bug in the C part; while he obviously has much other work to do, he may be able to provide at least some guidance. FilipGh has provided good patches for current bindings. I do not know at which extent those peoples who be interrested to help, but my hope is that with the visibility gained under PROJ wing more volunteers would appear.

kbevers commented 4 years ago

but my hope is that with the visibility gained under PROJ wing more volunteers would appear.

I agree with this last part. Hopefully it is possible to gain a bit of traction with more exposure. Let's see what the rest of the PROJ folks think about this.

desruisseaux commented 4 years ago

It seems that the only possible approach is OSGeo community project. Do we have a volunteer for doing a submission? I would stay around as a volunteer for development work.

willcohen commented 4 years ago

I'm interested in being helpful here as well however is useful, especially since the process of keeping proj4j up-to-date with upstream seems increasingly difficult given the rapid pace of change happening within PROJ itself. The major benefit to proj4j's approach is that it never needs native code and allows for much more flexibility, but now that proj requires the use of sqlite which isn't available as a pure java implementation, to stay current, proj4j would need to reimplement the entire database in a pure-java db type on top of also reimplementing all of proj's reworked internals. It seems that rather than requiring all of that work, a JNI approach might be a nice complementary effort that's a little less portable but much easier to keep current.

I do have one additional question, however, which might want to be solved before this gets submitted as a formal project under any umbrella: it seems increasingly common (and useful!) for jvm libraries embedding native code to include cross-platform compiled libraries inside (a la sqlite). The downside would be a larger jar, but the upside would be the ability for users to just include this as a dependency and not need anything beyond the JVM to do projections, which makes this dramatically more streamlined for users. Would a similar path be useful here? If size is a concern, there might be a way of splitting out the native library loader into a different dependency and keeping the overall bindings here, though I've never seen that done in practice. Having something released and available on Maven which embeds native PROJ would be a dramatic development!

kbevers commented 4 years ago

I'm interested in being helpful here as well however is useful

Any help is welcome! I'd be extremely happy if more people from the Java geospatial world would rally behind this library so it can gain some traction.

Would a similar path be useful here?

To me it sounds like a good approach. Mind you, I am not a Java expert in any way so my word doesn't carry much weight here. @desruisseaux probably have a better feeling for this. Regarding the size of the jar, I don't think that is going to be a problem. PROJ itself isn't that big, even including the dependencies (libtiff, sqlite). And compared to the ~1GB of grid files users are likely to install as well it is negligible.

desruisseaux commented 4 years ago

Hello @willcohen and thanks! Indeed help would be very welcome. We need a volunteer for all the "administrative" parts with OSGeo. If you can help in getting in touch with the right persons at OSGeo, setting up a project of the right category (it seems it would be a "community project") and transferring the GitHub repository, that would be very welcome! I can volunteer for maintenance of Java and C++ code, but would like the "project management" parts to be done by someone else. I don't know if OSGeo community projects have a chair, but if so and if you can volunteer for being that chair, that would be great!

About inclusion of native libraries in the JAR file, I do not have a strong opinion. I guess we would need to do more experiments. Current version already embeds native code, but only the JNI bindings (not the PROJ library itself) and only for the platform on which PROJ-JNI has been built.The difficulties that I see are:

Some of above-cited problems can be mitigated: we can try to copy PROJ in a more permanent location than temporary files (but where?) We can avoid providing native library for all platforms and instead creates a separated JAR file for each platform. JavaFX does that, but it requires non-trivial Maven configuration if we want it to download the right JAR file automatically (JavaFX does that with a Maven plugin that they developed themselves).

A possible approach could be to migrate the project to OSGeo first, then create a few branches for experimenting different ways of linking to PROJ library.

willcohen commented 4 years ago

Great. Happy to get started on this. I'd like to take a little time first to get a little more familiar with the internals of the code base first. A first prerequisite would be double-checking source code headers, contributing guidelines, code of conduct, but also to make sure I can more easily answer the basic questions about how this specifically fits into the ecosystem versus competing efforts, to make the application as streamlined as possible.

desruisseaux commented 4 years ago

For the internal, I recommend to read the following document first. This is basically the same document as ISO 19111 standard, which was developed conjointly by OGC and ISO:

OGC Abstract Specification Topic 2: Referencing by coordinates

The huge improvement of PROJ 6 compared to PROJ 4 is its alignment on that OGC/ISO standard. PROJ 4 (and consequently Proj4J) have limitations inherited from initial PROJ design as a map projection library, not a generic "referencing by coordinates" library. I described some problems here. Those problems are fixed in PROJ 6, but requires major changes in the way of thinking the library. The above OGC/ISO standard can be seen as a blueprint describing what should be the library objects and their relationship. It is much more powerful than PROJ 4 model.

I suggest to forget PROJ 4 API. If looking at PROJ 6 C++ API, we can see a close to one-to-one mapping between PROJ C++ objects and ISO 19111 objects. Peoples familiar with ISO 19111 should understand PROJ C++ API easily. Likewise the same ISO 19111 objects have been translated to Java interfaces in the GeoAPI project. We can also see a close to one-to-one mapping between GeoAPI and PROJ 6 C++ API, which is a natural consequence of the fact that both APIs are derived from the same ISO standard. So again, peoples familiar with ISO 19111 should understand GeoAPI (and consequently this PROJ-JNI project) easily.

willcohen commented 4 years ago

Hi there, I've had a little more time to try using this as a prototype in a few projects and I think I have a better handle on the codebase.

In terms of getting this submitted, based off of their community projects page:

OSGeo Community Projects

Once all these boxes are checked, I think it's as simple as me sending an email to the incubation list, and there's lots of examples from previous months.

If you'd all like, I can go ahead and start a pull request over the next few days for a CONTRIBUTIONS.md and code of conduct (many projects pick the Contributor Covenant, does that one look good to you both?). After that I can double check the headers and I think that's that!

Only last question is if you all want to pick another name or stick with org.kortforsynigen/proj as the project coordinates! The most ideal candidate for maven central, I suppose, would be org.proj/proj, but I'm not sure who controls that domain (whether that's the PROJ project or OSGeo itself).

kbevers commented 4 years ago

I think it's as simple as me sending an email to the incubation list,

Yep, it's as easy as that!

If you'd all like, I can go ahead and start a pull request over the next few days for a CONTRIBUTIONS.md and code of conduct (many projects pick the Contributor Covenant, does that one look good to you both?)

No objection from me, your effort is much appreciated! The Contributor Covenant is fine and is usually the recommended CoC by OSGeo. The PROJ project uses it as well.

Only last question is if you all want to pick another name or stick with org.kortforsynigen/proj as the project coordinates! The most ideal candidate for maven central, I suppose, would be org.proj/proj, but I'm not sure who controls that domain (whether that's the PROJ project or OSGeo itself).

Another name would be ideal. That's one of the reasons behind becomming an OSGeo community project, as that would allow os to use org.osgeo/proj, although org.proj/proj would be fine as well I think. proj.org is owned by OSGeo and managed by the OSGeo system administrators team, but the PROJ project of course can use it however it likes.

desruisseaux commented 4 years ago

Hello willcohen, thanks for looking at that! A CONTRIBUTIONS.md file and Contributor Convenant code of conduct seem fine to me. For project coordinates, I suggests org.osgeo/proj. It would match the "organization/project" pattern and I think there is benefit in making visible that the project is part of OSGeo. Those coordinates would apply to both Maven coordinates and package names, which would need to be renamed. I presume it is sufficient to said that we intent to use that name and wait for OSGeo acceptance before doing the renaming.

desruisseaux commented 3 years ago

ASF, OGC and OSGeo will hold a joint code sprint on February 17th to 19th. It could be an opportunity to advance on the topic of bringing PROJ-JNI to OSGeo. I proposed this topic among others on https://github.com/opengeospatial/joint-ogc-osgeo-asf-sprint-2021/issues/2.

willcohen commented 3 years ago

Many apologies for missing this Github notification and for dropping the ball on the incubation -- 2020 didn't disappoint with things to distract! I'll get the CoC and contributing in a PR this week and can draft an email. Happy to wait until after the sprint or before to send something to the list.

desruisseaux commented 3 years ago

Thanks for the update Will Cohen and no problem, I also got delays. What about advancing on this topic during the code sprint, if you have time? If you want, you can register to the code sprint (it is free and open to everyone), and connect next week to the GeoAPI, PROJ or OSGeo generic channel (I do not know which one would be most appropriate). Maybe we could take the opportunity of having OSGeo peoples there for getting this task done during those 3 days.

desruisseaux commented 3 years ago

Hello all. The GeoAPI channel is open from today to Friday inclusive for discussion: https://meet.jit.si/GeoAPI

It is okay if you can not connect this week. Continuing on this GitHub issue next week or later is fine as well.

If anyone is curious about what are the other channels, the complete list is here.

desruisseaux commented 3 years ago

For information, I deleted the geoapi-proj4 module in the GeoAPI source code repository and updated documentation for redirection to this project. I will update those links later when this project move.

willcohen commented 3 years ago

35, I think works as a baseline. Assuming it looks okay, I can send the email to the incubation committee this week and finally get this going.

kbevers commented 3 years ago

Assuming it looks okay, I can send the email to the incubation committee this week and finally get this going.

Looks good! Please go ahead with the incubation submission :)

willcohen commented 3 years ago

https://lists.osgeo.org/pipermail/incubator/2021-May/004328.html

desruisseaux commented 3 years ago

Thanks!

jodygarnett commented 3 years ago

This is moving along - project page is up:

I provided some feedback on the incubation list (mostly thinking through a CONTRIBUTING.md file)

desruisseaux commented 3 years ago

Should this github repository be transferred to https://github.com/OSGeo/ ?

jodygarnett commented 3 years ago

There is no need to transfer GitHub repository, but you are welcome to take that step once accepted into the community program.

OSGeo is very relaxed, really just wanting an opportunity to help on the open source side of things, and promote where we can.

Do keep in mind that OSGeo learns over time, so things like having a code of conduct are new, and in the future perhaps a security "responsible disclosure" policy, etc...

willcohen commented 3 years ago

@jodygarnett sent a thoughtful email to the incubation list as a follow-up with some decisions to make. @desruisseaux and @kbevers do you all have any thoughts on the following issues? (I'm putting my thoughts alongside Jody's text.)

This first thing I see adding a contribution policy to CONTRIBUTING.md file (so it is shown to folks making PRs).

  • You are really looking to remind folks that the results are going to be open source and they (and their employer) should be comfortable with that).
  • I also want to check you have considered the (c) of the resulting file? Do you ask for a CLA to Agency for Data Supply and Efficiency? Do you update the header to say "Copyright © 2021 Agency for Data Supply and Efficiency and others" ? If make a new file in my PR is it "Copyright © 2021 Jody Garnett" or "Copyright © 2021 GeoCat Canada" (my employer).

I don't think that administering a CLA is at all worth the effort involved here, especially given the chilling effect it might have on contributions. The easiest approach, I'd say, would be to adapt the approach taken by PROJ's contribution guidelines, which themselves are adapted from PDAL and GDAL. If those guidelines make sense, I can put a draft of that into a PR.

  • I did see your TransformPoints.java example is "public domain" (so not strictly open source, but we can consider it documentation eh?)

Given the issues Even and Jody raised with public domain (and keeping in mind the jurisdictions where it isn't recognized as a concept), perhaps it should be switched to MIT as well, which would still be about as unrestrictive as can be?

kbevers commented 3 years ago

If those guidelines make sense, I can put a draft of that into a PR.

Go ahead please. A CLA is not something we want on this. Approach should be similar to GDAL, PROJ and other OSGeo projects. That works well and will here too!

perhaps it should be switched to MIT as well, which would still be about as unrestrictive as can be?

Yes, that would be the preferred solution from my side. Generally, anything accepted into this repository should be either MIT or CC0 (or possibly CC-BY).

desruisseaux commented 3 years ago

No problem on my side for changing the public domain header to MIT or other permissive licence.

willcohen commented 3 years ago

37 and #38 address these two points, respectively.

desruisseaux commented 3 years ago

Since the project now appears on https://www.osgeo.org/projects/proj-jni/ should we close this issue?

willcohen commented 3 years ago

I think we need to wait for the motion (https://lists.osgeo.org/pipermail/incubator/2021-June/thread.html) to be completed and (ideally) approved. The voting close date was set for yesterday, but I don't think anything's officially happened yet.

desruisseaux commented 3 years ago

Ah sorry, I didn't knew that I vote was in progress. I let this issue in your hands then.

jodygarnett commented 3 years ago

Thanks for the reminder, I will go check the email list. If the proposal is successful I will ask the board to approve the project (they have a meeting at month end).

kbevers commented 3 years ago

Accepted into the OSGeo community program