Backport to Java 7 (used to include, "through RxJava")

[pakoito]

I would like to use this library in Java 6.5 on Android devices, but it's only ready for Java 8 usage.

The replacement consists on swapping the java.lang.func imports for rx.functions ones, moving the interfaces into abstract classes, and refactoring lambdas and method references into anonymous classes.

For the Either types I already have my own implementation based off Scala's you can leverage: https://github.com/pakoito/JavaSealedUnions and https://github.com/pakoito/RxSealedUnions

For Tuples I've been using JavaTuples: https://github.com/pakoito/RxTuples

In case this proves too much work, would you be open to accept a PR, branch it on the same repository, or have a new separate backport project?

[GlenKPeterson]

I wrote a lot of this in Java 7, and it could certainly be back-ported to that as-is. I'd even be willing to do some work towards that, though I'm not sure how much. As to your other suggested changes, each one seems like a significant decision to me, so I'd like us to both understand what justifies each choice. To that end:

Why rx.functions instead of the exception-wrapping functions already in this project?

Why JavaSealedUnions/RxSealedUnions instead of OneOf2? And would you use the Java or Rx version?

Why JavaTuples instead of the Tuples in this project? I like extending UncleJim tuples to quickly develop lightweight, immutable classes. The JavaTuples project makes all their tuples serializable, which you can't undo in a sub-class. Also, implementing Comparable creates bugs whenever you subclass. Both serializable and Comparable seem like anti-features for tuples.

Both the Apache and Eclipse licenses allow you to branch as much as you like so long as the license and attribution are maintained. Please read the license details carefully (see both here and here). I would consider allowing the Apache-licensed parts to be released under the Eclipse license if that would make a big difference for you. The Eclipse-licensed parts come from Clojure, so that license can't be changed without completely rewriting them.

[pakoito]

I wrote a lot of this in Java 7, and it could certainly be back-ported to that as-is. I'd even be willing to do some work towards that, though I'm not sure how much. As to your other suggested changes, each one seems like a significant decision to me, so I'd like us to both understand what justifies each choice.

Let's do this :D

Why rx.functions instead of the exception-wrapping functions already in this project?

Just for the extends clause on Func0, Func1 and Func2 changing from Supplier, Function and Bifunction from Java 8. RxJava comes with up to Func9, so those have to be extended instead.

Why JavaSealedUnions/RxSealedUnions instead of OneOf2?

The encoding of Either (union of 2 elements) is generalized for unions from 1 up to 9 elements, while keeping the same join/match method under a different name.

And would you use the Java or Rx version?

The Java version is for Java 8, so I'd have to use the Rx version.

Why JavaTuples instead of the Tuples in this project?

I didn't want duplication but the points you mention are quite valid. I don't feel strongly for the library, it was more about convenience.

Licensing

My concerns were not about licenses, but which maintenance option suits you personally better. Some people are protective of their project and feel strongly about multiple versions of a project inside their repositories, or modifying the existing library to add backwards support. In those cases I branch out a backport with the correct attributions and licenses.

I'm happy with either option, as I'm doing this because I'd like the collections to work on Android for professional and personal use.

[GlenKPeterson]

Just as there is a tuple generator in this project, there could be a FunctionGenerator and UnionGenerator. I just haven't needed more than 2 arguments, or 2 options, so it hasn't been a priority.

UncleJim's Function0 extends Supplier and Callable, Function1 extends Function and Consumer. Function2 extends BiFunction. So that shouldn't be an issue. UncleJim's functions also wrap exceptions which has really helped me keep my code consistent - that would be hard to give up. UncleJim's functions have memoization and boolean utilities in as well.

I definitely need more time to look at https://github.com/eleventigers/rxeither and https://github.com/pakoito/RxSealedUnions

There is a lot of functionality there! Nothing in UncleJim relies on OneOf2 and it's only 100 lines of code, so having it around shouldn't weigh you down or hurt anything. Can you just ignore UncleJim's OneOf2 and still use RxJava's sealed unions and UncleJim together?

[pakoito]

RxEither uses RxSealedUnion as the backend and expands on Union2 to support the Observable monad in RxJava :D All cleanup/merge changes can be deferred once the project works on Android.

The only concern left is the governance, do you mind if I just start a new backport repository and source GlenKPeterson/UncleJim as the original Java 8 version?

[GlenKPeterson]

I think back-porting is a really good idea. I developed the Function interfaces and early versions of Transformable in Java 7 and it was definitely useful there. It's better in Java 8, but some people can't use Java 8 for various reasons.

If you'd rather start a separate back-port repository, I don't mind. I hope you don't mind if I re-import it into this project at some point! I'm also willing to link to your back-port if you're willing to link to UncleJim.

If you are cool with what we discussed above, then I am willing to give you a "Java 7" (or whatever you want to call it) git-branch of this project. At first, I'll review your edits, and post them to this project in a timely manner (generally one weekend for major changes, one day for minor ones). As I get to know and trust you better, I'll give you commit access with the understanding that you will focus on your branch. I'll expect a "minimal diff" approach (that will be easier for all of us), but I'm willing to clean up messes in my code as you point them out and maybe adapt my style to make your work easier.

I don't know what would work better, but I thought I'd offer that. I'm interested to see what you produce!

My priorities are

1. Bug fixes in the primary branch
2. Meaningful enhancements in the primary branch

When I'm not working on 1 or 2 above, I'm willing to do some work on the back-port branch or answer questions, or whatever helps you most. Keep me in the loop!

Feel free to contact me with questions/discussion at my first name (just one n) at organicdesign dot org or on gittr

[pakoito]

^ I'm going to work it this weekend, I've just been out today. Not sure about the branching strategy yet.

[pakoito]

Progress so far: everything has been migrated to Java 7 except for the default methods on the interface hierarchy, and the tests. https://github.com/pakoito/UncleJim/commits/master

Fixing the mixins is a headscratcher because some of them are on tens of places and they can't be just copypasted. tl;dr deep inheritance is bad. I'd like your input on how to sort it out.

[GlenKPeterson]

Default methods on interfaces is one of my favorite features of Java 8. They might even be more useful than lambdas! That said, I can see how they would be a thorn in your side. I think my second best idea is to use static methods on helper classes:

public interface UnmodSortedMap<K,V> extends UnmodMap<K,V>, SortedMap<K,V>, UnmodSortedIterable<UnmodMap.UnEntry<K,V>> {

    /**
     ...
     <b>NOTE:</b> Implementers should use the implementation in UnmodSortedMapImplementations
     */
    @Override UnmodSortedSet<Entry<K,V>> entrySet();

public class UnmodSortedMapImplementations {
    private UnmodSortedMapImplementations() {
        throw new UnsupportedOperationException("No instantiation.  Just a holder for static methods.");
    }

    /**
     Returns a view of the mappings contained in this map.  The set should actually contain
     UnmodMap.Entry items, but that return signature is illegal in Java, so you'll just have to
     remember.
     */
    public static <K,V> UnmodSortedSet<Map.Entry<K,V>> entrySet(final UnmodSortedMap<K,V> parentMap) {
        return new UnmodSortedSet<Map.Entry<K,V>>() {
            @Override public int size() { return parentMap.size(); }

            @SuppressWarnings("unchecked")
            @Override public boolean contains(Object o) {
                if ( !(o instanceof Map.Entry) ) { return false; }

public abstract class ImSortedMap<K,V> implements UnmodSortedMap<K,V> {

    @Override
    public UnmodSortedSet<Entry<K,V>> entrySet() {
        return UnmodSortedMapImplementations.entrySet(this);
    }

I started my new project at work this week using Kotlin. I've only used it for 2 days, but so far it looks amazing.

"Kotlin generates bytecode which is compatible with Java 6 or newer. This ensures that Kotlin can be used in environments such as Android, where Java 6 is the latest supported version."

They have shaved off all the backwards-compatibility warts of Java (like primitives and arrays) and added a few of the safer parts of Scala. About the only syntax I really even had to look up was the lambda syntax which is different from either Java or Scala, but certainly no worse. I'm a little cautious about jumping on this bandwagon due to my lack of experience with it, but I think my best suggestion is to rewrite using Kotlin.

Instead of converting UncleJim to Java 7 or Java 6, we could convert the whole thing to Kotlin and compile it to Java 6 from there. It might still get funky with the function inheritance when Java 6 doesn't have those interfaces. But maybe Kotlin has already thought of that? Even if they haven't, fixing those is maybe 5% of the effort of what you're looking at now.

I'm going to work in Kotlin for a few more days this week before making a decision, but maybe if you try it out and I think about it a little more, we can circle back and discuss how well that direction might work for both of us?

If you are going to continue a separate back-port, I'd appreciate it if you would change the artifactId (or similar) in pom.xml and on the README.md pom-snippet so there is no possible confusion between the two versions of the project if you intentionally or accidentally publish yours (or someone else forks and publishes it).

[pakoito]

Because the intention is to get it working on Android without any external dependencies I can't use Kotlin, as IIRC it requires the compiler or linking the runtime.

Given the changes introduced and the differences between the branches I suspect that maintenance would be too big of a problem past the first version. I'll try and finish that one first, then decide what to do going forward.

[GlenKPeterson]

Hmm... So you compile your Android code on an Android device? We should look into the runtime. If it's small, I would think Android devs would be psyched to have lambda syntax and mix-in interfaces.

I put a link to your project from the home page of this one.

[pakoito]

Migrating to Kotlin is a project-wide choice. In my experience in several different companies, most have a large years-old codebase that would take a hit from adding any large framework or change that magnitude. It's a choice to be heavily considered and the community is migrating, albeit quite slowly.

Thanks for the link!

[GlenKPeterson]

You might want to look at this: http://stackoverflow.com/questions/23318109/is-it-possible-to-use-java-8-for-android-development

What if you compiled UncleJim for Java 7, then de-compiled it to see how the Java compiler manages default methods on interfaces? The Java dev team is pretty serious about backward compatibility. They probably made very good decisions about how to compile the Java 8 features to earlier Java versions. If you can copy them, that's probably a better start than anything I suggested above.

Do you really need Java 7 source? Or just a Java 7 "binary"?

[pakoito]

Android doesn't exactly use the same stack as desktop java, the code compiled with javac has to later be recompiled by a different tool called Dex. Android also has a different JVM called Dalvik/ART depending on the version, that's not binary compatible with the desktop one, nor contains the full JavaSE, causing inconsistencies, and the final reason why Oracle sued Google.

Some Java 8 features (lambdas & method references) are available through a new compiling tool called Jack, but it's still in development and doesn't allow vital tools like code coverage, Transform API and byte weaving yet. Annotation processing was added just weeks ago. To enable the full Java 8 toolset (mixins, statics on interfaces, stream api...) you also need a minSdkVersion of 23, which is this year's release. Most apps today are between 15 and 17, and it takes 1 year for the old devices to be phased out to allow apps to raise the minSdkVersion. So, most of the Java 8 toolset isn't expected to be usable in at least 5 years.

The alternative is Kotlin, which comes with the problems mentioned above plus a few others, like the lack of a good static analyzer. It's still the better choice, and there's even a rumor that after the Oracle vs Google trial it's going to get official google endorsement as the official language.

In the Android dev world most high visibility libraries are shipping with both versions at the same time: java7 and kotlin.

[pakoito]

Sadly I have to drop this port. My biggest regret is letting you down, but I hope you don't think too low of me.

The reason is that the many changes required for the Java 7 version would require separate maintenance from the main branch, which I don't think I can provide mid-term.

Apologies again.

[GlenKPeterson]

I take it as a compliment to this code that you spent any time on it at all! The original name of this project was "FP for Java 7" so it's not unreasonable that a Java 7 version should rise again from the ashes at some point. I'm getting the impression that having such a version would improve adoption, so it's on my wish list to pursue some day. I'll update this ticket if I make it happen.

[GlenKPeterson]

I'm re-opening this issue because it is in my top 3 priorities for development.

1. Bug Fixes
2. RRB-Tree
3. Java 7 version.

This way, others will know it's being considered and can add their +1 or comments and maybe make it a higher priority.

I'm guessing 1-3 weeks for Bug fixes, 1-3 months for the RRB-Tree, then maybe 1 month for Java 7 if nothing gets in the way.

[GlenKPeterson]

Hi. I don't remember if Google had announced that Kotlin would be the new Android development language yet when we had this discussion. I've been using Kotlin increasingly at work and I love it. With a few tweaks, Paguro will work better with Kotlin than it ever did with Java. I've been converting Java code to Kotlin at work by simply cutting and pasting from a Java file to a Kotlin one using IntelliJ and it does a good start at converting the code for me! Java compatibility is near 100% and by that I don't mean one-way compatibility like Scala. Yes, you can call any Java code from Kotlin. But you can also call just about any Kotlin code from Java and not even know you are calling Kotlin code. The result is that I can convert projects one file at a time, which is very attractive!

I haven't seen anyone thumbs-up this discussion (besides me). I did the bug fixes and the RRB Tree, but my next priority is now additional Kotlin compatibility. Java 7 has more-or-less fallen off my radar. I wouldn't be that shocked if this entire project were written in Kotlin a year from now. I'm shooting for all Kotlin unit tests going forward.

I'm sorry you didn't get what you wanted. I thought I should be up front about my changed priorities. I'm closing this issue.

I do still intend to look at RxJava and other react frameworks, because that's great stuff and I'd like to this to be as good for reactive programming as possible. So maybe that's a better mutual goal?

[pakoito]

We're working in a full FP stack for Kotlin with typeclasses, free monad, implicits, optics, and monadic comprehensions over coroutines. It's been working for months now, with all the features you'd expect, and we're finishing the QoL improvements before release. We'd like to add a persistent collection library and didn't have the people/time to do it. If you're interested in joining the project, do tell us.

Here's the page: https://github.com/kategory/kategory We'll launch a documentation site soon, the PR is underway.

[GlenKPeterson]

Awesome! This is why I posted - I didn't know if you were going to respond, "Java 7 or die!" but it turns out you might be more converted to Kotlin than I am at this point!

I'm going to have to take some time to look at that. I think that Kotlin compatibility might come before combining the projects anyway, so I may stay focused on that. Hopefully it will be quick. Then you can always use Paguro as a dependency. Remember that the code here that came from Clojure (classes in the "collections" package starting with the word, "Persistent") is licensed under the Eclipse license which may be awkward for people who want to use it in GPL code: https://github.com/GlenKPeterson/Paguro/blob/master/README2.md#licenses-continued You'd have to keep the Ecllipse license on that code.

That said, the RRB Tree can substitute for the PersistentVector. That and the rest of this project is my original work and/or Apache 2.0 licensed, so you could grab it for your project today! I may write insertion-order versions of the Map and Set in Kotlin which could be different enough to avoid copyright issues with the Clojure collections, but don't hold your breath for that! The RRB Tree took almost a year and a half and it was hard enough that it wasn't always fun. Rich Hickey made all three collections and the Clojure language in 2 years, but I think he didn't have a day job - that's all he did. I can't do that right now. We could ask maybe Rich about converting these versions of his collections to Apache.

Wow. Kategory is several bunches of code! Is there an overview somewhere? A list of motivations? A presentation posted on YouTube? A project manifesto? Where do I start?

[raulraja]

@GlenKPeterson we should have some docs addressing motivation for Kategory and many typeclasses and datatypes documented with ready to go examples in a few weeks. The motivation of Kategory was porting most of the concepts we use daily in Scala programming from cats https://typelevel.org/cats/ to a lang that does not have implicits or higher kinded types but could be somehow emulated. Kategory enables typed FP in Kotlin with the traditional typeclasses and techniques you find in other langs like Haskell or Scala. Through it's HKs emulation it provides a way to write fully polymorphic code that can abstract away datatypes in favor of coding to behaviors as expressed by typeclasses more or less similar to what you can accomplish with Free/Tagless final encodings.

[pakoito]

@GlenKPeterson Raul is one of the maintainers, I hope his explanation clarified what our intentions are. We're very interested in bringing these libraries as part of a kategory subproject. We estimate that the overhead for adding idiomatic Kotlin would be focused on requirements like extending from kotlin.collections and support higher order functions from the standard library. Although we could just lift them from the project today, we'd rather have something that's more fully featured. I'd like to extend an invitation for you to help us make the FP gold standard in Kotlin!

[GlenKPeterson]

@raulraja and @pakoito - Thank you very much for your kind invitation. I don't mean to ignore you. I've just been unsure how to respond and swamped at work. I also need to read: https://github.com/kategory/kategory/tree/master/kategory-docs/docs/typeclasses/functor and https://github.com/kategory/kategory/tree/master/kategory-docs/docs/typeclasses/applicative To better understand what you are doing.

One thing we might do is make an inventory of what each project provides and come up one or more plans for eliminating duplicates. Maybe after September 23rd, the three of us could set up a google hangout / mumble conversation / whatever, to talk about what you're envisioning and how I could help, or how these two projects could fit together?

There is currently namespace collision between the Kotlin standard library and Paguro's classes MutableMap, MutableSet, and MutableList. This makes using Paguro in Kotlin occasionally nightmarish. Kotlin's functions already take care of checked exceptions, so the Paguro functions package may just disappear, or be a single class of singletons or special cases of functions. Similarly, Kotlin's data classes are generally better than tuples, so that Paguro package may change or disappear. When I'm done fitting Paguro into Kotlin, it might be just what you need to plug into your project as a dependency? Or vice-versa.

If you can explain to me how something in Paguro is unfriendly to what I'm going to call "monadic thinking" there is a good chance I will prioritize fixing it.

[pakoito]

I've got your email now. You can remove it from the post for safety reasons :D Let's move the discussion there!

[GlenKPeterson]

I'm closing this. There will be no Java 7 backport, or at least I'm not planning to work on one. If all goes well, it looks like I'm going to contribute to https://github.com/kategory/kategory and that seems to be causing me to do a fair amount of a total Kotlin rewrite, which is probably a good thing. Thanks @pakoito and @raulraja for your support and encouragement. I look forward to working more with you in the future!