move HTSJDK to java 8

[lbergelson]

I suspect this suggestion will be controversial.

It would be helpful for a lot of reasons to allow java 8 code in htsjdk. There are many useful features which included in 8 which were not in 6, or 7. See (here)[http://www.oracle.com/technetwork/java/javase/8-whats-new-2157071.html] for a complete list.

In my view, the biggest and most desirable changes are the introduction of lambdas, default methods in interfaces, and the stream APIs.

[geoffjentry]

I can assure you that this would cause great wailing and gnashing of teeth, There is a non-trivial portion of the user base who are still tied to Java 6 and I don't think we can abandon them. My understanding is that you can't use -target down to even 1.7 much less 1.6 from 1.8 but I could be wrong there.

Personally I'm a fan of telling people to get with the times, but that's not really how things work. If you really want the cool features of Java 8, just go all the way and use Scala :)

[lbergelson]

I would also support a move to scala, but there is even less appetite for that. It's unclear to me who is still using java 6, and for what reason. If you can't update your software versions once every decade, I'm not sure you need the newest htsjdk?

[mattsooknah]

1. Meta question: what's the best place to talk about this? If we want to have a discussion we should possibly try to maximize its visibility to users. At the same time, we need to make sure it doesn't turn into a flamewar.
2. I'll echo Louis' question - what are some reasons why some users are tied to Java 6?

[lbergelson]

Yeah, we need to find out who is using java 6 and for what reason. Maybe there is a good one, but I don't know what it is.

[geoffjentry]

When we moved Picard to 1.7 the decision to keep htsjdk at 1.6 was intentional to make it as inclusive as possible - it's often the case that an organization will be tied to an old version of something due to a least common denominator effect. We even get the occasional person trying to get 1.5 to work.

Also note that 1.8 can't (again, I might be wrong) be targeted to 1.7 - and while I agree that it's ludicrous that people are still sticking around on 1.6, you can't say that about 1.7.

[lbergelson]

It's not ludicrous to be on 1.7, but I don't know a very good reason for it either. Java 8 is mostly compatible with 7 binaries see http://www.oracle.com/technetwork/java/javase/8-compatibility-guide-2156366.html.

[mattsooknah]

If we moved to 1.8, we could still keep a pointer to the last 1.6-compatible release, so people could use that if they needed to. Moving to 1.8 doesn't need to mean "remove all references to older HTSJDK releases". And as Louis implied, if you're on 1.6 then you're probably not eagerly updating your HTSJDK on every release (although I could be wrong).

On the other hand, supporting the older version (even just passively) could be a headache.

[lh3]

Although I am not directly involved in htsjdk, I work with various endusers quite often. I am strongly against moving to java 8 which is only one year old. Htsjdk is a library intended to be used by all java programs processing BAM/CRAM. When you are dealing with this large user base, you will have many exceptions. For example, some biologists may not know how to install java but the system only provides an old version; some developers may have to follow policies set by the company and use a specific java version; some semi-developers may not know the new features of java 8 and may be confused by the APIs involving java 8 features.

As to binary compatibility, the article only says that java 8 is backward compatible with java 7, which means if htsjdk is java 8 only, all java 7 programs cannot use htsjdk any more. This is a big deal. We'll need to embrace new java versions, but not right now.

[geoffjentry]

The problem is that HTSJDK is designed to be at the bottom of a tech stack and in general you want to be more universal the further down you go.

I'm all for Java 8 - if I was forced to use java as my primary language again I'd shed a tear if it wasn't 8 but I'll just reiterate my initial statement that if one was to bump HTSJDK all the way to 8 there'd be a lot of commotion. I suspect you'd have a lot of commotion (albeit less so) going to 7.

I'd be a lot more pro-Java 8 for Picard and 100% pro-Java 8 for Broad internal tools but for HTSJDK this way leads to pain.

[lbergelson]

The upcoming changes to Picard/Gatk are moving them to be java 8, so anyone who isn't doing their own development is going to have to update already. If you are doing java development based on htsjdk yourself, I'd wager you know how to install a new version of java.

For example, some biologists may not know how to install java but the system only provides an old version

We're moving to 8 for new tool development, so they're not going to be running anything in this case

some developers may have to follow policies set by the company and use a specific java version

That's a real problem for them, I think that it might be a bit of a red herring absent any specific examples though. Any company that is still on java 6 seems like they're opening themselves to major security problems too.

some semi-developers may not know the new features of java 8 and may be confused by the APIs involving java 8 features

Most of the new features are pretty easily understandable, I'd argue that they would make the code more clear rather than less.

If I proposed moving to 7 would that be less of an issue?

[geoffjentry]

@lbergelson "so anyone who isn't doing their own development" - And that's exactly why htsjdk is separate from Picard.

Edit: I realize that wasn't entirely clear. htsjdk is separate specifically so people can develop w/ it separate from picard. Not everyone will even be on J7, much less J8. Not everyone can upgrade. You'll probably face less resistance to J7 but you'll still face some resistance.

[nh13]

Lets discuss in person.

[akiezun]

Java 6 is from 2006, almost a decade ago. Is there a principle that will be followed to decide when an upgrade to newer versions happens? There needs to be one or we may be forever stuck on technology from 2006.

[lh3]

What is more relevant is that Java 7 is from 2011. Has everyone moved to Java 7 in these 4 years? (As a non-java programmer, I don't know the answer.) Are there life-changing features in Java 7/8 we have to use? (Again, I don't know the answer.)

[geoffjentry]

@lh3 There are some nice things in 7 but 8 is where the real goodies are, although as I've stated I think that 8 is far too new to move to for htsjdk. I'd personally support a move to 7 but I'm not sure the advantages of that are worth the potential disruptions to others.

[pgrosu]

@lh3, when you use Java 8, the clouds part, the sun starts shining, the snow begins to melt and the birds harmoniously are chirping :) I'm kidding, Java 7 is still being used quite a lot. Java 8 just offers some new features that can be pretty and efficient from a programming perspective, as well as some optimizations:

• Lambda expression (anonymous functions or inner classes, which can make for quite efficient coding)
• Stream collection types (operations on streams [i.e. map/reduce] which can be parallelized, but can become slow)
• Type annotations (annotate generic types)
• Some improvements in performance
• Several other things that get quite specific

Even though Oracle is talking Java 9 for 2016, for now people will be happy with either 7 or 8.

~p

[lbergelson]

There are some pretty major improvements in java 8. Java 7 is a minor upgrade from 6, but still has nice features. I'm not sure there would be much advantage in a move to 7. There's major advantages in in terms of convenience and improved capabilities in 8 though. As of this month, java 7 is automatically upgrading itself to java 8 on home installations. Oracle is discontinuing public support for 7 in april, which is not very long from now. Six has been unsupported for 2 years now. The support roadmap is here: http://www.oracle.com/technetwork/java/javase/eol-135779.html

Since 8 is broadly backwards compatible both at source and binary levels, it's unclear to me that there is any advantage to anyone to stay on an older version. The only reason good reason to still be using 6 or 7 is that you either have a very specific compatibility issue or you have extremely an extremely slow IT department that also doesn't care about security at all.

[nh13]

There are many use cases (think clinical) where extremely slow IT is a feature. The issue is closed so we can discuss this with a few folks internally before making a proposal, if any, to the community of htsjdk users and developers.