search

twitter / scalding

A Scala API for Cascading
http://twitter.com/scalding
Apache License 2.0
3.5k stars 706 forks source link

Upgrade scala and java versions #1941

Closed ttim closed 3 years ago

ttim commented 3 years ago

I'm using post java 13 on my laptop and turns out it's incompatible with scala 2.12.8 we use in scalding. I've updated both scala version to the latest available, and java version to 1.8 because PermGen related code wasn't working properly in sbt runner we use, and java versions before 1.8 seem obsolete at this point (and not supported).

ttim commented 3 years ago

@johnynek while doing this I found a comment in chill repo made by you (https://github.com/twitter/chill/commit/4d1ea5d08fc8c723d263d2638201597312928d6a):

// Twitter Hadoop needs this, sorry 1.7 fans

=)

ttim commented 3 years ago

@johnynek I removed https://github.com/HairyFotr/linter from our build file, it seems unsupported anymore (last commit from 2016), and it doesn't work with latest 2.12 Scala =(

I also bumped scrooge version, because in java 13 javax.annotation.Generated was removed, and now should be explicitly imported (with javax.annotation jar), which was fixed in scrooge 19.8.0. And finally method resolving seems changed a bit, so I ended up making position to have explicit () argument list.

johnynek commented 3 years ago

I think the linter is fine to remove.

The new hotness is scalafix:

https://scalacenter.github.io/scalafix/

that's what we should probably set up for linting.