Revamp coders with magnolia derivation

[nevillelyh]

Tentative plan:

• [x] Implement magnolia based coder derivation in a separate namespace
• [x] Provide implicits for built-in Beam coders
• [x] Implement implicit coders for common Scala and other data types
• [x] Make KryoAtomicCoder a fallback that needs to be explicitly imported and supplied
• [x] Test compile some customer pipelines to verify coder coverage
• [ ] Update downstream projects, e.g. featran, ratatool
• ~[ ] Automatic migration to v0.7.0 using scalafix~ (Impossible / really hard because of limitations in scalafix)

WIP / TODO

• [x] Fix Scalac warnings for derived coders (type parameter F shadows...). Make a PR on Magnolia.
• ~[ ] Fix Wart remover warnings for derived coders. Make a PR on Magnolia.~ (can't do. Annotations generated by macros are ignored)
• [x] Write documentation
• [x] More tests / benchmarks
• [x] Delegate verifyDeterministric in composite coders
• [x] Delegate registerByteSizeObserver in composite coders
• ~[ ] Remove disjunction~ (???)
• [x] Make sure macro fallback warnings works when compiling with scala 2.11
• [x] Implement coders for types used by Scio internally:
  • [x] org.joda.time.LocalDate
  • [x] org.joda.time.LocalTime
  • [x] org.joda.time.LocalDateTime
  • [x] ~java.nio.ByteBuffer~ double check if we should include this coder
  • [x] double check if we need bigtable MutationCase coder
  • [x] org.apache.beam.sdk.values.KV[String,Double]
  • [x] org.apache.beam.sdk.transforms.windowing.PaneInfo
  • [x] org.apache.avro.generic.GenericRecord
  • [x] com.spotify.scio.extra.annoy.AnnoyUri
  • [x] breeze.linalg.SparseVector[Double]
  • [x] Void
• [x] Support scala objects

[jto]

WIP on branch https://github.com/spotify/scio/tree/julient/serde-bench

[kanterov]

Good stuff, please ping us when it's ready to be tested. We have some hand-optimized pipelines that could be much more simple.

[nevillelyh]

Running into a outer$ serialization issue that we need to look into. https://github.com/spotify/scio/blob/julient/serde-bench/scio-core/src/test/scala/com/spotify/scio/coders/coders.scala

[jto]

@kanterov we're planning on releasing this with scio 0.7.0. I'd be very interested in testing your pipelines before that.

[nevillelyh]

Regarding release strategy: we will merge this into a 0.7.0 branch, make a few alpha & beta releases, before merging it to master and releasing stable 0.7.0.

[nevillelyh]

This is considered done. Closing for now and will create new issues for coder related stuff.