harsha2010 / magellan

Geo Spatial Data Analytics on Spark
Apache License 2.0
534 stars 149 forks source link

Add Encoders for Magellan Data Types #201

Closed halfabrane closed 6 years ago

halfabrane commented 6 years ago

This PR adds support for Encoders for Magellan Data Types

Currently we use codegen fallback and only support evaluated mode A subsequent PR will add support for codegen

Usage:

import magellan.encoders.Encoders._
import sqlContext.implicits._
val points = sc.parallelize(Seq(
      Point(-1.0, -0.9),
      Point(1.1, -0.8),
      Point(1.2, 0.9),
      Point(-0.8, 0.9)
    )).toDS()

points.filter(_.getX() == -1.0) 
codecov-io commented 6 years ago

Codecov Report

Merging #201 into master will increase coverage by 0.2%. The diff coverage is 95.12%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #201     +/-   ##
=======================================
+ Coverage    89.8%    90%   +0.2%     
=======================================
  Files          50     52      +2     
  Lines        1755   1791     +36     
  Branches      128    125      -3     
=======================================
+ Hits         1576   1612     +36     
  Misses        179    179
Impacted Files Coverage Δ
...in/scala/org/apache/spark/sql/types/PointUDT.scala 96.87% <100%> (ø) :arrow_up:
.../scala/org/apache/spark/sql/types/PolygonUDT.scala 100% <100%> (+6.25%) :arrow_up:
...scala/org/apache/spark/sql/types/PolyLineUDT.scala 94.44% <100%> (+5.55%) :arrow_up:
src/main/scala/magellan/encoders/Encoders.scala 100% <100%> (ø)
src/main/scala/magellan/PolyLine.scala 89.47% <100%> (+0.28%) :arrow_up:
...apache/spark/sql/catalyst/expressions/serdes.scala 71.42% <71.42%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 1c5b99f...eba4bea. Read the comment docs.