Feat/stop pause summary

gisaia / ARLAS-proc

Workaround about data ingestion with computing frameworks

Apache License 2.0

4 stars 0 forks source link

Feat/stop pause summary #148

Closed laurent-thiebaud-gisaia closed 4 years ago

laurent-thiebaud-gisaia commented 4 years ago

Add stop-pause summary transformer, implementing an abstract fragment-summary transformer.

Please note that the precision of track_precision fields (arlasTrackLocationPrecisionValueLat, arlasTrackLocationPrecisionValueLon) have been decreased; this is for the tests to pass. Because in very few cases the spark calculation (with stddev_pop) slightly differs from manual calculation. However keeping 12 digits has no impact on the correctness (it has been discussed with data scientist).

laurent-thiebaud-gisaia commented 4 years ago

To be merged after https://github.com/gisaia/ARLAS-proc/pull/150

laurent-thiebaud-gisaia commented 4 years ago

rebasing