search

astrolabsoftware / spark3D

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …
https://astrolabsoftware.github.io/spark3D/
Apache License 2.0
30 stars 16 forks source link

Octree: check for initial null value when computing the data envelope #124

Closed JulienPeloton closed 4 years ago

JulienPeloton commented 4 years ago

Linked to issue: #121

When applying seqOp to determine the envelope of the dataset (Octree.scala), the first comparison is done between a null BoxEnvelope and a point of the dataset. The null value has coordinate x=(0, -1, 0, -1, 0, -1), and the implicit assumption was that the dataset contains the (0, 0, 0, 0, 0, 0) point. But for a dataset completely shifted from the origin, the code was producing a huge envelope (to include the origin), leading to huge empty partitions.

codecov-io commented 4 years ago

Codecov Report

Merging #124 into master will increase coverage by 0.01%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #124      +/-   ##
==========================================
+ Coverage   93.67%   93.68%   +0.01%     
==========================================
  Files          33       33              
  Lines        1233     1235       +2     
  Branches      197      188       -9     
==========================================
+ Hits         1155     1157       +2     
  Misses         78       78
Flag Coverage Δ
#python 93.65% <ø> (ø) :arrow_up:
#scala 93.69% <100%> (+0.01%) :arrow_up:
Impacted Files Coverage Δ
src/main/scala/com/spark3d/Partitioners.scala 87.5% <100%> (+0.46%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 047bd85...9e3fe2c. Read the comment docs.