GSoC: going deeper! - Githubissues

astrolabsoftware / spark3D

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteorology, …

https://astrolabsoftware.github.io/spark3D/

Apache License 2.0

30 stars 16 forks source link

GSoC: going deeper! #46

Open JulienPeloton opened 6 years ago

JulienPeloton commented 6 years ago

First steps logbook: #6

Week 5:

Added Octree Partitioned RDD support #36
Add SphereRDD #38
Code refactoring: Replace Sphere by ShellEnvelope, Shape3D heritage, and move everything under geometryObjects #40
v0.1.1 #43
Remove confusion between cartesian and spherical coordinates for ShellEnvelope and BoxEnvelope #44

A few TODO for week 6:

Fix bug while loading Octree #45
Treatment of objects at the border of partitions (see #25)
Multi-resolution algorithm for cross-match (see #26)
Tackling large shuffle? #32
Implement RangeQuery methods (to find neighbours).
spark3D website

JulienPeloton commented 6 years ago

Weeks 6 & 7 & 8:

0.1.4

Unify the IO: One constructor to rule them all! (#69)
Octree and Geometry Objects Bug Fixes (#67)

0.1.3

Add KNN routines (#59, #60, #62)
Unify API to load data (#63, #64)
Speed-up cross-match methods by using native Scala methods (#58)
Add a new website + spark3D belongs to AstroLab Software (website)
Update tutorials (tuto).
Many fixes! (see #46 - #47)

JulienPeloton commented 6 years ago

Week 9:

0.1.5

Allow user to cache the rawRDD while loading data to speed-up re-partitioning. (#81)
Script to benchmark the partitioning (#72)
Several minor fix (#71, #76, #80)