GELOG / adam-ibs

Ports the IBS/MDS/IBD functionality of Plink to Spark / ADAM
Apache License 2.0
3 stars 6 forks source link

Persistence for MDS using ADAM format #19

Open davidonlaptop opened 9 years ago

davidonlaptop commented 9 years ago

Description

This feature allow persistence of the models created in issue #17 and #18 (MDS, MIBS, MDIST).

The Spark models should integrates with the ADAM format. New record types may need to be added to the ADAM format.

The ADAM formats can be added as a maven dependency. The structure of the ADAM format is defined in an Avro file. The Avro file can be compiled into Java classes.

For updates to the ADAM format, choose the most simple solution for you as long as it is easy to diff the changes to the Avro file.

Analysis

Add a comment to this issue with:

Add a comment to this issue describing how this will be implemented in Spark, and explain how the persistence model will work.

Implementation

The implementation should use: