IBS clustering: outlier detection

[davidonlaptop]

Description

This feature adds the --neighbour option(s) based on the input file described in #3. For more info, check: http://pngu.mgh.harvard.edu/~purcell/plink/strat.shtml#outlier

Analysis

Add a comment to this issue with:

• plink version used as reference (1.07 or 1.90 beta 3)
• relevant C++ function name(s) and the file name(s) where they appear
• Document the structure of the NEAREST file format(s) generated by plink.
• Document the algorithm and/or mathematical formula to compute the missing fields:
  • NN
  • MIN_DST
  • Z
  • PROP_DIFF

    Design

Add a comment to this issue describing how this will be implemented in Spark, and how it differs from plink.

Also update the class diagram on the wiki page describing PLink formats (when incomplete) and add a class diagram describing the models implemented in Scala for this feature on the wiki page on the MGL804 formats.

Implementation

The implementation should integrates with the models implemented in scala by this project and use:

• Scala
• Spark RDD
• Spark MLlib / GraphX (if appropriate)

Important note: The model can be only in memory for now, but you'll need to integrate into the ADAM format later on. You'll probably need to create a new record type.