Spark 2.0.0 support for feature selection and MDL

sramirez / spark-infotheoretic-feature-selection

This package contains a generic implementation of greedy Information Theoretic Feature Selection (FS) methods. The implementation is based on the common theoretic framework presented by Gavin Brown. Implementations of mRMR, InfoGain, JMI and other commonly used FS filters are provided.

http://sci2s.ugr.es/BigData

Apache License 2.0

134 stars 46 forks source link

Spark 2.0.0 support for feature selection and MDL #7

Closed kiranvaranasi81 closed 7 years ago

kiranvaranasi81 commented 7 years ago

Currently feature selection and MDL techniques are built using spark 1.6.1. Is there plan to support spark 2.0.0 or latest versions?.

sramirez commented 7 years ago

Actually, this packages has support for 2.0.0. Just take a look at the last commit. Thanks to @jabenitez you can build it on Spark 2.0.0. Concerning the MDLP discretizer, I am working on this feature now.