gisaia / ARLAS-proc

Workaround about data ingestion with computing frameworks
Apache License 2.0
4 stars 0 forks source link

Optimize HMM, without rdd but only dataframe / UDF. #76

Closed laurent-thiebaud-gisaia closed 5 years ago

laurent-thiebaud-gisaia commented 5 years ago

With real data, processing is now 9 min instead of 1 hour

laurent-thiebaud-gisaia commented 5 years ago

Waiting for the results to be validated by data scientist, before merging it

laurent-thiebaud-gisaia commented 5 years ago

Added some comments related to the new algorithm