search

gisaia / ARLAS-proc

Workaround about data ingestion with computing frameworks
Apache License 2.0
4 stars 0 forks source link

WIP Optimize HMM processing #75

Closed laurent-thiebaud-gisaia closed 5 years ago

laurent-thiebaud-gisaia commented 5 years ago

Avoid to translate all data to RDD, but only the required ones

laurent-thiebaud-gisaia commented 5 years ago

In WIP because trying to make it even better. The processing of 1 day took previously 1 hour. With this branch it needs about 15 min.

laurent-thiebaud-gisaia commented 5 years ago

Removing the PR, https://github.com/gisaia/ARLAS-proc/pull/76 is even better