search

cloudera / CML_AMP_Structural_Time_Series

Applying a structural time series approach to California hourly electricity demand data.
Apache License 2.0
9 stars 11 forks source link

Installing dependencies takes excessively long #2

Open andrewrreed opened 3 years ago

andrewrreed commented 3 years ago

The Problem

When launching this AMP as an ML Prototype on CML, the install_dependencies job takes ~25 minutes which is much longer than it should take. Increasing vCPU does not help.

After further investigation, it appears that the prolonged install is due to a failure of fbprophet to be installed from the PyPi wheel and therefor, it is manually built by a fallback run of setup.py which ultimately works, but takes a significant amount of time.

image

Attempted Fixes

andrewrreed commented 3 years ago

@cjwallace Do you have any ideas/experience with this?

andrewrreed commented 3 years ago

It looks like this is a known and open issue:

cjwallace commented 3 years ago

Good news: I fixed it in PR #3. We can install prophet without error.

Bad news: It still takes forever to run the install step. The error wasn't adding much time - it simply takes a long time to build (I think because of Stan). It's a lot faster on my local machine than cloud, which could just be CPU speeds.