search

Azure / cortana-intelligence-price-optimization

Retail industry solutions for product price optimization using the Cortana Intelligence Suite with end-to-end walkthrough
82 stars 59 forks source link

Updating Solution for Spark 2.3 (HDI 3.6) #9

Open lbenassi opened 5 years ago

lbenassi commented 5 years ago

Hi I am trying to implement this solution to deploy a POC for the forecasting solution. I was able to update the json files to the version v2 of Data Factory, but I still can not run the pipeline because the spark version has also changed. Spark 1.6.3 (HDI 3.5) version is used in the solution, but i have a Spark 2.3 (HDI 3.6). It would be possible to update the solution to work with new versions of Azure Data Factory and Spark? Or someone knows how to update the solution for this new versions of Azure Data Factory and Spark? Thanks

Schmidtbit commented 5 years ago

The documentation says Spark 2.0.2 (HDI 3.5) which isn't even an option. I am trying right now to deploy with the older version of spark. I'll let you know if I can get the rest of the tutorial to properly execute.