sealuzh / cloud-stove

The Cloud Stove Backend
https://api.thestove.io
Apache License 2.0
4 stars 0 forks source link

Ingredient assignment in recommendations is incorrect #174

Closed inz closed 7 years ago

inz commented 7 years ago

It looks like mapping ingredients back from the MiniZinc model has an issue.

When I generate a recommendation for an instance of the new 'Rails app with PostgreSQL' template (where PostgreSQL master scales vertically, all other components scale horizontally), I get the following recommendations:

screen shot 2016-11-18 at 1 44 52 pm screen shot 2016-11-18 at 1 45 02 pm screen shot 2016-11-18 at 1 45 20 pm screen shot 2016-11-18 at 1 45 28 pm

The recommendations look sensible overall, but 'Delayed Job Worker' apparently scales vertically instead of 'PostgreSQL Master'.

I think we had a similar issue before, there is probably just an ordering mismatch between generating the model and parsing recommendations.

joe4dev commented 7 years ago

I could reproduce the same behavior and will look into that.

joe4dev commented 7 years ago

Interestingly, this error only occurs at production: https://app.thestove.io I couldn't reproduce this error locally or at staging: http://staging.frontend.thestove.io/ It appears that the order of recommendations is simply reversed (the ratio across instance count is roughly the same): production-vs-staging

This image compares production (left) vs staging (right). Notice that production uses pricing data from 1-2 months ago whereas staging is up to date. However, the price difference for Amazon is surprisingly huge 🙄 (945 vs 270)

joe4dev commented 7 years ago

I now updated the pricing data in production and also the dependent admin recommendation seeds. This seems to fix this ordering issue with the production data. The mapping is correct now although the ingredients are displayed in different order in production and staging.

The sanity check (production vs staging) looks fine producing exactly the same results: production-vs-staging_v2 production-vs-staging_v2-2

joe4dev commented 7 years ago

Newly generated recommendation in production still suffer from the ordering issue. Generating exactly the same recommendation (Rails app with 5k users) as the admin recommendation (created based on admin seeds) after instantiating the Rails template yielded this issue again: production-vs-staging_v3