facebookexperimental / Robyn

Robyn is an experimental, AI/ML-powered and open sourced Marketing Mix Modeling (MMM) package from Meta Marketing Science. Our mission is to democratise modeling knowledge, inspire the industry through innovation, reduce human bias in the modeling process & build a strong open source marketing science community.
https://facebookexperimental.github.io/Robyn/
MIT License
1.12k stars 332 forks source link

Error importing JSON Robyn Refresh #711

Closed jkngale closed 4 months ago

jkngale commented 1 year ago

Project Robyn

Describe issue

I tried to refresh robyn's model with new data and got an error when refreshing even that the model is imported correctly. It appears to search for an empty model even that the model is imported and recreated in the same step.

Provide reproducible example

json_file <- gsub(" ","",paste(plot_folder,"RobynModel-2_339_1.json", sep=""))

"MMM/output/Robyn_outputs/Robyn_202304141255_init/RobynModel-2_339_1.json"

RobynRefresh <- robyn_refresh( json_file = json_file, #json_file, dt_input = df_weekly, dt_holidays = dt_prophet_holidays, refresh_steps = 1, refresh_iters = 20, refresh_trials = 1, ts_validation = FALSE )

Ouput in the log

Recreating model 2_339_1 Imported JSON file succesfully: MMM/output/Robyn_outputs/Robyn_202304141255_init/RobynModel-2_339_1.json Input 'window_end' is adapted to the closest date contained in input data: 2023-03-16 Running feature engineering... NOTE: potential improvement on splitting channels for better exposure fitting. Threshold (Minimum R2) = 0.8 Check: InputCollect$plotNLSCollect outputs Weak relationship for: "Paid_BingAds_Imp", "Paid_GoogleAds_Shopping_ES_Mesas_Imp", "Paid_GoogleAds_Text_Generic_Imp"and their spend Input data has 170 weeks in total: 2019-12-30 to 2023-03-16 Initial model is built on rolling window of 170 week: 2019-12-30 to 2023-03-16 Successfully recreated model ID: 2_339_1 Error in robyn_read(filename, quiet = TRUE) (MMM.R#361): JSON file can't be imported: MMM/output/Robyn_outputs/RobynModel-.json

Environment & Robyn version

Make sure you're using the latest Robyn version before you post an issue.

laresbernardo commented 1 year ago

Hi @jkngale thanks for reporting. Noticed you're using version 3.9.0. Would you mind testing with the latest dev version (>=3.10.3.9000)? I'm not able to reproduce your error, which actually doesn't make much sense given, as you mentioned, that it created the original model correctly. I also noticed from the error that:

jkngale commented 1 year ago

Hello @laresbernardo,

I tried again with the new version and had the same error. I'm running it on windows, is it possible that in mac works correctly?

Thanks

laresbernardo commented 1 year ago

Hi! I don't have a Windows to test but it should be exactly the same behaviour. Would you mind sharing a reproducible example (zip folder with all the structure - you can keep the json and data files only) and the code to replicate the error? Thanks.

AngelLopezSanz commented 1 year ago

Hi @laresbernardo. I faced the same issue that @jkngale explained. Trying to find the reason in the code, I run the github code of robyn_refresh function and find that the error arises on robyn_chain function. Checking the code here I think the problem is on the second robyn_read, where I think filename is not well created. I haven't check to find a solution yet, but I suppose it's easier for you to check and change there. I'll check to find a quick fix for me (and share here if it could be useful for anyone)

AngelLopezSanz commented 1 year ago

Hi @laresbernardo. I could fix that function in my side, but running all the code was not easy for me and multiple errors appeared. Did you have the chance to test it? By the way, I'm on mac. @jkngale did you find a solution? I need to recover a model and currently I cannot...

banalytics commented 1 year ago

@laresbernardo I am running into the same issue (Mac) with the newest version still. I think came to the same conclusion as @AngelLopezSanz on where the issue lies. Fixing it however requires writing custom functions that I would run instead of the package provided functions.

Are there any plans on fixing? Happy to help.

laresbernardo commented 1 year ago

Hi! I'll try and take a look later next week and see if there's an easy fix. Sadly I was laid off from Meta a couple of weeks ago and won't continue giving active support to Robyn. Feel free to continue this thread @gufengzhou ;)

shuvayan commented 4 months ago

Any updates on this ? It has been more than 1 year for this issue and it is still open. I am also facing the same issue now with model refresh, this just defeats the purpose of open sourcing solutions if there is no effort to maintain and fix it.

gufengzhou commented 4 months ago

this is being handled in your latest issue so i'm closing this. there're many reasons why reply is slowing down. so please have understanding and be fair