search

facebookexperimental / Robyn

Robyn is an experimental, AI/ML-powered and open sourced Marketing Mix Modeling (MMM) package from Meta Marketing Science. Our mission is to democratise modeling knowledge, inspire the industry through innovation, reduce human bias in the modeling process & build a strong open source marketing science community.
https://facebookexperimental.github.io/Robyn/
MIT License
1.08k stars 322 forks source link

Error in robyn_outputs step when recreating model from previous model inputs and hyperparameters #804

Open cynthia10wang opened 10 months ago

cynthia10wang commented 10 months ago

Hi there,

During my previous model run I forgot to save the model json file for the selected solution that I want, so I'm trying to recreate it with the same model inputs, inputs json file, hyperparameters from the previous model solution. I can successfully recreate the model OutputModels object with the following code and it also generated the exact one-pager as my model run in the past.

OutputModels <- robyn_run( InputCollect = InputCollect, cores = 1, dt_hyper_fixed = dt_hyper_fixed_1_4293_1, iterations = 5000, trials = 10, ts_validation = FALSE, add_penalty_factor = FALSE, outputs = FALSE )

However, because the outputs feature is deprecated in the robyn_run step and I have to run the robyn_outputs step, it gave the following error, while I can see that in my OutputModels object the hyper_fixed argument is True.

Running Pareto calculations for 1 models on auto fronts... Error in !hyper_fixed : invalid argument type

I was using the following robyn_outputs code. Can you please advise how to fix this?

OutputCollect <- robyn_outputs( InputCollect, OutputModels, pareto_fronts = 1, select_model = "1_4293_1" )

Environment & Robyn version

Robyn version: 3.10.3 R version 4.2.2

gufengzhou commented 10 months ago

the robyn_run(outputs = TRUE) argument is not deprecated, you should be able to use it. Otherwise, it doesn't make sense to put select_model = "1_4293_1" into robyn_outputs

cynthia10wang commented 10 months ago

the robyn_run(outputs = TRUE) argument is not deprecated, you should be able to use it. Otherwise, it doesn't make sense to put select_model = "1_4293_1" into robyn_outputs

Isnt it released here? https://github.com/facebookexperimental/Robyn/pull/560. Anyways, I did try with outputs=True but it doesnt generate the json output for the model either.

gufengzhou commented 10 months ago

Are you using the latest version? Because I just tried and it worked

cynthia10wang commented 10 months ago

Yes I'm using 3.10.3

Tried today again, and when I ran the robyn_run step with dt_hyper_fixed condition and outputs=True, it only recreated the one-pager, model inputs json, raw_data csv and 4 pareto csv files. No other outputs

Then when I ran the robyn_outputs step, it gave me the error of "Error in !hyper_fixed: invalid argument type" as I mentioned earlier.

FYI, it's working totally fine for me when I'm running a new model - all steps are giving its outputs as intended. Just the recreation of a previous model from a certain solution ID's hyperparameters and then outputs is giving me trouble. We need to make it work as we forgot to save that specific model json output during that run, and didnt use a seed when training either, however we need to replicate that model run and get the outputs to run optimizer/refresh for later exercises.

gufengzhou commented 10 months ago

I just tested the following:

df_hyppar <- read.csv("/Users/gufengzhou/Desktop/Robyn_202309041517_init/pareto_hyperparameters.csv") %>% filter(solID == "1_134_6")
test_out <- robyn_run(
  InputCollect = InputCollect,
  dt_hyper_fixed = df_hyppar
)

This gives me the following output image

Then I have also succesfully recreated and exported the model json with the following: 

export_json <- robyn_write(InputCollect, test_out, select_model = "1_134_6", export = TRUE)

Does this work for you?