Save every iteration on pareto_hyperparameters.csv and pareto_agreggated.csv

[joangcc]

Project Robyn

Describe issue

Hi, We are interested in analyzing how different iterations evolve. Currently, pareto_hyperparameters.csv and pareto_agreggated.csv do not save each and every iteration.

It seems we could obtain that by adapting robyn_run() function. See workaround (last two CSVs being created):

fwrite(resultHypParam[solID %in% allSolutions], paste0(plot_folder, 
    "/", plot_folder_sub, "/", "pareto_hyperparameters.csv"))

  fwrite(xDecompAgg[solID %in% allSolutions], paste0(plot_folder, 
    "/", plot_folder_sub, "/", "pareto_aggregated.csv"))

  fwrite(resultHypParam, paste0(plot_folder, 
    "/", plot_folder_sub, "/", "pareto_hyperparameters_all.csv"))

  fwrite(xDecompAgg, paste0(plot_folder, 
    "/", plot_folder_sub, "/", "pareto_aggregated_all.csv"))

Would it be possible to have it implemented in future versions? Maybe we could have an option to choose whether we want to save the csv with all data, or just the briefed version.

Best,

Joan

[gufengzhou]

Hey thanks for the suggestion. Actually all results are saved in OutputCollect (and MyRobyn.RDS) object. Do these serve your purpose?

The because some tables take on row number of iterations * num. indep. vars, which scales up very fast with a larger dataset and weibull adstock that requires more iteration. But Of course we could make csv for all data optional, too.

[joangcc]

Thanks for your quick response! Sure, I can see how quickly those CSVs can become bigger with more iterations and vars. Nevertheless, we would really appreciate having this option to create those CSVs. Best, Joan

[gufengzhou]

Hi, I've just added an parameter to do this. set csv_out = "all" in robyn_run() to output all iterations in csv. Commit is here: https://github.com/facebookexperimental/Robyn/commit/094ea5e2ee94dde907681e067ef5cab7a56d8f1c. Let me know if it works.

[joangcc]

Hi, thank you so much. It works. It is going to be really useful.

[gufengzhou]

I'm curious what how would you like to use the data and generally how do you use Robyn, if you can reveal a bit of your use case? FYI we've introduced a new ROAS convergence plot showing how paid media ROAS evolves over iterations. An example is included in the readme now.

[joangcc]

Hi gufengzhou, I'm part of a wider team responsible for the Marketing Mix modeling at Vista (formerly VistaPrint). We are evolving our current MMM approach, and since attending your last summit, we are looking at the potential of Robyn for our case. We advertise in upper and lower funnel channels, both online and offline. We expect these channels interact with each other, for example we expect Brand oriented TV and Display ads drive users to Paid Search. The context of this and other issues we have opened on GitHub is our interest in Robyn and developing trust on how it works and how it can fit our needs. We are using those CSVs with all the iterations to see how stable the solutions are along the different trials and iterations, whether some contributors are more volatile than the rest, and select the best modelling approach. We really appreciate having the opportunity to openly discuss about Robyn here on GitHub, and looking forward to further collaboration. Best! Joan