Script for Running Experiment

coraallensavietta / RankingMethods

A Weighted Loss-Based Bayesian Ranking Method

0 stars 0 forks source link

Script for Running Experiment #9

Open coraallensavietta opened 6 years ago

coraallensavietta commented 6 years ago

[x] draft code
[x] create data frame to store results, have code add rows to this
[x] write df to csv to save
[x] write testing code for parameters

coraallensavietta commented 6 years ago

made progress, but stuck on adding parameters and results to data frame 7eea5a5

coraallensavietta commented 6 years ago

now returns dataframe 33679e4

coraallensavietta commented 6 years ago

decided to just rbind dataframe in the simulation. temporary solution. 38325fe

coraallensavietta commented 6 years ago

fixed it! 6a19e9c

coraallensavietta commented 6 years ago

testing n_min and n_max 4d1a9e5

coraallensavietta commented 6 years ago

From meeting 6/8: Make the following changes to RunSimulation.

[x] Ron says: for lack of convergence, try increasing burn-in (actually warmup, right?) https://cran.r-project.org/web/packages/rstanarm/vignettes/rstanarm.html#markov-chains-did-not-converge Increased iterations. fixed at d247d41 and edd9582
[x] check parameters in WeightedLossRanking
[x] Save as an RData object, not csv.
[x] where is rankPriority used? Answer: rankWeights
[x] fix NAs in df.
[ ] Create a data frame of lists. (Make an element of df be a list or matrix.) We want rankings to be a matrix within a list, one for each simulation. For every simulation, we need a matrix of ranks. Look for this discussion in tidyverse data science R book.

coraallensavietta commented 6 years ago

6/15/18 NOTES

downloading R packages on the server? yes. doing it correctly, but need newer version of R.
[x] email server admins about updating R
[x] keep track of time. can use other strategies to get approx samples function: system time wrap everything in this to see.

coraallensavietta commented 6 years ago

[x] add rankMetric to code for runExperiment.r in commit ea6396e moved rankMetric code to sim_ranking_experiment.r file for simplicity of organization.

coraallensavietta commented 6 years ago

server code now works. run using /s/pkg/linux64/R/3.4.1/bin/Rscript ranking_experiment_run_server.r

commit 0d15f30

coraallensavietta commented 6 years ago

new bug:

Error in RunSimulation(n_sim = 1, metric = TRUE) : object 'returnDF' not found Calls: source -> withVisible -> eval -> eval -> RunSimulation In addition: Warning message: 'rBind' is deprecated. Since R version 3.2.0, base's rbind() should work fine with S4 objects Execution halted

coraallensavietta commented 6 years ago

check: n = 10 simulation (running since Fri at 5:30 pm) check: n = 100 simulation (running since Fri at 5:30 pm) Error in $<-.data.frame(*tmp*, "ranking", value = list(c(2L, 1L, 4L, : replacement has 2 rows, data has 1 Calls: system.time -> rbind -> RunSimulation -> $<- -> $<-.data.frame