problem jars

[pmcelhany]

Some of the jars have big increases in the raw counts from one time step to the next. I wrote code and labeled jars with an increase of 10 or more per time step as "problem jars". Because the underlying raw counts include such large increase. Finding monotonically decreasing similation series will slow going because most trajectories will be rejected. You should look through the data on those jars to make sure the counts in the file are correct.

[lalma]

@pmcelhany I looked over my raw data and I found only 1 typo, in jar 28, where a number should have been 2 instead of 22. Otherwise, those counts for the remaining 7 problem jars are just what we got. Larger/smaller counts may be due to inconstant mixing or some other extraneous factor. Would it make sense to throw out certain days that have problems that are above or below a certain threshold?

[pmcelhany]

If there are no obvious problems in the data, we should just use them. It will just take a little longer to get valid decreasing trajectories for those jars. Theoretically, the number of larvae in the jar could be any integer >=0, though if, say, the sample count is 20 the probability that there are 10^8 larvae in the jar is impossibly small. The point is that you will always eventually draw a decreasing trajectory. I added some code and pushed a new version to montor how long it take do the simulation loop. To make 5 replicates of the 42 "good jars" took ~12minutes. To make 5 replicates of the 6 "problem jars" took ~25 minutes. Doable. We can run it over multiple computers and get the files we need in a day.

On Wed, Sep 29, 2021 at 2:43 PM Lindsay Alma @.***> wrote:

I looked over my raw data and I found only 1 typo, in jar 28, where a number should have been 2 instead of 22. Otherwise, those counts for the remaining 7 problem jars are just what we got. Larger/smaller counts may be due to inconstant mixing or some other extraneous factor. Would it make sense to throw out certain days that have problems that are above or below a certain threshold?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/lalma/RacoonOlyLarvalSurvival/issues/5#issuecomment-930566740, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOOQSLCNIQFTQ66RR7HMPLUEOB6NANCNFSM5E6YNPKQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[lalma]

Great! good to hear that the wonky counting for those few jars wont ruin the stats completely. I have some events with the department to attend all afternoon today so I may not have time to run much today, but I will dive in tomorrow afternoon. As always, thanks so much for your help!

[lalma]

I also ran 5 reps the good and problem jars. Took my computer about 30 and 45 minutes (my poor 6 year old laptop probably cant process as fast). I just started running 500 reps of the good jars. Depending on how many hours/days that takes, I will run the problem jars next. Then I'll prep them for the cox model. Do you think we should go for 1000 reps, or is 500 sufficient?

[pmcelhany]

I think 500 is probably enough. I can run some on my computer if needed. -paul

On Fri, Oct 1, 2021 at 7:00 PM Lindsay Alma @.***> wrote:

I also ran 5 reps the good and problem jars. Took my computer about 30 and 45 minutes (my poor 6 year old laptop probably cant process as fast). I just started running 500 reps of the good jars. Depending on how many hours/days that takes, I will run the problem jars next. Then I'll prep them for the cox model. Do you think we should go for 1000 reps, or is 500 sufficient?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lalma/RacoonOlyLarvalSurvival/issues/5#issuecomment-932662088, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOOQSOLES2YKOM3ZM73CBDUEZRS5ANCNFSM5E6YNPKQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[lalma]

My computer just finished running the "good jars" It took 1.6 days. The csv for the good jars is in the output folder called "d_sim" I will go ahead and run the "problem jars" now. Not sure how long It will take, but my computer did eventually get through the good jars so I'm guessing it will be a day or 2.

[pmcelhany]

Why don't you start your computer running 200 and I'll run 300 and can then append them together.

On Mon, Oct 4, 2021 at 12:59 PM Lindsay Alma @.***> wrote:

My computer just finished running the "good jars" It took 1.6 days. The csv for the good jars is in the output folder called "d_sim" I will go ahead and run the "problem jars" now. Not sure how long It will take, but my computer did eventually get through the good jars so I'm guessing it will be a day or 2.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lalma/RacoonOlyLarvalSurvival/issues/5#issuecomment-933808686, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOOQSKMNWYGO25THFGIJEDUFIBRNANCNFSM5E6YNPKQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[lalma]

Sounds like a plan. I will let you know when 200 are done

[pmcelhany]

great. I started the 300. Don't forget to change the name to something other than "d_sim.csv" when you save the file so you don't overwrite the good jars

[pmcelhany]

I made 300 problem jar. the result is in output/d_sim_pt_300.csv

[lalma]

Thanks for running those Paul! looks like my computer is almost done running the 200. It has 52 lines left on Jar 48. Next, I will put them all together and format the data for the cox model.