search

csiro-risk-assessment / mozzie

Mosquito lifecycle, diffusion and advection
GNU General Public License v3.0
0 stars 0 forks source link

include mosquito ODE system #25

Closed WilkAndy closed 4 months ago

WilkAndy commented 4 years ago

Hi @nickbeeton

You'll see that i've put your ode.py into the cellDynamics class. I could have easily made a typo!!!

I've included some simple unit tests in tests/TestCellDynamicsMosquito23.py but we obviously need to do more tests of the actual evolution.

One thing i'd really like to see is a reduction in our use of numpy and an increase in the use of cython arrays, even though they're more painful to use because they're 1D. Part of the reason for this is that in cellDynamics.pxd we have to cimport numpy which results in lots of deprecation messages. If we could make all the np.ndarrays in cellDynamics.pxd to 1D cython arrays (which will also be really fast), we can use scipy in cellDynamics.pyx without having to do a cimport (just an ordinary import will be fine)

WilkAndy commented 4 years ago

I was thinking about the np.ndarray -> cython array thing. I think the best strategy here is to first make a bunch of tests and then do the refactor. The problem with the refactor is it's going to make the code much more opaque, and it'll be easy to mess something up. But if we've got a lot of tests it'll be very clear if there is a mistake.

The tests should be explicit hand-calculations of the solution to \dot{x}=f , in different situations, ie with different parameter sets.

WilkAndy commented 4 years ago

Also, we have to either made sure the code is correct for num_species != 1. At the moment i'm pretty sure it's not. Or, we simply specify that num_species = 1 always.

nickbeeton commented 4 years ago

Yep, I haven't incorporated that but I will! And happy to try and remove numpy dependence, though I'd like to be able to keep the matrix multiplication, will that be possible without numpy? I'm flat out today with other stuff so unlikely to get much done today either.

Nick

From: Andy Wilkins notifications@github.com Sent: Tuesday, 12 November 2019 8:02:01 AM To: WilkAndy/mozzie mozzie@noreply.github.com Cc: nickbeeton nick.beeton@utas.edu.au; Mention mention@noreply.github.com Subject: Re: [WilkAndy/mozzie] include mosquito ODE system (#25)

Also, we have to either made sure the code is correct for num_species != 1. At the moment i'm pretty sure it's not. Or, we simply specify that num_species = 1 always.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/WilkAndy/mozzie/issues/25?email_source=notifications&email_token=ABTDQOD6ORUWIAABNG7RIQLQTHB4TA5CNFSM4JLUZWEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDYDSNI#issuecomment-552614197, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABTDQOEFXKWIUQ7F4LDT5MLQTHB4TANCNFSM4JLUZWEA.

WilkAndy commented 4 years ago

The tests might be quite easy, and i can take it from there. If you have an old fortran code that you're sure of, you can specify some parameters, record the input "X", then just run one timestep and get the output "X". Do that for all sorts of parameters and initial values, put the results into a file, or just paste here, and i'll make the tests. Even just one "input" and one "output" would help for now.

WilkAndy commented 4 years ago

I'm having fair success at getting rid of most of numpy. The bad news is that numpy+scipy appears to be quite slow. Of course i haven't optimised stuff AT ALL (in fact, quite the opposite), but there must be just quite a lot of overheads there that isn't in our native cython/C code. I'm not suggeting we refactor numpy+scipy out of the code, but we should just keep the disappointing slowness in mind.

WilkAndy commented 4 years ago

This is ready for your thorough checking, @nickbeeton . You will see that the code is much more verbose than yours. Partly this is to attempt to optimise speed, partly to make it fit with the other code. I've included just one somewhat non-trivial test. Tomorrow i'll write some more, BUT i really need you to check the code in fun and setInternalParameters, especially the indexing, before i write too many tests. There's no point in me testing a code whose dynamics is fundamentally flawed!

WilkAndy commented 4 years ago

Hi @nickbeeton . Is the inheritance cube correct? For Gw x GG we've got [0, 0.25, 0.25], while i would think it was [0, 0.5, 0.5]

WilkAndy commented 4 years ago

Hey @nickbeeton . Do you have a PDF describing the ODEs? I'm trying to understand these lines:

https://github.com/WilkAndy/mozzie/blob/3c05f105de73b7d039131768ff009a18b664f569/code/ode.py#L76-L77

because i'm not sure about the indexing. But my document describing the ODEs doesn't look much like those lines.

nickbeeton commented 4 years ago

Hi Andy,

Hi @nickbeetonhttps://github.com/nickbeeton . Is the inheritance cube correct? For Gw x GG we've got [0, 0.25, 0.25], while i would think it was [0, 0.5, 0.5] Yes of course, it’s [0, 0.5, 0.5] – I have no idea how that happened sorry! It didn’t even add to 1 the way I had it…

Hey @nickbeetonhttps://github.com/nickbeeton . Do you have a PDF describing the ODEs? I'm trying to understand these lines:

https://github.com/WilkAndy/mozzie/blob/3c05f105de73b7d039131768ff009a18b664f569/code/ode.py#L76-L77

because i'm not sure about the indexing. But my document describing the ODEs doesn't look much like those lines. I’ve attached the original PDF, which doesn’t include multiple species, so I’ll try to add that in. The way I’ve done it is a bit obfuscated (sorry) because I’ve essentially rearranged the formula to pair

[cid:image006.png@01D59A15.07138C50] as the ratio. That way everything that remains can be expressed as a multiple of the size of the mother population [cid:image007.png@01D59A15.07138C50] and we can write the ODE in matrix form, which I figured might be handy if we need to go the implicit route, or might simply be more efficient. I’ve tried to make sure the indices are all right in the resulting matrix, but as mentioned I haven’t tested that thoroughly yet.

Oh and note that we don’t use [cid:image008.png@01D59A15.07138C50] and [cid:image009.png@01D59A15.07138C50] as in the PDF formula anymore, but I’ve created a larger [cid:image008.png@01D59A15.07138C50] tensor to take into account the fact that X-shredding reduces males without increasing females.

I’ve been distracted by other tasks this morning annoyingly but I’ll write some tests as you suggested now.

Cheers, Nick

Dr Nick Beeton Biological Modeller | Data61 Ecological and Environmental Risk Assessment (DEERA) team CSIRO | Data61 E nick.beeton@data61.csiro.au T +61 3 6232 5318 3 Castray Esplanade, Battery Point TAS 7004 https://www.data61.csiro.au/en/Our-Research/Programs-and-Facilities/Analytics-and-decision-sciences/DEERA

CSIRO acknowledges the Traditional Owners of the lands that we live and work on across Australia and pays its respect to Elders past and present.

PLEASE NOTE The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference. Please consider the environment before printing this email.

[Data61 | CSIRO logo]

From: Andy Wilkins notifications@github.com Sent: Wednesday, 13 November 2019 10:28 AM To: WilkAndy/mozzie mozzie@noreply.github.com Cc: nickbeeton nick.beeton@utas.edu.au; Mention mention@noreply.github.com Subject: Re: [WilkAndy/mozzie] include mosquito ODE system (#25)

Hi @nickbeetonhttps://github.com/nickbeeton . Is the inheritance cube correct? For Gw x GG we've got [0, 0.25, 0.25], while i would think it was [0, 0.5, 0.5]

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/WilkAndy/mozzie/issues/25?email_source=notifications&email_token=ABTDQODW5E6FLF56N5BBK5LQTM3W3A5CNFSM4JLUZWEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED4KDAI#issuecomment-553165185, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABTDQOBWUGRXME55QOFM6N3QTM3W3ANCNFSM4JLUZWEA.

WilkAndy commented 4 years ago

Hey @nickbeeton . I didn't get those png images. Perhaps they didn't link properly to github? You can just send by email if you want.

WilkAndy commented 4 years ago

Hi @nickbeeton . I've really done all i can with the code. I'm not confident about the code in fun and only a little confident about setInternalParameters. One thing i can't see is any hybridisation between species. Perhaps we're not considering that for now.

I'm now going to look at the data you've put on /scratch/projects/mozzie

nickbeeton commented 4 years ago

No worries, thanks Andy, I’m working on making at least one or two tests now (with one age class and species), and will build hybridisation into the code then start testing that.

Let me know if you have any issues with the data.

Nick

Dr Nick Beeton Biological Modeller | Data61 Ecological and Environmental Risk Assessment (DEERA) team CSIRO | Data61 E nick.beeton@data61.csiro.au T +61 3 6232 5318 3 Castray Esplanade, Battery Point TAS 7004 https://www.data61.csiro.au/en/Our-Research/Programs-and-Facilities/Analytics-and-decision-sciences/DEERA

CSIRO acknowledges the Traditional Owners of the lands that we live and work on across Australia and pays its respect to Elders past and present.

PLEASE NOTE The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference. Please consider the environment before printing this email.

[Data61 | CSIRO logo]

From: Andy Wilkins notifications@github.com Sent: Wednesday, 13 November 2019 2:19 PM To: WilkAndy/mozzie mozzie@noreply.github.com Cc: nickbeeton nick.beeton@utas.edu.au; Mention mention@noreply.github.com Subject: Re: [WilkAndy/mozzie] include mosquito ODE system (#25)

Hi @nickbeetonhttps://github.com/nickbeeton . I've really done all i can with the code. I'm not confident about the code in fun and only a little confident about setInternalParameters. One thing i can't see is any hybridisation between species. Perhaps we're not considering that for now.

I'm now going to look at the data you've put on /scratch/projects/mozzie

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/WilkAndy/mozzie/issues/25?email_source=notifications&email_token=ABTDQOEVQCBTRO6V25SZ67TQTNW3ZA5CNFSM4JLUZWEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED4XSJQ#issuecomment-553220390, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABTDQOEPFDPFKOUG47IH673QTNW3ZANCNFSM4JLUZWEA.

WilkAndy commented 4 years ago

@nickbeeton , make sure you "git push" - otherwise you'll get tonnes of conflicts with my zillions of tiny pushes!

WilkAndy commented 4 years ago

@nickbeeton about the data: i don't have to worry about "africa rain.zip", right? All i care about is carrying capacity.

nickbeeton commented 4 years ago

Oh yeah, ignore the name, the data is all carrying capacity. In fact feel free to change the name because every time I see the filename I get Toto stuck in my head.

Nick

Dr Nick Beeton Biological Modeller | Data61 Ecological and Environmental Risk Assessment (DEERA) team CSIRO | Data61 E nick.beeton@data61.csiro.au T +61 3 6232 5318 3 Castray Esplanade, Battery Point TAS 7004 https://www.data61.csiro.au/en/Our-Research/Programs-and-Facilities/Analytics-and-decision-sciences/DEERA

CSIRO acknowledges the Traditional Owners of the lands that we live and work on across Australia and pays its respect to Elders past and present.

PLEASE NOTE The information contained in this email may be confidential or privileged. Any unauthorised use or disclosure is prohibited. If you have received this email in error, please delete it immediately and notify the sender by return email. Thank you. To the extent permitted by law, CSIRO does not represent, warrant and/or guarantee that the integrity of this communication has been maintained or that the communication is free of errors, virus, interception or interference. Please consider the environment before printing this email.

[Data61 | CSIRO logo]

From: Andy Wilkins notifications@github.com Sent: Wednesday, 13 November 2019 2:24 PM To: WilkAndy/mozzie mozzie@noreply.github.com Cc: nickbeeton nick.beeton@utas.edu.au; Mention mention@noreply.github.com Subject: Re: [WilkAndy/mozzie] include mosquito ODE system (#25)

@nickbeetonhttps://github.com/nickbeeton about the data: i don't have to worry about "africa rain.ziphttp://rain.zip", right? All i care about is carrying capacity.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/WilkAndy/mozzie/issues/25?email_source=notifications&email_token=ABTDQOG6WJ5A365HOK4BDRTQTNXODA5CNFSM4JLUZWEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED4X22I#issuecomment-553221481, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABTDQODFMOE6KCXDPVFPWIDQTNXODANCNFSM4JLUZWEA.

WilkAndy commented 4 years ago

Hey @nickbeeton , There is a bug inCellDynamicsMosquito - haven't got time to fix now - but don't use.

WilkAndy commented 4 months ago

Done, with above tests and so fort