Memory leak issue fixed in distributed version but another found in multiprocessing version
Minor issue with single processing version
Existing distribution strategies may be considered and used in the experimentation stage. We need to discuss them. We may consider others too.
Biggest bottleneck is the communication overhead related to assigning work to the workers. This exists in both the itinerary and contact network. However, it is more evident in the itinerary because the data is quite large.
We may meet on a Saturday to discuss potential ways to optimise the itinerary (for e.g. data structures being used, data being retained, methods being used, etc). Call Prof Abela in the afternoon to see whether to meet this Saturday or next Saturday.
As a potential alternative distribution strategy we may use Dask or Dask on Ray to use the concept of a shared memory space for each node, which is shared by all the workers.
Re-iterated the concept that epidemiological evaluation is out-of-scope. Expressing the complexity of the epidemiological simulation is important nonetheless. This is something that we will focus on when working on the storyboard.
Use Overleaf (latex for writeup) and draw.io / PowerPoint (for diagrams)
Fix issues with load balancing (with load balancing it is slower than without!). Also implement ordering by slowest node first
Implement checkpointing - store every data structure at the end of every day (which may allow revising contact tracing, if desired)
Epidemiological DataFrame by day. To check the parameters being maintained and include all relevant data. (similar to timings / memory CSVs)
To see whether it is possible to hold the information as to how many cells were visited by each agent on each day (a bit difficult to represent this given 500k agents & more, so maybe an average is more possible)
Need to come up with 3-5 scenarios, e.g. herd immunity (no interventions/limitations), conservative (strict), very open (not-strict).
Need to revise the dynamics parameters and what they trigger exactly. Need to communicate this information with John so he approves them as enough or not (some work has been done in this regard but I don't think they were properly communicated with Prof Abela). Particularly we have mentioned closing airports and schools - this level needs to be implemented, at the very least.
Later on we will meet up and create a story board (sections are as follows: intro, background, literature review, methodology, implementation, experimentation & results, evaluation, conclusion & future work, appendix)
Points discussed during meeting
Pertinent issues