NYC example with plot --- works with 1e3 nodes; serious KeyError

[lubo93]

The example "simulate_simple_epidemic_NYC.py" now contains the relevant NYC case data, daily simulation data averages, and further plots (e.g., reproduction number evolution).

The fast_contact_simulator works well. Probably we have to go back to my original implementation where I simulated only one diurnal cycle (it's useful for system sizes N > 1e3). The current (continuous) contact simulation seems to be relatively slow.

For the 1e4 network, we again get a KeyError:

It could be the same issue that @glwagner and I discussed on Friday.

[glwagner]

@lubo93 I am going to write an EpidemicSimulator tool. I'll include a flag for "reusing" a diurnal cycle of contacts.

[glwagner]

The KeyError occurs because the node numbers do not increase monotonically from 0 to population-1.

[glwagner]

It looks like the contact network has an "empty" node with ID 10017:

In [7]: contact_network.nodes()[10017]                                                                                                                                                       
Out[7]: {}

The reason is that a node is missing: the code

In [14]: for i in range(10018): 
    ...:     contact_network.nodes()[i] 
    ...:     print(i) 

produces

6124
6125
6126
6127
6128
6129
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-13-aced8b49c5c4> in <module>
      1 for i in range(10018):
----> 2     contact_network.nodes()[i]
      3     print(i)
      4 
      5 

~/opt/anaconda3/envs/risknet/lib/python3.6/site-packages/networkx/classes/reportviews.py in __getitem__(self, n)
    176 
    177     def __getitem__(self, n):
--> 178         return self._nodes[n]
    179 
    180     # Set methods

KeyError: 6130

In [14]:

Much of the code assumes that the nodes increase from 0 to population-1 with no missing nodes.

If we want to handle missing nodes, we have to refactor a few things. This is fine, but should be tackled in a future PR.

@lubo93 please inform whether it is important that we have this functionality.

[glwagner]

Here's a snippet of code to reproduce what I've written (this assumes you are in /examples):

import os, sys; sys.path.append(os.path.join(".."))
import networkx as nx
from epiforecast.scenarios import load_edges

edges = load_edges(os.path.join('..', 'data', 'networks', 'edge_list_SBM_1e4.txt')) 

contact_network = nx.Graph()
contact_network.add_edges_from(edges)

for i in range(len(contact_network)): 
    try: 
        node_info = contact_network[i] 
    except: 
        print("Node {} does not exist!".format(i)) 
        raise 

Hopefully this produces

Node 6130 does not exist!
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-11-b687de39c16a> in <module>
     10 for i in range(len(contact_network)):
     11     try:
---> 12         node_info = contact_network[i]
     13     except:
     14         print("Node {} does not exist!".format(i))

~/opt/anaconda3/envs/risknet/lib/python3.6/site-packages/networkx/classes/graph.py in __getitem__(self, n)
    473         AtlasView({1: {}})
    474         """
--> 475         return self.adj[n]
    476 
    477     def add_node(self, node_for_adding, **attr):

~/opt/anaconda3/envs/risknet/lib/python3.6/site-packages/networkx/classes/coreviews.py in __getitem__(self, name)
     79 
     80     def __getitem__(self, name):
---> 81         return AtlasView(self._atlas[name])
     82 
     83     def copy(self):

KeyError: 6130

[lubo93]

Ok, let me check the numbering again. I will correct our networks.

In general, we should use G.nodes() as our node iteration object. This would require some more code revision as far as I remember.

[glwagner]

In general, we should use G.nodes() as our node iteration object. This would require some more code revision as far as I remember.

This is fine, but does require some refactoring. We should do this as soon as possible. Let's discuss on issue #62.