Closed ethho closed 2 years ago
@kellypierce summary:
excellent. here's my revised math: i = {local source vertices} j = i k = {all local and contextual destination vertices} pr(travel source i -> destination k) = n_ik/n_i pr(from source j and in destination k) = n_jk/n_k pr(someone from i contacts someone from j in destination k) = pr(travel i -> k) pr(j in k) = (n_ik/n_i) (n_jk/n_k) ... with that definition, sources are either i or j and destination is always k
First pass method of travel partitioning using DataArrays instead of DataFrame/Dataset:
Tested using travel10.csv
in new pytest:
travel9
and travel8
destination_type=contextual
travel11
date
s passed to get_travel_da
contacts.csv
)Single test case of travel9.csv
failing due to handling of destination_type
Correct handling of destination_type
Currently, still runs both the Dask/pandas partitioning routine:
...as well as the new xarray routine:
Three-process Partition pipeline does partitioning in xarray:
As of fdffc7d, no destination_type
column is read from the travel patterns CSV file:
travel.csv
that we can push and use for testsUsed the following script to generate a dummy tests/data/travel_pat0.csv
:
import pandas as pd
import numpy as np
rng = np.random.default_rng()
df = pd.read_csv('./data/20200311_travel.csv')
rand_n = np.array([rng.random() * 2 * row for row in df['n']])
rng.shuffle(rand_n)
df['n'] = rand_n
df.to_csv('./tests/data/travel_pat0.csv', index=False)
df.head()
Overdue for clean up. Editing file by file:
TAGS = ('deprecated',)
or simply delete themTAGS
annotations for models and processes
Fix #48