cannot process Odense

[gboeing]

I have created/merged PRs #8, #9, and #10 to patch code and configs to get Odense processing working. However, I'm hitting a scikit-learn error after pop/intersection density calculation, when it tries to do POI access calculations. Details documented below.

Steps to reproduce:

1. If you have forked the repo before cloning a local copy to your machine, run git pull upstream master to update your copy with my updates.
2. Download the Odense data files that @carlhiggs shared and place them in the /process/data folder.
3. Run docker pull gboeing/global-indicators:latest
4. Run docker run --rm -it -v "%cd%":/home/jovyan/work gboeing/global-indicators /bin/bash if you're on Win, or if you're on Mac/Linux run docker run --rm -it -v "$PWD":/home/jovyan/work gboeing/global-indicators /bin/bash
5. Change directories to the process folder
6. Run python sv_sp.py odense.json true

I get the following error after a couple hours of processing:

(base) root@7efec08f0f5d:/home/jovyan/work/process# python sv_sp.py odense.json true
Start to process city: odense
Start to reproject network
/opt/conda/lib/python3.8/site-packages/pyproj/crs.py:77: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  return _prepare_from_string(" ".join(pjargs))
/opt/conda/lib/python3.8/site-packages/pyproj/crs.py:77: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  return _prepare_from_string(" ".join(pjargs))
Start to calculate average poplulation and intersection density.
0 / 50118
1000 / 50118
2000 / 50118
3000 / 50118
3100 / 50118
1100 / 50118
100 / 50118
2100 / 50118
3200 / 50118
3300 / 50118
1200 / 50118
2200 / 50118
3400 / 50118
200 / 50118
3500 / 50118
2300 / 50118
3600 / 50118
1300 / 50118
300 / 50118
3700 / 50118
3800 / 50118
2400 / 50118
1400 / 50118
3900 / 50118
400 / 50118
4000 / 50118
1500 / 50118
4100 / 50118
2500 / 50118
500 / 50118
4200 / 50118
1600 / 50118
2600 / 50118
4300 / 50118
600 / 50118
1700 / 50118
4400 / 50118
2700 / 50118
700 / 50118
1800 / 50118
2800 / 50118
4500 / 50118
800 / 50118
2900 / 50118
1900 / 50118
4600 / 50118
5000 / 50118
4700 / 50118
900 / 50118
5100 / 50118
6000 / 50118
4800 / 50118
6100 / 50118
7000 / 50118
5200 / 50118
4900 / 50118
6200 / 50118
7100 / 50118
8000 / 50118
5300 / 50118
7200 / 50118
6300 / 50118
7300 / 50118
8100 / 50118
7400 / 50118
5400 / 50118
7500 / 50118
6400 / 50118
8200 / 50118
6500 / 50118
5500 / 50118
7600 / 50118
8300 / 50118
7700 / 50118
6600 / 50118
8400 / 50118
5600 / 50118
8500 / 50118
7800 / 50118
6700 / 50118
8600 / 50118
6800 / 50118
5700 / 50118
7900 / 50118
8700 / 50118
6900 / 50118
5800 / 50118
9000 / 50118
8800 / 50118
10000 / 50118
5900 / 50118
8900 / 50118
10100 / 50118
9100 / 50118
10200 / 50118
11000 / 50118
9200 / 50118
12000 / 50118
10300 / 50118
11100 / 50118
12100 / 50118
9300 / 50118
10400 / 50118
11200 / 50118
11300 / 50118
10500 / 50118
12200 / 50118
11400 / 50118
9400 / 50118
10600 / 50118
11500 / 50118
10700 / 50118
11600 / 50118
12300 / 50118
9500 / 50118
10800 / 50118
9600 / 50118
11700 / 50118
12400 / 50118
10900 / 50118
9700 / 50118
11800 / 50118
11900 / 50118
12500 / 50118
9800 / 50118
13000 / 50118
12600 / 50118
9900 / 50118
13100 / 50118
14000 / 50118
12700 / 50118
15000 / 50118
13200 / 50118
14100 / 50118
12800 / 50118
15100 / 50118
14200 / 50118
13300 / 50118
12900 / 50118
14300 / 50118
15200 / 50118
14400 / 50118
16000 / 50118
13400 / 50118
15300 / 50118
16100 / 50118
14500 / 50118
16200 / 50118
16300 / 50118
15400 / 50118
16400 / 50118
16500 / 50118
14600 / 50118
15500 / 50118
13500 / 50118
16600 / 50118
15600 / 50118
14700 / 50118
16700 / 50118
16800 / 50118
13600 / 50118
16900 / 50118
15700 / 50118
14800 / 50118
15800 / 50118
17000 / 50118
14900 / 50118
13700 / 50118
18000 / 50118
15900 / 50118
18100 / 50118
13800 / 50118
19000 / 50118
18200 / 50118
17100 / 50118
19100 / 50118
18300 / 50118
13900 / 50118
19200 / 50118
18400 / 50118
19300 / 50118
18500 / 50118
17200 / 50118
20000 / 50118
18600 / 50118
19400 / 50118
17300 / 50118
18700 / 50118
19500 / 50118
18800 / 50118
19600 / 50118
17400 / 50118
18900 / 50118
20100 / 50118
21000 / 50118
19700 / 50118
21100 / 50118
21200 / 50118
19800 / 50118
20200 / 50118
19900 / 50118
21300 / 50118
17500 / 50118
21400 / 50118
20300 / 50118
17600 / 50118
22000 / 50118
17700 / 50118
20400 / 50118
21500 / 50118
22100 / 50118
20500 / 50118
17800 / 50118
20600 / 50118
21600 / 50118
22200 / 50118
17900 / 50118
20700 / 50118
20800 / 50118
22300 / 50118
20900 / 50118
21700 / 50118
22400 / 50118
23000 / 50118
24000 / 50118
24100 / 50118
23100 / 50118
21800 / 50118
24200 / 50118
24300 / 50118
24400 / 50118
23200 / 50118
24500 / 50118
21900 / 50118
24600 / 50118
23300 / 50118
23400 / 50118
24700 / 50118
22500 / 50118
23500 / 50118
25000 / 50118
23600 / 50118
24800 / 50118
23700 / 50118
23800 / 50118
23900 / 50118
24900 / 50118
26000 / 50118
22600 / 50118
25100 / 50118
26100 / 50118
25200 / 50118
26200 / 50118
27000 / 50118
25300 / 50118
26300 / 50118
27100 / 50118
25400 / 50118
22700 / 50118
25500 / 50118
26400 / 50118
25600 / 50118
25700 / 50118
27200 / 50118
26500 / 50118
25800 / 50118
26600 / 50118
27300 / 50118
25900 / 50118
26700 / 50118
27400 / 50118
28000 / 50118
26800 / 50118
26900 / 50118
28100 / 50118
29000 / 50118
28200 / 50118
27500 / 50118
22800 / 50118
28300 / 50118
22900 / 50118
27600 / 50118
30000 / 50118
27700 / 50118
28400 / 50118
29100 / 50118
28500 / 50118
29200 / 50118
27800 / 50118
30100 / 50118
28600 / 50118
29300 / 50118
27900 / 50118
29400 / 50118
29500 / 50118
31000 / 50118
28700 / 50118
28800 / 50118
29600 / 50118
29700 / 50118
30200 / 50118
29800 / 50118
28900 / 50118
32000 / 50118
29900 / 50118
30300 / 50118
31100 / 50118
33000 / 50118
30400 / 50118
33100 / 50118
30500 / 50118
32100 / 50118
31200 / 50118
33200 / 50118
30600 / 50118
33300 / 50118
31300 / 50118
33400 / 50118
32200 / 50118
33500 / 50118
31400 / 50118
33600 / 50118
31500 / 50118
30700 / 50118
33700 / 50118
32300 / 50118
31600 / 50118
33800 / 50118
30800 / 50118
32400 / 50118
33900 / 50118
32500 / 50118
31700 / 50118
32600 / 50118
34000 / 50118
32700 / 50118
31800 / 50118
32800 / 50118
30900 / 50118
34100 / 50118
32900 / 50118
31900 / 50118
34200 / 50118
35000 / 50118
34300 / 50118
36000 / 50118
37000 / 50118
34400 / 50118
36100 / 50118
34500 / 50118
35100 / 50118
34600 / 50118
37100 / 50118
36200 / 50118
35200 / 50118
36300 / 50118
37200 / 50118
34700 / 50118
35300 / 50118
36400 / 50118
37300 / 50118
34800 / 50118
36500 / 50118
37400 / 50118
35400 / 50118
34900 / 50118
37500 / 50118
36600 / 50118
38000 / 50118
37600 / 50118
35500 / 50118
37700 / 50118
35600 / 50118
36700 / 50118
38100 / 50118
37800 / 50118
38200 / 50118
35700 / 50118
37900 / 50118
36800 / 50118
35800 / 50118
38300 / 50118
35900 / 50118
39000 / 50118
36900 / 50118
40000 / 50118
38400 / 50118
39100 / 50118
40100 / 50118
41000 / 50118
38500 / 50118
41100 / 50118
39200 / 50118
40200 / 50118
39300 / 50118
40300 / 50118
41200 / 50118
39400 / 50118
40400 / 50118
38600 / 50118
41300 / 50118
39500 / 50118
40500 / 50118
38700 / 50118
39600 / 50118
41400 / 50118
38800 / 50118
39700 / 50118
40600 / 50118
41500 / 50118
38900 / 50118
39800 / 50118
40700 / 50118
41600 / 50118
42000 / 50118
39900 / 50118
40800 / 50118
41700 / 50118
40900 / 50118
42100 / 50118
41800 / 50118
43000 / 50118
44000 / 50118
42200 / 50118
43100 / 50118
44100 / 50118
41900 / 50118
43200 / 50118
44200 / 50118
42300 / 50118
45000 / 50118
44300 / 50118
43300 / 50118
45100 / 50118
44400 / 50118
42400 / 50118
45200 / 50118
43400 / 50118
44500 / 50118
43500 / 50118
42500 / 50118
45300 / 50118
44600 / 50118
43600 / 50118
42600 / 50118
44700 / 50118
45400 / 50118
42700 / 50118
45500 / 50118
44800 / 50118
43700 / 50118
42800 / 50118
45600 / 50118
42900 / 50118
44900 / 50118
43800 / 50118
46000 / 50118
45700 / 50118
46100 / 50118
43900 / 50118
47000 / 50118
46200 / 50118
45800 / 50118
47100 / 50118
48000 / 50118
46300 / 50118
48100 / 50118
47200 / 50118
47300 / 50118
45900 / 50118
48200 / 50118
46400 / 50118
47400 / 50118
48300 / 50118
49000 / 50118
48400 / 50118
46500 / 50118
47500 / 50118
48500 / 50118
49100 / 50118
47600 / 50118
46600 / 50118
46700 / 50118
48600 / 50118
49200 / 50118
46800 / 50118
47700 / 50118
49300 / 50118
46900 / 50118
48700 / 50118
47800 / 50118
49400 / 50118
48800 / 50118
47900 / 50118
50000 / 50118
50100 / 50118
48900 / 50118
49500 / 50118
49600 / 50118
49700 / 50118
49800 / 50118
49900 / 50118
The time to finish average pop and intersection density is: 6921.293860673904
Start to create hex_id for sample points
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 4 threads.
Setting CH node vector of size 50118
Setting CH edge vector of size 124297
Range graph removed 123314 edges of 248594
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Traceback (most recent call last):
  File "sv_sp.py", line 151, in <module>
    gdf_poi_dist1 = sss.cal_dist2poi(gdf_poi1, distance, net, *(names1))
  File "/home/jovyan/work/process/sv_setup_sp.py", line 114, in cal_dist2poi
    network.set_pois(x[0], distance, 1,
  File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 551, in set_pois
    node_ids = self.get_node_ids(x_col, y_col)
  File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 388, in get_node_ids
    distances, indexes = self.kdtree.query(xys.values)
  File "sklearn/neighbors/_binary_tree.pxi", line 1316, in sklearn.neighbors._kd_tree.BinaryTree.query
  File "/opt/conda/lib/python3.8/site-packages/sklearn/utils/validation.py", line 583, in check_array
    raise ValueError("Found array with %d sample(s) (shape=%s) while a"
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required.

[shiqin-liu]

Thank you, Geoff! I was testing the scripts for odense this morning as well, and it seems I was able to process and get the hex grid output successfully. I will try some other cities and keep you updated if there is an issue.

Here is the processing:

Start to process city: odense
Start to reproject network
/opt/conda/lib/python3.8/site-packages/pyproj/crs.py:77: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method.
  return _prepare_from_string(" ".join(pjargs))
Start to calculate average poplulation and intersection density.
0 / 22668
1000 / 22668
2000 / 22668
3000 / 22668
2100 / 22668
1100 / 22668
3100 / 22668
100 / 22668
2200 / 22668
3200 / 22668
1200 / 22668
200 / 22668
1300 / 22668
3300 / 22668
2300 / 22668
300 / 22668
3400 / 22668
1400 / 22668
400 / 22668
2400 / 22668
3500 / 22668
3600 / 22668
1500 / 22668
500 / 22668
2500 / 22668
3700 / 22668
1600 / 22668
600 / 22668
2600 / 22668
3800 / 22668
1700 / 22668
700 / 22668
2700 / 22668
1800 / 22668
2800 / 22668
3900 / 22668
800 / 22668
2900 / 22668
1900 / 22668
4000 / 22668
900 / 22668
5000 / 22668
6000 / 22668
4100 / 22668
5100 / 22668
7000 / 22668
6100 / 22668
4200 / 22668
7100 / 22668
5200 / 22668
4300 / 22668
6200 / 22668
7200 / 22668
5300 / 22668
7300 / 22668
6300 / 22668
5400 / 22668
4400 / 22668
7400 / 22668
5500 / 22668
7500 / 22668
6400 / 22668
4500 / 22668
7600 / 22668
5600 / 22668
6500 / 22668
4600 / 22668
7700 / 22668
7800 / 22668
4700 / 22668
6600 / 22668
5700 / 22668
4800 / 22668
7900 / 22668
6700 / 22668
8000 / 22668
5800 / 22668
4900 / 22668
6800 / 22668
5900 / 22668
8100 / 22668
6900 / 22668
9000 / 22668
8200 / 22668
10000 / 22668
9100 / 22668
8300 / 22668
11000 / 22668
10100 / 22668
9200 / 22668
8400 / 22668
10200 / 22668
9300 / 22668
11100 / 22668
8500 / 22668
9400 / 22668
10300 / 22668
10400 / 22668
11200 / 22668
9500 / 22668
8600 / 22668
10500 / 22668
9600 / 22668
11300 / 22668
8700 / 22668
10600 / 22668
11400 / 22668
9700 / 22668
8800 / 22668
11500 / 22668
10700 / 22668
9800 / 22668
8900 / 22668
10800 / 22668
12000 / 22668
9900 / 22668
11600 / 22668
12100 / 22668
10900 / 22668
12200 / 22668
13000 / 22668
11700 / 22668
12300 / 22668
14000 / 22668
12400 / 22668
11800 / 22668
11900 / 22668
13100 / 22668
14100 / 22668
12500 / 22668
15000 / 22668
13200 / 22668
14200 / 22668
12600 / 22668
15100 / 22668
13300 / 22668
12700 / 22668
14300 / 22668
13400 / 22668
15200 / 22668
12800 / 22668
14400 / 22668
13500 / 22668
14500 / 22668
15300 / 22668
12900 / 22668
13600 / 22668
15400 / 22668
16000 / 22668
14600 / 22668
13700 / 22668
15500 / 22668
16100 / 22668
13800 / 22668
14700 / 22668
13900 / 22668
15600 / 22668
16200 / 22668
14800 / 22668
17000 / 22668
15700 / 22668
14900 / 22668
16300 / 22668
17100 / 22668
15800 / 22668
18000 / 22668
16400 / 22668
15900 / 22668
17200 / 22668
16500 / 22668
17300 / 22668
18100 / 22668
16600 / 22668
19000 / 22668
17400 / 22668
18200 / 22668
19100 / 22668
16700 / 22668
17500 / 22668
19200 / 22668
16800 / 22668
18300 / 22668
17600 / 22668
16900 / 22668
18400 / 22668
19300 / 22668
17700 / 22668
20000 / 22668
17800 / 22668
18500 / 22668
19400 / 22668
17900 / 22668
20100 / 22668
18600 / 22668
21000 / 22668
19500 / 22668
18700 / 22668
20200 / 22668
21100 / 22668
18800 / 22668
19600 / 22668
21200 / 22668
20300 / 22668
19700 / 22668
18900 / 22668
21300 / 22668
22000 / 22668
19800 / 22668
20400 / 22668
21400 / 22668
22100 / 22668
19900 / 22668
20500 / 22668
21500 / 22668
22200 / 22668
20600 / 22668
21600 / 22668
20700 / 22668
21700 / 22668
22300 / 22668
21800 / 22668
22400 / 22668
20800 / 22668
21900 / 22668
22500 / 22668
20900 / 22668
22600 / 22668
The time to finish average pop and intersection density is: 3807.905283689499
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 4 threads.
Setting CH node vector of size 22668
Setting CH edge vector of size 58078
Range graph removed 56486 edges of 116156
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Total time is : 1.07 hours or 3849.24 seconds

[gboeing]

@shiqin-liu were you able to do this using my steps 1-6 above?

Your output (including graph size) look so different from mine, I want to make sure we're using the same code, data, and steps to reproduce.

[shiqin-liu]

For steps 3: I do docker pull gboeing/global-indicators. All else, I follow the same steps. But another note, I was actually using the odense data file that was prepared for SV earlier, but not the file that Carl just updated on Cloudstor. That may be the reason... I will try the new file and see what will happen

[gboeing]

Try docker pull gboeing/global-indicators:latest just to double-check that you're not using the old image.

[shiqin-liu]

I used the latest image and the new file, but this time I was only able to run the scripts partially. I tried a few times and left the scripts processed overnight, but it stoped here (as shown below) without even showing the error message. Not sure about the problem yet... but here is where I am at:

Start to process city: odense
Start to read network.
Start to calculate average poplulation and intersection density.
2000 / 50118
3000 / 50118
3100 / 50118
2100 / 50118
3200 / 50118
3300 / 50118
3400 / 50118
2200 / 50118
3500 / 50118
2300 / 50118
3600 / 50118
3700 / 50118
3800 / 50118
2400 / 50118
3900 / 50118
2500 / 50118
2600 / 50118
2700 / 50118
2800 / 50118
2900 / 50118

[shiqin-liu]

Here is the message when I interrupt the kernel:

2800 / 50118
2900 / 50118
^CProcess ForkPoolWorker-4:
Process ForkPoolWorker-3:
Process ForkPoolWorker-10:
Process ForkPoolWorker-9:
Traceback (most recent call last):
  File "sv_sp.py", line 80, in <module>
Traceback (most recent call last):
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/opt/conda/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/opt/conda/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
    result_objects = pool.starmap_async(
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 762, in get
    self.wait(timeout)
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 759, in wait
    self._event.wait(timeout)
  File "/opt/conda/lib/python3.8/threading.py", line 558, in wait
    signaled = self._cond.wait(timeout)
  File "/opt/conda/lib/python3.8/threading.py", line 302, in wait
    waiter.acquire()
KeyboardInterrupt
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/opt/conda/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/conda/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/lib/python3.8/multiprocessing/pool.py", line 114, in worker
    task = get()
  File "/opt/conda/lib/python3.8/multiprocessing/queues.py", line 355, in get
    with self._rlock:
  File "/opt/conda/lib/python3.8/multiprocessing/synchronize.py", line 95, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt

[gboeing]

Multiprocessing makes debugging tricky. Try running with false instead of true. And try to allocate at least 4gb of memory in docker's settings in case it's a memory issue...

[shiqin-liu]

Thanks, Geoff! After setting the multiprocessing to False, I was able to run the scripts, but unfortunately, I got the same error message as yours after a few hours processing.

Here is the end of the processing results:

49800 / 50118
49900 / 50118
50000 / 50118
50100 / 50118
The time to finish average pop and intersection density is: 19641.678276777267
Start to create hex_id for sample points
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 4 threads.
Setting CH node vector of size 50118
Setting CH edge vector of size 124297
Range graph removed 123314 edges of 248594
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Traceback (most recent call last):
  File "sv_sp.py", line 151, in <module>
    gdf_poi_dist1 = sss.cal_dist2poi(gdf_poi1, distance, net, *(names1))
  File "/home/jovyan/work/process/sv_setup_sp.py", line 114, in cal_dist2poi
    network.set_pois(x[0], distance, 1,
  File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 551, in set_pois
    node_ids = self.get_node_ids(x_col, y_col)
  File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 388, in get_node_ids
    distances, indexes = self.kdtree.query(xys.values)
  File "sklearn/neighbors/_binary_tree.pxi", line 1316, in sklearn.neighbors._kd_tree.BinaryTree.query
  File "/opt/conda/lib/python3.8/site-packages/sklearn/utils/validation.py", line 583, in check_array
    raise ValueError("Found array with %d sample(s) (shape=%s) while a"
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required.

[gboeing]

Yeah same problem. @carlhiggs same for you?

[gboeing]

FWIW I can't get it running with SV's zacwang/global-indicator docker image either (using the Odense files Carl shared on Cloudstor). Similar error:

>>> docker run --rm -it -v "%cd%":/home/jovyan/work zacwang/global-indicator /bin/bash
jovyan@e08bf635a3ce:~/work$ python sv_sp.py odense.json false
Start to process city: odense
Start to reproject network
Start to calculate average poplulation and intersection density.
The time to finish average pop and intersection density is: 200.1857614517212
Start to create hex_id for sample points
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 4 threads.
Setting CH node vector of size 50118
Setting CH edge vector of size 124297
Range graph removed 123314 edges of 248594
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Traceback (most recent call last):
  File "sv_sp.py", line 151, in <module>
    gdf_poi_dist1 = sss.cal_dist2poi(gdf_poi1, distance, net, *(names1))
  File "/home/jovyan/work/sv_setup_sp.py", line 116, in cal_dist2poi
    gdf_poi[gdf_poi['dest_name_full'] == x[0]]['y'])
  File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 551, in set_pois
    node_ids = self.get_node_ids(x_col, y_col)
  File "/opt/conda/lib/python3.7/site-packages/pandana/network.py", line 388, in get_node_ids
    distances, indexes = self.kdtree.query(xys.values)
  File "sklearn/neighbors/binary_tree.pxi", line 1314, in sklearn.neighbors.kd_tree.BinaryTree.query
  File "/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py", line 550, in check_array
    context))
ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required.

[thuanguy]

I am running the code for hanoi. It has been running for a whole day and still running. I wonder if that's normal.

[gboeing]

@thuanguy Hanoi's a big city, so that's entirely possible. Can you try running Odense to see what you get? It's smaller and should be faster.

[thuanguy]

112200 / 112402 112300 / 112402 112400 / 112402 The time to finish average pop and intersection density is: 267166.51315426826 Start to create hex_id for sample points Start to calculate assessbility to POIs. Generating contraction hierarchies with 2 threads. Setting CH node vector of size 112402 Setting CH edge vector of size 275045 Range graph removed 261546 edges of 550090 . 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100% Traceback (most recent call last): File "sv_sp.py", line 152, in gdf_poi_dist1 = sss.cal_dist2poi(gdf_poi1, distance, net, *(names1)) File "/home/jovyan/work/process/sv_setup_sp.py", line 114, in cal_dist2poi network.set_pois(x[0], distance, 1, File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 551, in set_pois node_ids = self.get_node_ids(x_col, y_col) File "/opt/conda/lib/python3.8/site-packages/pandana/network.py", line 388, in get_node_ids distances, indexes = self.kdtree.query(xys.values) File "sklearn/neighbors/_binary_tree.pxi", line 1316, in sklearn.neighbors._kd_tree.BinaryTree.query File "/opt/conda/lib/python3.8/site-packages/sklearn/utils/validation.py", line 583, in check_array raise ValueError("Found array with %d sample(s) (shape=%s) while a" ValueError: Found array with 0 sample(s) (shape=(0, 2)) while a minimum of 1 is required.

Finally got the first step done. Here is what I got.

[duelran]

Using Zac's original code from the sv branch and the original rather than the updated input data, I was able to get both Odense and Phoenix working successfully today. As per our email discussion with the SV team, it looks like the problems we are having can be traced back to differences in the input files, and by patching the code in the master branch we should be able to get the scripts working again with the slightly different input data.

For reference I had 8 CPUs, 24 GB of RAM and 2GB of swap space on my 2017-era Core i7 laptop.

root@dfc5e24fc57c:~/work# python ./process/sv_sp.py odense.json true
Start to process city: odense
Start to read network.
Start to calculate average poplulation and intersection density.
0 / 22668
1000 / 22668
2000 / 22668
3000 / 22668
4000 / 22668
5000 / 22668
6000 / 22668
7000 / 22668
2100 / 22668
7100 / 22668
1100 / 22668
6100 / 22668
4100 / 22668
5100 / 22668
3100 / 22668
100 / 22668
7200 / 22668
2200 / 22668
6200 / 22668
5200 / 22668
3200 / 22668
4200 / 22668
1200 / 22668
7300 / 22668
200 / 22668
4300 / 22668
5300 / 22668
7400 / 22668
6300 / 22668
3300 / 22668
1300 / 22668
5400 / 22668
2300 / 22668
7500 / 22668
300 / 22668
4400 / 22668
3400 / 22668
6400 / 22668
7600 / 22668
1400 / 22668
5500 / 22668
2400 / 22668
3500 / 22668
400 / 22668
7700 / 22668
4500 / 22668
6500 / 22668
3600 / 22668
1500 / 22668
7800 / 22668
5600 / 22668
2500 / 22668
500 / 22668
3700 / 22668
7900 / 22668
4600 / 22668
6600 / 22668
1600 / 22668
8000 / 22668
6700 / 22668
2600 / 22668
5700 / 22668
600 / 22668
4700 / 22668
3800 / 22668
8100 / 22668
4800 / 22668
6800 / 22668
1700 / 22668
5800 / 22668
8200 / 22668
2700 / 22668
5900 / 22668
700 / 22668
6900 / 22668
4900 / 22668
8300 / 22668
3900 / 22668
2800 / 22668
9000 / 22668
1800 / 22668
10000 / 22668
800 / 22668
2900 / 22668
9100 / 22668
8400 / 22668
11000 / 22668
1900 / 22668
12000 / 22668
10100 / 22668
9200 / 22668
13000 / 22668
12100 / 22668
8500 / 22668
900 / 22668
14000 / 22668
10200 / 22668
9300 / 22668
12200 / 22668
11100 / 22668
13100 / 22668
12300 / 22668
14100 / 22668
8600 / 22668
15000 / 22668
12400 / 22668
9400 / 22668
10300 / 22668
13200 / 22668
10400 / 22668
14200 / 22668
12500 / 22668
9500 / 22668
11200 / 22668
8700 / 22668
15100 / 22668
13300 / 22668
10500 / 22668
14300 / 22668
12600 / 22668
9600 / 22668
8800 / 22668
11300 / 22668
10600 / 22668
13400 / 22668
12700 / 22668
11400 / 22668
15200 / 22668
8900 / 22668
9700 / 22668
14400 / 22668
10700 / 22668
13500 / 22668
16000 / 22668
11500 / 22668
9800 / 22668
10800 / 22668
14500 / 22668
12800 / 22668
15300 / 22668
16100 / 22668
13600 / 22668
9900 / 22668
11600 / 22668
10900 / 22668
15400 / 22668
12900 / 22668
14600 / 22668
13700 / 22668
16200 / 22668
15500 / 22668
17000 / 22668
18000 / 22668
13800 / 22668
19000 / 22668
11700 / 22668
14700 / 22668
16300 / 22668
13900 / 22668
15600 / 22668
11800 / 22668
17100 / 22668
19100 / 22668
18100 / 22668
11900 / 22668
14800 / 22668
20000 / 22668
16400 / 22668
21000 / 22668
15700 / 22668
17200 / 22668
19200 / 22668
16500 / 22668
14900 / 22668
18200 / 22668
20100 / 22668
21100 / 22668
15800 / 22668
16600 / 22668
17300 / 22668
22000 / 22668
19300 / 22668
21200 / 22668
18300 / 22668
15900 / 22668
16700 / 22668
20200 / 22668
17400 / 22668
22100 / 22668
21300 / 22668
18400 / 22668
19400 / 22668
17500 / 22668
16800 / 22668
22200 / 22668
20300 / 22668
21400 / 22668
17600 / 22668
16900 / 22668
18500 / 22668
19500 / 22668
21500 / 22668
20400 / 22668
22300 / 22668
17700 / 22668
21600 / 22668
18600 / 22668
17800 / 22668
19600 / 22668
20500 / 22668
22400 / 22668
17900 / 22668
19700 / 22668
18700 / 22668
21700 / 22668
20600 / 22668
18800 / 22668
22500 / 22668
19800 / 22668
21800 / 22668
18900 / 22668
20700 / 22668
19900 / 22668
21900 / 22668
22600 / 22668
20800 / 22668
20900 / 22668
The time to finish average pop and intersection density is: 2822.5885388851166
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 22668
Setting CH edge vector of size 58078
Range graph removed 56486 edges of 116156
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Total time is : 0.80 hours or 2865.23 seconds
root@dfc5e24fc57c:~/work#
root@dfc5e24fc57c:~/work# python ./process/sv_sp.py phoenix.json true
Start to process city: phoenix
Start to read network.
Start to calculate average poplulation and intersection density.
0 / 135775
1000 / 135775
2000 / 135775
3000 / 135775
4000 / 135775
5000 / 135775
6000 / 135775
7000 / 135775
100 / 135775
1100 / 135775
3100 / 135775
2100 / 135775
4100 / 135775
5100 / 135775
6100 / 135775
7100 / 135775
200 / 135775
1200 / 135775
3200 / 135775
4200 / 135775
2200 / 135775
5200 / 135775
7200 / 135775
1300 / 135775
300 / 135775
6200 / 135775
3300 / 135775
2300 / 135775
4300 / 135775
5300 / 135775
1400 / 135775
7300 / 135775
6300 / 135775
400 / 135775
4400 / 135775
2400 / 135775
3400 / 135775
5400 / 135775
7400 / 135775
1500 / 135775
6400 / 135775
500 / 135775
4500 / 135775
3500 / 135775
2500 / 135775
5500 / 135775
7500 / 135775
1600 / 135775
600 / 135775
4600 / 135775
2600 / 135775
3600 / 135775
6500 / 135775
5600 / 135775
1700 / 135775
4700 / 135775
700 / 135775
7600 / 135775
2700 / 135775
3700 / 135775
5700 / 135775
6600 / 135775
1800 / 135775
4800 / 135775
800 / 135775
2800 / 135775
7700 / 135775
5800 / 135775
3800 / 135775
6700 / 135775
1900 / 135775
4900 / 135775
900 / 135775
2900 / 135775
5900 / 135775
7800 / 135775
3900 / 135775
6800 / 135775
8000 / 135775
9000 / 135775
10000 / 135775
11000 / 135775
12000 / 135775
7900 / 135775
13000 / 135775
6900 / 135775
8100 / 135775
9100 / 135775
10100 / 135775
12100 / 135775
11100 / 135775
14000 / 135775
13100 / 135775
15000 / 135775
8200 / 135775
9200 / 135775
10200 / 135775
11200 / 135775
12200 / 135775
13200 / 135775
14100 / 135775
8300 / 135775
15100 / 135775
9300 / 135775
11300 / 135775
12300 / 135775
10300 / 135775
13300 / 135775
14200 / 135775
15200 / 135775
9400 / 135775
8400 / 135775
11400 / 135775
12400 / 135775
10400 / 135775
13400 / 135775
14300 / 135775
9500 / 135775
15300 / 135775
8500 / 135775
10500 / 135775
12500 / 135775
11500 / 135775
13500 / 135775
14400 / 135775
9600 / 135775
15400 / 135775
12600 / 135775
11600 / 135775
8600 / 135775
10600 / 135775
13600 / 135775
9700 / 135775
14500 / 135775
12700 / 135775
15500 / 135775
8700 / 135775
11700 / 135775
13700 / 135775
10700 / 135775
14600 / 135775
9800 / 135775
12800 / 135775
15600 / 135775
11800 / 135775
8800 / 135775
10800 / 135775
13800 / 135775
14700 / 135775
9900 / 135775
11900 / 135775
15700 / 135775
12900 / 135775
8900 / 135775
10900 / 135775
13900 / 135775
16000 / 135775
14800 / 135775
17000 / 135775
18000 / 135775
15800 / 135775
19000 / 135775
20000 / 135775
21000 / 135775
16100 / 135775
17100 / 135775
14900 / 135775
15900 / 135775
19100 / 135775
18100 / 135775
20100 / 135775
21100 / 135775
16200 / 135775
17200 / 135775
22000 / 135775
23000 / 135775
19200 / 135775
18200 / 135775
20200 / 135775
21200 / 135775
16300 / 135775
17300 / 135775
22100 / 135775
18300 / 135775
20300 / 135775
19300 / 135775
23100 / 135775
21300 / 135775
16400 / 135775
17400 / 135775
22200 / 135775
19400 / 135775
20400 / 135775
23200 / 135775
18400 / 135775
21400 / 135775
16500 / 135775
17500 / 135775
23300 / 135775
22300 / 135775
20500 / 135775
19500 / 135775
21500 / 135775
18500 / 135775
16600 / 135775
17600 / 135775
19600 / 135775
20600 / 135775
21600 / 135775
22400 / 135775
23400 / 135775
18600 / 135775
16700 / 135775
17700 / 135775
21700 / 135775
19700 / 135775
20700 / 135775
22500 / 135775
23500 / 135775
18700 / 135775
16800 / 135775
17800 / 135775
19800 / 135775
21800 / 135775
20800 / 135775
22600 / 135775
23600 / 135775
18800 / 135775
16900 / 135775
17900 / 135775
19900 / 135775
21900 / 135775
22700 / 135775
20900 / 135775
23700 / 135775
24000 / 135775
18900 / 135775
25000 / 135775
26000 / 135775
22800 / 135775
27000 / 135775
28000 / 135775
23800 / 135775
29000 / 135775
24100 / 135775
26100 / 135775
22900 / 135775
27100 / 135775
25100 / 135775
23900 / 135775
28100 / 135775
29100 / 135775
24200 / 135775
26200 / 135775
27200 / 135775
30000 / 135775
25200 / 135775
31000 / 135775
28200 / 135775
29200 / 135775
24300 / 135775
27300 / 135775
26300 / 135775
30100 / 135775
28300 / 135775
25300 / 135775
31100 / 135775
29300 / 135775
24400 / 135775
26400 / 135775
27400 / 135775
30200 / 135775
28400 / 135775
25400 / 135775
31200 / 135775
29400 / 135775
27500 / 135775
24500 / 135775
26500 / 135775
30300 / 135775
28500 / 135775
25500 / 135775
31300 / 135775
29500 / 135775
27600 / 135775
24600 / 135775
26600 / 135775
30400 / 135775
28600 / 135775
31400 / 135775
25600 / 135775
29600 / 135775
27700 / 135775
24700 / 135775
26700 / 135775
30500 / 135775
28700 / 135775
31500 / 135775
25700 / 135775
29700 / 135775
27800 / 135775
24800 / 135775
26800 / 135775
31600 / 135775
30600 / 135775
28800 / 135775
25800 / 135775
29800 / 135775
27900 / 135775
24900 / 135775
30700 / 135775
26900 / 135775
31700 / 135775
28900 / 135775
25900 / 135775
29900 / 135775
32000 / 135775
33000 / 135775
30800 / 135775
31800 / 135775
34000 / 135775
35000 / 135775
36000 / 135775
37000 / 135775
32100 / 135775
31900 / 135775
30900 / 135775
33100 / 135775
34100 / 135775
35100 / 135775
36100 / 135775
37100 / 135775
34200 / 135775
32200 / 135775
33200 / 135775
38000 / 135775
39000 / 135775
35200 / 135775
34300 / 135775
37200 / 135775
36200 / 135775
32300 / 135775
33300 / 135775
38100 / 135775
39100 / 135775
35300 / 135775
34400 / 135775
32400 / 135775
33400 / 135775
36300 / 135775
37300 / 135775
35400 / 135775
38200 / 135775
39200 / 135775
34500 / 135775
32500 / 135775
33500 / 135775
35500 / 135775
36400 / 135775
37400 / 135775
34600 / 135775
39300 / 135775
38300 / 135775
32600 / 135775
35600 / 135775
34700 / 135775
36500 / 135775
37500 / 135775
33600 / 135775
38400 / 135775
39400 / 135775
32700 / 135775
35700 / 135775
33700 / 135775
36600 / 135775
37600 / 135775
34800 / 135775
32800 / 135775
38500 / 135775
39500 / 135775
33800 / 135775
36700 / 135775
37700 / 135775
35800 / 135775
32900 / 135775
33900 / 135775
39600 / 135775
38600 / 135775
34900 / 135775
37800 / 135775
36800 / 135775
40000 / 135775
41000 / 135775
35900 / 135775
38700 / 135775
42000 / 135775
39700 / 135775
36900 / 135775
37900 / 135775
41100 / 135775
40100 / 135775
41200 / 135775
43000 / 135775
38800 / 135775
44000 / 135775
45000 / 135775
40200 / 135775
39800 / 135775
43100 / 135775
41300 / 135775
42100 / 135775
38900 / 135775
45100 / 135775
43200 / 135775
40300 / 135775
41400 / 135775
44100 / 135775
43300 / 135775
39900 / 135775
46000 / 135775
45200 / 135775
40400 / 135775
43400 / 135775
44200 / 135775
42200 / 135775
41500 / 135775
47000 / 135775
40500 / 135775
46100 / 135775
45300 / 135775
43500 / 135775
44300 / 135775
47100 / 135775
46200 / 135775
42300 / 135775
45400 / 135775
41600 / 135775
44400 / 135775
43600 / 135775
40600 / 135775
47200 / 135775
46300 / 135775
44500 / 135775
43700 / 135775
42400 / 135775
45500 / 135775
47300 / 135775
40700 / 135775
44600 / 135775
41700 / 135775
46400 / 135775
42500 / 135775
47400 / 135775
43800 / 135775
40800 / 135775
44700 / 135775
46500 / 135775
45600 / 135775
42600 / 135775
40900 / 135775
44800 / 135775
41800 / 135775
47500 / 135775
43900 / 135775
46600 / 135775
48000 / 135775
45700 / 135775
42700 / 135775
44900 / 135775
47600 / 135775
41900 / 135775
49000 / 135775
48100 / 135775
46700 / 135775
42800 / 135775
50000 / 135775
45800 / 135775
47700 / 135775
42900 / 135775
48200 / 135775
49100 / 135775
50100 / 135775
47800 / 135775
45900 / 135775
51000 / 135775
46800 / 135775
52000 / 135775
50200 / 135775
48300 / 135775
51100 / 135775
47900 / 135775
49200 / 135775
46900 / 135775
53000 / 135775
52100 / 135775
51200 / 135775
49300 / 135775
50300 / 135775
48400 / 135775
54000 / 135775
55000 / 135775
53100 / 135775
52200 / 135775
51300 / 135775
49400 / 135775
50400 / 135775
51400 / 135775
49500 / 135775
50500 / 135775
48500 / 135775
53200 / 135775
52300 / 135775
51500 / 135775
55100 / 135775
49600 / 135775
54100 / 135775
50600 / 135775
49700 / 135775
51600 / 135775
53300 / 135775
52400 / 135775
50700 / 135775
48600 / 135775
55200 / 135775
49800 / 135775
54200 / 135775
51700 / 135775
52500 / 135775
53400 / 135775
48700 / 135775
49900 / 135775
50800 / 135775
55300 / 135775
52600 / 135775
51800 / 135775
54300 / 135775
53500 / 135775
48800 / 135775
56000 / 135775
55400 / 135775
52700 / 135775
51900 / 135775
50900 / 135775
54400 / 135775
53600 / 135775
48900 / 135775
55500 / 135775
56100 / 135775
57000 / 135775
52800 / 135775
58000 / 135775
53700 / 135775
55600 / 135775
54500 / 135775
52900 / 135775
58100 / 135775
59000 / 135775
56200 / 135775
57100 / 135775
53800 / 135775
55700 / 135775
58200 / 135775
60000 / 135775
54600 / 135775
59100 / 135775
58300 / 135775
53900 / 135775
57200 / 135775
56300 / 135775
60100 / 135775
59200 / 135775
58400 / 135775
54700 / 135775
55800 / 135775
57300 / 135775
61000 / 135775
59300 / 135775
58500 / 135775
60200 / 135775
54800 / 135775
56400 / 135775
55900 / 135775
61100 / 135775
57400 / 135775
59400 / 135775
60300 / 135775
58600 / 135775
54900 / 135775
56500 / 135775
61200 / 135775
62000 / 135775
59500 / 135775
58700 / 135775
57500 / 135775
60400 / 135775
61300 / 135775
62100 / 135775
63000 / 135775
59600 / 135775
56600 / 135775
62200 / 135775
58800 / 135775
60500 / 135775
57600 / 135775
61400 / 135775
63100 / 135775
59700 / 135775
57700 / 135775
61500 / 135775
60600 / 135775
58900 / 135775
62300 / 135775
56700 / 135775
57800 / 135775
60700 / 135775
59800 / 135775
61600 / 135775
63200 / 135775
64000 / 135775
62400 / 135775
57900 / 135775
60800 / 135775
56800 / 135775
61700 / 135775
62500 / 135775
59900 / 135775
65000 / 135775
63300 / 135775
64100 / 135775
61800 / 135775
60900 / 135775
63400 / 135775
62600 / 135775
65100 / 135775
56900 / 135775
66000 / 135775
64200 / 135775
67000 / 135775
61900 / 135775
62700 / 135775
63500 / 135775
66100 / 135775
65200 / 135775
68000 / 135775
64300 / 135775
69000 / 135775
62800 / 135775
66200 / 135775
63600 / 135775
67100 / 135775
68100 / 135775
65300 / 135775
62900 / 135775
64400 / 135775
66300 / 135775
63700 / 135775
67200 / 135775
69100 / 135775
70000 / 135775
65400 / 135775
64500 / 135775
68200 / 135775
66400 / 135775
63800 / 135775
67300 / 135775
64600 / 135775
70100 / 135775
66500 / 135775
68300 / 135775
69200 / 135775
65500 / 135775
67400 / 135775
63900 / 135775
64700 / 135775
66600 / 135775
70200 / 135775
69300 / 135775
65600 / 135775
68400 / 135775
71000 / 135775
64800 / 135775
67500 / 135775
69400 / 135775
68500 / 135775
65700 / 135775
66700 / 135775
70300 / 135775
71100 / 135775
64900 / 135775
67600 / 135775
69500 / 135775
65800 / 135775
68600 / 135775
66800 / 135775
69600 / 135775
68700 / 135775
72000 / 135775
71200 / 135775
70400 / 135775
67700 / 135775
65900 / 135775
73000 / 135775
66900 / 135775
68800 / 135775
69700 / 135775
72100 / 135775
71300 / 135775
70500 / 135775
67800 / 135775
68900 / 135775
69800 / 135775
73100 / 135775
67900 / 135775
72200 / 135775
74000 / 135775
75000 / 135775
71400 / 135775
70600 / 135775
76000 / 135775
69900 / 135775
72300 / 135775
75100 / 135775
74100 / 135775
71500 / 135775
73200 / 135775
70700 / 135775
75200 / 135775
77000 / 135775
76100 / 135775
71600 / 135775
72400 / 135775
74200 / 135775
73300 / 135775
70800 / 135775
72500 / 135775
76200 / 135775
74300 / 135775
75300 / 135775
71700 / 135775
77100 / 135775
70900 / 135775
73400 / 135775
77200 / 135775
74400 / 135775
71800 / 135775
72600 / 135775
76300 / 135775
75400 / 135775
78000 / 135775
77300 / 135775
71900 / 135775
74500 / 135775
73500 / 135775
76400 / 135775
75500 / 135775
72700 / 135775
77400 / 135775
79000 / 135775
74600 / 135775
76500 / 135775
73600 / 135775
75600 / 135775
78100 / 135775
77500 / 135775
72800 / 135775
74700 / 135775
79100 / 135775
76600 / 135775
73700 / 135775
75700 / 135775
77600 / 135775
74800 / 135775
72900 / 135775
79200 / 135775
76700 / 135775
77700 / 135775
78200 / 135775
75800 / 135775
73800 / 135775
79300 / 135775
80000 / 135775
77800 / 135775
74900 / 135775
76800 / 135775
75900 / 135775
79400 / 135775
80100 / 135775
81000 / 135775
73900 / 135775
76900 / 135775
77900 / 135775
78300 / 135775
79500 / 135775
82000 / 135775
81100 / 135775
80200 / 135775
79600 / 135775
83000 / 135775
84000 / 135775
82100 / 135775
78400 / 135775
80300 / 135775
81200 / 135775
85000 / 135775
79700 / 135775
84100 / 135775
82200 / 135775
83100 / 135775
80400 / 135775
81300 / 135775
85100 / 135775
79800 / 135775
78500 / 135775
84200 / 135775
80500 / 135775
82300 / 135775
83200 / 135775
81400 / 135775
79900 / 135775
85200 / 135775
80600 / 135775
84300 / 135775
78600 / 135775
83300 / 135775
86000 / 135775
82400 / 135775
81500 / 135775
85300 / 135775
80700 / 135775
84400 / 135775
83400 / 135775
86100 / 135775
82500 / 135775
78700 / 135775
83500 / 135775
81600 / 135775
85400 / 135775
80800 / 135775
84500 / 135775
86200 / 135775
82600 / 135775
83600 / 135775
78800 / 135775
81700 / 135775
85500 / 135775
84600 / 135775
86300 / 135775
80900 / 135775
83700 / 135775
82700 / 135775
81800 / 135775
85600 / 135775
84700 / 135775
86400 / 135775
78900 / 135775
87000 / 135775
83800 / 135775
84800 / 135775
81900 / 135775
82800 / 135775
85700 / 135775
86500 / 135775
83900 / 135775
87100 / 135775
88000 / 135775
84900 / 135775
89000 / 135775
85800 / 135775
86600 / 135775
82900 / 135775
87200 / 135775
90000 / 135775
91000 / 135775
88100 / 135775
87300 / 135775
86700 / 135775
85900 / 135775
88200 / 135775
92000 / 135775
91100 / 135775
89100 / 135775
90100 / 135775
87400 / 135775
86800 / 135775
91200 / 135775
93000 / 135775
92100 / 135775
90200 / 135775
88300 / 135775
89200 / 135775
87500 / 135775
93100 / 135775
92200 / 135775
86900 / 135775
91300 / 135775
90300 / 135775
93200 / 135775
89300 / 135775
87600 / 135775
92300 / 135775
88400 / 135775
93300 / 135775
91400 / 135775
94000 / 135775
90400 / 135775
89400 / 135775
87700 / 135775
93400 / 135775
92400 / 135775
91500 / 135775
89500 / 135775
94100 / 135775
88500 / 135775
90500 / 135775
87800 / 135775
93500 / 135775
92500 / 135775
91600 / 135775
89600 / 135775
94200 / 135775
88600 / 135775
87900 / 135775
93600 / 135775
90600 / 135775
91700 / 135775
92600 / 135775
94300 / 135775
89700 / 135775
93700 / 135775
95000 / 135775
90700 / 135775
92700 / 135775
88700 / 135775
91800 / 135775
94400 / 135775
90800 / 135775
93800 / 135775
95100 / 135775
92800 / 135775
91900 / 135775
89800 / 135775
88800 / 135775
94500 / 135775
90900 / 135775
95200 / 135775
92900 / 135775
93900 / 135775
96000 / 135775
89900 / 135775
94600 / 135775
88900 / 135775
95300 / 135775
97000 / 135775
98000 / 135775
96100 / 135775
99000 / 135775
94700 / 135775
95400 / 135775
100000 / 135775
97100 / 135775
101000 / 135775
98100 / 135775
99100 / 135775
96200 / 135775
94800 / 135775
97200 / 135775
100100 / 135775
95500 / 135775
98200 / 135775
97300 / 135775
101100 / 135775
99200 / 135775
94900 / 135775
96300 / 135775
100200 / 135775
95600 / 135775
97400 / 135775
99300 / 135775
98300 / 135775
101200 / 135775
102000 / 135775
96400 / 135775
95700 / 135775
100300 / 135775
99400 / 135775
97500 / 135775
96500 / 135775
102100 / 135775
101300 / 135775
98400 / 135775
95800 / 135775
100400 / 135775
99500 / 135775
97600 / 135775
96600 / 135775
102200 / 135775
95900 / 135775
101400 / 135775
98500 / 135775
100500 / 135775
99600 / 135775
102300 / 135775
97700 / 135775
96700 / 135775
101500 / 135775
103000 / 135775
96800 / 135775
102400 / 135775
97800 / 135775
100600 / 135775
99700 / 135775
98600 / 135775
101600 / 135775
96900 / 135775
103100 / 135775
97900 / 135775
102500 / 135775
99800 / 135775
98700 / 135775
100700 / 135775
101700 / 135775
103200 / 135775
104000 / 135775
105000 / 135775
99900 / 135775
102600 / 135775
98800 / 135775
104100 / 135775
101800 / 135775
100800 / 135775
105100 / 135775
103300 / 135775
106000 / 135775
104200 / 135775
98900 / 135775
102700 / 135775
101900 / 135775
105200 / 135775
106100 / 135775
100900 / 135775
104300 / 135775
103400 / 135775
102800 / 135775
106200 / 135775
107000 / 135775
108000 / 135775
105300 / 135775
103500 / 135775
104400 / 135775
106300 / 135775
109000 / 135775
102900 / 135775
107100 / 135775
108100 / 135775
105400 / 135775
103600 / 135775
106400 / 135775
104500 / 135775
109100 / 135775
107200 / 135775
105500 / 135775
103700 / 135775
110000 / 135775
108200 / 135775
106500 / 135775
104600 / 135775
107300 / 135775
105600 / 135775
109200 / 135775
108300 / 135775
103800 / 135775
106600 / 135775
110100 / 135775
105700 / 135775
107400 / 135775
104700 / 135775
109300 / 135775
108400 / 135775
103900 / 135775
106700 / 135775
110200 / 135775
105800 / 135775
108500 / 135775
104800 / 135775
107500 / 135775
109400 / 135775
106800 / 135775
111000 / 135775
105900 / 135775
110300 / 135775
108600 / 135775
104900 / 135775
112000 / 135775
106900 / 135775
110400 / 135775
107600 / 135775
109500 / 135775
108700 / 135775
113000 / 135775
108800 / 135775
107700 / 135775
110500 / 135775
111100 / 135775
114000 / 135775
112100 / 135775
109600 / 135775
113100 / 135775
108900 / 135775
107800 / 135775
110600 / 135775
114100 / 135775
112200 / 135775
115000 / 135775
109700 / 135775
111200 / 135775
110700 / 135775
112300 / 135775
107900 / 135775
114200 / 135775
115100 / 135775
113200 / 135775
109800 / 135775
112400 / 135775
115200 / 135775
110800 / 135775
116000 / 135775
113300 / 135775
111300 / 135775
109900 / 135775
114300 / 135775
112500 / 135775
110900 / 135775
113400 / 135775
117000 / 135775
116100 / 135775
115300 / 135775
114400 / 135775
112600 / 135775
113500 / 135775
111400 / 135775
117100 / 135775
115400 / 135775
118000 / 135775
116200 / 135775
114500 / 135775
112700 / 135775
111500 / 135775
113600 / 135775
115500 / 135775
117200 / 135775
116300 / 135775
118100 / 135775
114600 / 135775
111600 / 135775
113700 / 135775
112800 / 135775
115600 / 135775
117300 / 135775
118200 / 135775
116400 / 135775
111700 / 135775
114700 / 135775
113800 / 135775
111800 / 135775
112900 / 135775
115700 / 135775
118300 / 135775
116500 / 135775
114800 / 135775
117400 / 135775
115800 / 135775
113900 / 135775
119000 / 135775
118400 / 135775
111900 / 135775
116600 / 135775
114900 / 135775
115900 / 135775
120000 / 135775
118500 / 135775
117500 / 135775
119100 / 135775
116700 / 135775
121000 / 135775
122000 / 135775
117600 / 135775
118600 / 135775
123000 / 135775
120100 / 135775
119200 / 135775
116800 / 135775
121100 / 135775
122100 / 135775
123100 / 135775
120200 / 135775
118700 / 135775
117700 / 135775
119300 / 135775
121200 / 135775
116900 / 135775
122200 / 135775
123200 / 135775
118800 / 135775
120300 / 135775
117800 / 135775
122300 / 135775
121300 / 135775
119400 / 135775
124000 / 135775
123300 / 135775
120400 / 135775
118900 / 135775
117900 / 135775
122400 / 135775
121400 / 135775
119500 / 135775
124100 / 135775
123400 / 135775
120500 / 135775
125000 / 135775
122500 / 135775
126000 / 135775
124200 / 135775
121500 / 135775
119600 / 135775
120600 / 135775
123500 / 135775
122600 / 135775
124300 / 135775
125100 / 135775
119700 / 135775
126100 / 135775
122700 / 135775
120700 / 135775
121600 / 135775
123600 / 135775
124400 / 135775
126200 / 135775
125200 / 135775
119800 / 135775
124500 / 135775
121700 / 135775
120800 / 135775
122800 / 135775
123700 / 135775
125300 / 135775
126300 / 135775
119900 / 135775
124600 / 135775
121800 / 135775
120900 / 135775
122900 / 135775
125400 / 135775
123800 / 135775
126400 / 135775
124700 / 135775
127000 / 135775
125500 / 135775
121900 / 135775
128000 / 135775
129000 / 135775
123900 / 135775
126500 / 135775
129100 / 135775
124800 / 135775
130000 / 135775
127100 / 135775
128100 / 135775
125600 / 135775
131000 / 135775
126600 / 135775
129200 / 135775
124900 / 135775
128200 / 135775
130100 / 135775
125700 / 135775
127200 / 135775
126700 / 135775
131100 / 135775
129300 / 135775
126800 / 135775
132000 / 135775
125800 / 135775
130200 / 135775
127300 / 135775
128300 / 135775
129400 / 135775
131200 / 135775
126900 / 135775
125900 / 135775
132100 / 135775
127400 / 135775
129500 / 135775
131300 / 135775
130300 / 135775
128400 / 135775
132200 / 135775
133000 / 135775
127500 / 135775
134000 / 135775
131400 / 135775
129600 / 135775
132300 / 135775
134100 / 135775
130400 / 135775
127600 / 135775
128500 / 135775
133100 / 135775
129700 / 135775
134200 / 135775
131500 / 135775
127700 / 135775
132400 / 135775
133200 / 135775
128600 / 135775
130500 / 135775
131600 / 135775
134300 / 135775
129800 / 135775
132500 / 135775
133300 / 135775
127800 / 135775
128700 / 135775
130600 / 135775
134400 / 135775
131700 / 135775
132600 / 135775
133400 / 135775
129900 / 135775
128800 / 135775
127900 / 135775
132700 / 135775
134500 / 135775
131800 / 135775
130700 / 135775
133500 / 135775
135000 / 135775
128900 / 135775
132800 / 135775
134600 / 135775
130800 / 135775
131900 / 135775
133600 / 135775
135100 / 135775
132900 / 135775
130900 / 135775
134700 / 135775
133700 / 135775
135200 / 135775
134800 / 135775
135300 / 135775
133800 / 135775
135400 / 135775
134900 / 135775
135500 / 135775
133900 / 135775
135600 / 135775
135700 / 135775
The time to finish average pop and intersection density is: 17370.908341646194
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 8 threads.
Setting CH node vector of size 135775
Setting CH edge vector of size 370102
Range graph removed 358962 edges of 740204
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Total time is : 5.01 hours or 18038.86 seconds
root@dfc5e24fc57c:~/work#```

[gboeing]

Thanks @duelran.

@thuanguy can you take the lead and coordinate with @shiqin-liu to get this fix done accordingly in the next couple days? Sounds like it should be relatively simple. Just cc me on any email traffic in the meantime to keep me abreast.

[shiqin-liu]

Thanks all! @gboeing Sure, @thuanguy and I are connected through email, I plan to take a look at that today, will keep you and @thuanguy posted for any update.

[thuanguy]

Thanks all. I will be working on this later today.

On Mon, Feb 3, 2020 at 7:26 AM Shirley Liu notifications@github.com wrote:

Thanks all! @gboeing https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_gboeing&d=DwMCaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=WxV7U_tKet6-p0WLL2jy-Q&m=ax_9OBR0hHokOJEKzvzva5bUzNT2hWDbd4ucAKnBoGI&s=hNCV5YCpA5kiVKwvGU-bskUJ5CKpanNPRDtI3y7XYqA&e= Sure, @thuanguy https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_thuanguy&d=DwMCaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=WxV7U_tKet6-p0WLL2jy-Q&m=ax_9OBR0hHokOJEKzvzva5bUzNT2hWDbd4ucAKnBoGI&s=g4D85NevzeOcyIqUzP7fQeQcrs4DarEs81Jf6HmCt7A&e= and I are connected through email, I plan to take a look at that today, will keep you and @thuanguy https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_thuanguy&d=DwMCaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=WxV7U_tKet6-p0WLL2jy-Q&m=ax_9OBR0hHokOJEKzvzva5bUzNT2hWDbd4ucAKnBoGI&s=g4D85NevzeOcyIqUzP7fQeQcrs4DarEs81Jf6HmCt7A&e= posted for any update.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_gboeing_global-2Dindicators_issues_11-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAOJHXNS4W5FCSWDZMVHDDF3RBAZRVA5CNFSM4KLKM5XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKUH4TI-23issuecomment-2D581467725&d=DwMCaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=WxV7U_tKet6-p0WLL2jy-Q&m=ax_9OBR0hHokOJEKzvzva5bUzNT2hWDbd4ucAKnBoGI&s=gQfNypRg4iS7J-iXLfLU6cLhWo4AWSyBiTHKu26UBcI&e=, or unsubscribe https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AOJHXNVJRJ233BIOO7SY2SDRBAZRVANCNFSM4KLKM5XA&d=DwMCaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=WxV7U_tKet6-p0WLL2jy-Q&m=ax_9OBR0hHokOJEKzvzva5bUzNT2hWDbd4ucAKnBoGI&s=totD2FpFpMDuvvCS_l0lipVJt0V3NJ_XlUr-dIKQA58&e= .

[shiqin-liu]

I have successfully run through the scripts for Odense.

A quick fix is adjusting the city-specific configuration file:

    "parameters": {
        "samplePointsData_withoutNan": "samplePointsData_withoutNan",
        "samplePoints": "urban_sample_points",
        "destinations": "destinations",
        "supermarket": "Fresh Food / Market",
        "convenience": "Convenience",
        "PT": "Public transport stop (any)",

In this case, I simply changed the value under "supermarket" to "Fresh Food / Market", which used to be "Supermarket".

Here is the end of processing:

49800 / 50118
49900 / 50118
50000 / 50118
50100 / 50118
The time to finish average pop and intersection density is: 19591.868626594543
Start to create hex_id for sample points
Start to calculate assessbility to POIs.
Generating contraction hierarchies with 4 threads.
Setting CH node vector of size 50118
Setting CH edge vector of size 124297
Range graph removed 123314 edges of 248594
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
Total time is : 5.46 hours or 19659.90 seconds

I have not checked the other cities yet, so for future reference:

1. make sure the destination parameter value in the configuration file is the same as the value in "dest_name_full" column under the input destination file.

2. if we decide to use fresh_food_market as the new category name, we shall maybe change all "supermarket" value name under the config to "fresh_food_market" for consistency. But I am not sure about this since I am not aware of this change earlier.

Hopefully, this fix will solve the issue for others as well.

[gboeing]

Thanks @shiqin-liu! @thuanguy do you think you can you get Hanoi working similarly prior to our RA meeting this week?

[thuanguy]

@gboeing yes I am running hanoi. It should be done by then. Last time it took a bit more than 2 days.