Closed rjhanes closed 4 months ago
Investigation:
Reproduce error by running on issue-198
(identical to master
) and the national datasets, with the following debugging code:
Result:
10100
10120
AR
finding routes
0
> c:\users\rhanes\github\celavi\celavi\routing.py(325)get_all_routes()
-> _vkmt_by_county["source_long"] = _routes.source_long.iloc[i]
(Pdb) i
0
(Pdb) _routes
source_long source_lat destination_long destination_lat
1144 -92.867156 34.608767 NaN NaN
2289 -94.416600 33.693600 NaN NaN
3035 NaN NaN -92.149700 34.837100
3040 NaN NaN -94.370400 35.292600
3052 NaN NaN -92.302410 34.651290
... ... ... ... ...
1141 -92.867156 34.608767 -86.475468 40.519850
1142 -92.867156 34.608767 -68.994546 45.032014
1143 -92.867156 34.608767 -84.343820 36.123894
3535 NaN NaN -100.419370 32.451670
3536 NaN NaN -93.041458 41.715004
[1170 rows x 4 columns]
(Pdb) _routes.source_long.iloc[i]
-92.867156
(Pdb) _routes.source_lat.iloc[i]
34.6087675
(Pdb) _routes.destination_long.iloc[i]
nan
(Pdb) start=(
*** SyntaxError: unexpected EOF while parsing
(Pdb) _routes.source_long.iloc[i],
(-92.867156,)
(Pdb) _routes.source_lat.iloc[i],
(34.6087675,)
(Pdb) )
*** SyntaxError: unmatched ')'
(Pdb) start=(
*** SyntaxError: unexpected EOF while parsing
(Pdb) _routes.source_long.iloc[i],
(-92.867156,)
(Pdb) _routes.source_lat.iloc[i],
(34.6087675,)
(Pdb) start=(_routes.source_long.iloc[i],_routes.source_lat.iloc[i])
(Pdb) end=(_routes.destination_long.iloc[i],_routes.destination_lat.iloc[i])
(Pdb) _start_point = np.array(start)
(Pdb) _start_point_idx = self._btree.query(_start_point, k=1)[1]
*** NameError: name 'self' is not defined
(Pdb) _start_point_idx = router._btree.query(_start_point, k=1)[1]
(Pdb) from_node = router.node_map.loc[_start_point_idx, "node_id"]
(Pdb) _end_point = np.array(end)
(Pdb) _end_point_idx = router._btree.query(_end_point, k=1)[1]
(Pdb) to_node = router.node_map.loc[_end_point_idx, "node_id"]
*** KeyError: 122978
(Pdb) _end_point_idx
122978
(Pdb) _end_point
array([nan, nan])
(Pdb)
Looks like the AR-originating routes with NANs for destination lat/longs are all going to future-deployment power plants (the destination_facility_id
values indicate so):
The future-deployment plant in AR likewise has NAN lat/longs, as per route_list.
So: The future-deployment plants have assumed locations at the
... This is probably related to f9fb9c8
The main fix there was to specify that only lat/long columns were being averaged, and I added a filter that only power plant locations should be included in that average. Therefore, states with no existing wind plants would have NAN lat/longs for future deployment plants (no existing lat/longs to average --> NAN output).
Test fix: Change line 469 of compute_locations.py from
self.locs.loc[self.locs.facility_type == 'power plant'].groupby(
to
self.locs.groupby(
(I.e. reverting only the change made to that line in f9fb9c8
After re-running, the code proceeds past the AR error and no NANs are present in the route_list file.
I'm not going to test all the way through to the end as it'll take too long and eat up processing power, but this particular bug should be resolved with the next couple of commits.
closed in release v1.3.3
Error output:
Specifically it looks like the problem cropped up in
router.get_route
, line 310, when it's used to get VKMT by county for all routes between facilities.This comes from running
master
on the national dataset files packaged with release v1.3.1 (and v1.3.2 - identical datasets).There is a node with ID 122978 at long = -82.505 lat = 35.5868 (see node_locations.csv row 122376). However this node does not show up in the locations_computed file produced from the run that threw the error - unclear if it wasn't processed yet or if the fact it's missing is the problem.
Plan for debugging:
Set up
issue-198
branch to investigate and implement fix Re run with aset_trace
inget_route
to catch theKeyError
before it stops the code, then investigate further.