search

kraina-ai / srai

Spatial Representations for Artificial Intelligence - a Python library toolkit for geospatial machine learning focused on creating embeddings for downstream tasks
https://kraina-ai.github.io/srai/
Apache License 2.0
213 stars 16 forks source link

Fix randomized VoronoiRegionalizer test error #313

Closed RaczeQ closed 1 year ago

RaczeQ commented 1 year ago

Sometimes the tests fail since its' data is randomized (generate points on a sphere).

The error is below:

=================================== FAILURES ===================================
_______________________ test_big_number_of_seeds_regions _______________________
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/process.py", line 246, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/process.py", line 205, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/process.py", line 205, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/home/runner/work/srai/srai/srai/regionalizers/_spherical_voronoi.py", line 243, in _create_region
    multi_polygon = MultiPolygon(multi_polygon_parts)
  File "/home/runner/work/srai/srai/.tox/python3.9/lib/python3.9/site-packages/shapely/geometry/multipolygon.py", line 79, in __new__
    shell = ob[0]
TypeError: 'GeometryCollection' object is not subscriptable
"""

The above exception was the direct cause of the following exception:

gdf_earth_bbox =                                             geometry
0  POLYGON ((180.00000 -90.00000, 180.00000 90.00...
earth_bbox = <POLYGON ((180 -90, 180 90, -180 90, -180 -90, 180 -90))>

    def test_big_number_of_seeds_regions(gdf_earth_bbox: gpd.GeoDataFrame, earth_bbox: Polygon) -> None:
        """Test checks if regions are generated correctly and multiprocessing working."""
        number_of_points = 1000
        minx, miny, maxx, maxy = earth_bbox.bounds
        rng = np.random.default_rng()
        randx = rng.uniform(minx, maxx, number_of_points)
        randy = rng.uniform(miny, maxy, number_of_points)
        coords = np.vstack((randx, randy)).T

        pts = [p for p in list(map(Point, coords)) if p.covered_by(earth_bbox)]

        random_points_gdf = gpd.GeoDataFrame(
            {GEOMETRY_COLUMN: pts},
            index=list(range(len(pts))),
            crs=WGS84_CRS,
        )

        vr = VoronoiRegionalizer(seeds=random_points_gdf)
>       result_gdf = vr.transform(gdf=gdf_earth_bbox)

tests/regionalizers/test_voronoi_regionalizer.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
srai/regionalizers/voronoi_regionalizer.py:114: in transform
    generated_regions = generate_voronoi_regions(
srai/regionalizers/_spherical_voronoi.py:308: in generate_voronoi_regions
    process_map(
.tox/python3.9/lib/python3.9/site-packages/tqdm/contrib/concurrent.py:105: in process_map
    return _executor_map(ProcessPoolExecutor, fn, *iterables, **tqdm_kwargs)
.tox/python3.9/lib/python3.9/site-packages/tqdm/contrib/concurrent.py:51: in _executor_map
    return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
.tox/python3.9/lib/python3.9/site-packages/tqdm/std.py:1178: in __iter__
    for obj in iterable:
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/process.py:562: in _chain_from_iterable_of_lists
    for element in iterable:
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/_base.py:609: in result_iterator
    yield fs.pop().result()
/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/_base.py:439: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = None

    def __get_result(self):
        if self._exception:
            try:
>               raise self._exception
E               TypeError: 'GeometryCollection' object is not subscriptable

/opt/hostedtoolcache/Python/3.9.17/x64/lib/python3.9/concurrent/futures/_base.py:391: TypeError
----------------------------- Captured stderr call -----------------------------

Generating regions:   0%|          | 0/1000 [00:00<?, ?it/s]
Generating regions:   0%|          | 1/1000 [00:13<3:42:53, 13.39s/it]
Generating regions:  25%|██▌       | 250/1000 [00:13<00:40, 18.67it/s]
RaczeQ commented 1 year ago

Another problem with randomized data - topology simplification error:

=================================== FAILURES ===================================
_______________________ test_big_number_of_seeds_regions _______________________
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/process.py", line 198, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/process.py", line 198, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/home/runner/work/srai/srai/srai/regionalizers/_spherical_voronoi.py", line 235, in _create_region
    polygon = _create_polygon(
  File "/home/runner/work/srai/srai/srai/regionalizers/_spherical_voronoi.py", line 202, in _create_polygon
    polygon = polygon.intersection(bbox)
  File "/home/runner/work/srai/srai/.tox/python3.8/lib/python3.8/site-packages/shapely/geometry/base.py", line 582, in intersection
    return shapely.intersection(self, other, grid_size=grid_size)
  File "/home/runner/work/srai/srai/.tox/python3.8/lib/python3.8/site-packages/shapely/decorators.py", line 77, in wrapped
    return func(*args, **kwargs)
  File "/home/runner/work/srai/srai/.tox/python3.8/lib/python3.8/site-packages/shapely/set_operations.py", line 133, in intersection
    return lib.intersection(a, b, **kwargs)
shapely.errors.GEOSException: TopologyException: Input geom 0 is invalid: Self-intersection at -83.045622539999997 -89.987218617532889
"""

The above exception was the direct cause of the following exception:

gdf_earth_bbox =                                             geometry
0  POLYGON ((180.00000 -90.00000, 180.00000 90.00...
earth_bbox = <POLYGON ((180 -90, 180 90, -180 90, -180 -90, 180 -90))>

    def test_big_number_of_seeds_regions(gdf_earth_bbox: gpd.GeoDataFrame, earth_bbox: Polygon) -> None:
        """Test checks if regions are generated correctly and multiprocessing working."""
        number_of_points = 1000
        minx, miny, maxx, maxy = earth_bbox.bounds
        rng = np.random.default_rng()
        randx = rng.uniform(minx, maxx, number_of_points)
        randy = rng.uniform(miny, maxy, number_of_points)
        coords = np.vstack((randx, randy)).T

        pts = [p for p in list(map(Point, coords)) if p.covered_by(earth_bbox)]

        random_points_gdf = gpd.GeoDataFrame(
            {GEOMETRY_COLUMN: pts},
            index=list(range(len(pts))),
            crs=WGS84_CRS,
        )

        vr = VoronoiRegionalizer(seeds=random_points_gdf)
>       result_gdf = vr.transform(gdf=gdf_earth_bbox)

tests/regionalizers/test_voronoi_regionalizer.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
srai/regionalizers/voronoi_regionalizer.py:114: in transform
    generated_regions = generate_voronoi_regions(
srai/regionalizers/_spherical_voronoi.py:308: in generate_voronoi_regions
    process_map(
.tox/python3.8/lib/python3.8/site-packages/tqdm/contrib/concurrent.py:105: in process_map
    return _executor_map(ProcessPoolExecutor, fn, *iterables, **tqdm_kwargs)
.tox/python3.8/lib/python3.8/site-packages/tqdm/contrib/concurrent.py:51: in _executor_map
    return list(tqdm_class(ex.map(fn, *iterables, chunksize=chunksize), **kwargs))
.tox/python3.8/lib/python3.8/site-packages/tqdm/std.py:1178: in __iter__
    for obj in iterable:
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/process.py:484: in _chain_from_iterable_of_lists
    for element in iterable:
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/_base.py:619: in result_iterator
    yield fs.pop().result()
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/_base.py:444: in result
    return self.__get_result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = None

    def __get_result(self):
        if self._exception:
            try:
>               raise self._exception
E               shapely.errors.GEOSException: TopologyException: Input geom 0 is invalid: Self-intersection at -83.045622539999997 -89.987218617532889

/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/concurrent/futures/_base.py:389: GEOSException
----------------------------- Captured stderr call -----------------------------

Generating regions:   0%|          | 0/1000 [00:00<?, ?it/s]
Generating regions:   0%|          | 0/1000 [00:01<?, ?it/s]