yassersami / sampler

0 stars 1 forks source link

Voronoi #5

Open yassersami opened 1 month ago

yassersami commented 1 month ago

I got following error for only interest experiment on R2 region:

 WARNING  There are 3 nodes that have not run.                                                       runner.py:205
                             You can resume the pipeline run from the nearest nodes with persisted inputs by adding the
                             following argument to your previous command:
                               --from-nodes ""
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\ysami\.conda\envs\venv_lc\lib\runpy.py:194 in _run_module_as_main                       │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\runpy.py:87 in _run_code                                  │
│                                                                                                  │
│ in <module>:7                                                                                    │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\framework\cli\cli.py:211 in main      │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\click\core.py:1157 in __call__              │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\framework\cli\cli.py:139 in main      │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\click\core.py:1078 in main                  │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\click\core.py:1688 in invoke                │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\click\core.py:1434 in invoke                │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\click\core.py:783 in invoke                 │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\framework\cli\project.py:455 in run   │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\framework\session\session.py:421 in   │
│ run                                                                                              │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\runner.py:91 in run            │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\sequential_runner.py:70 in     │
│ _run                                                                                             │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\runner.py:319 in run_node      │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\runner.py:415 in               │
│ _run_node_sequential                                                                             │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\runner.py:381 in               │
│ _call_node_run                                                                                   │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\runner\runner.py:371 in               │
│ _call_node_run                                                                                   │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\pipeline\node.py:355 in run           │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\pipeline\node.py:348 in run           │
│                                                                                                  │
│ C:\Users\ysami\.conda\envs\venv_lc\lib\site-packages\kedro\pipeline\node.py:398 in               │
│ _run_with_dict                                                                                   │
│                                                                                                  │
│ C:\Local\LAAS\projets\sampler\src\sampler\pipelines\metrics\nodes.py:114 in get_metrics          │
│                                                                                                  │
│   111 │   │   │   "features_targets": np.array([0]*n_interest[key])                              │
│   112 │   │   }                                                                                  │
│   113 │   │   if params_voronoi["compute_voronoi"]["features"]:                                  │
│ ❱ 114 │   │   │   volume_voronoi[key]["features"] = get_volume_voronoi(                          │
│   115 │   │   │   │   scaled_data_interest_f,                                                    │
│   116 │   │   │   │   len(features),tol=params_voronoi["tol"], isFilter=params_voronoi["isFilt   │
│   117 │   │   │   )                                                                              │
│                                                                                                  │
│ C:\Local\LAAS\projets\sampler\src\sampler\pipelines\metrics\voronoi.py:111 in get_volume_voronoi │
│                                                                                                  │
│   108 │   │                                                                                      │
│   109 │   │                                                                                      │
│   110 │   │   # * Add points of the hypercube inside the polytope (region) if within the regio   │
│ ❱ 111 │   │   delaunay = Delaunay(polytope)                                                      │
│   112 │   │   for p in hypercube_points:                                                         │
│   113 │   │   │   if delaunay.find_simplex(p)>=0:                                                │
│   114 │   │   │   │   polytope = np.vstack([polytope, p])                                        │
│                                                                                                  │
│ in scipy.spatial._qhull.Delaunay.__init__:1841                                                   │
│                                                                                                  │
│ in scipy.spatial._qhull._Qhull.__init__:353                                                      │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
QhullError: QH6154 Qhull precision error: Initial simplex is flat (facet 1 is coplanar with the interior point)

While executing:  | qhull d Qz Q12 Qbb Qt Qc
Options selected for Qhull 2019.1.r 2019/06/21:
  run-id 1483190572  delaunay  Qz-infinity-point  Q12-allow-wide  Qbbound-last
  Qtriangulate  Qcoplanar-keep  _pre-merge  _zero-centrum  Qinterior-keep
  Pgood  _max-width 0.56  Error-roundoff 1.8e-15  _one-merge 1.7e-14
  Visible-distance 1.1e-14  U-max-coplanar 1.1e-14  Width-outside 2.2e-14
  _wide-facet 6.6e-14  _maxoutside 2.2e-14

The input to qhull appears to be less than 4 dimensional, or a
computation has overflowed.

Qhull could not construct a clearly convex simplex from points:
- p7(v5):  0.38  0.76 0.055  0.39
- p4(v4):  0.28  0.67 0.055 0.093
- p12(v3):  0.27  0.78 0.055  0.91
- p8(v2):  0.38   0.9 0.055  0.75
- p1(v1): -0.18  0.65 0.055 0.00055

The center point is coplanar with a facet, or a vertex is coplanar
with a neighboring facet.  The maximum round off error for
computing distances is 1.8e-15.  The center point, facets and distances
to the center point are as follows:

center point   0.2229   0.7524    0.055    0.429

facet p4 p12 p8 p1 distance= 1.9e-18
facet p7 p12 p8 p1 distance= -8.5e-18
facet p7 p4 p8 p1 distance= -6.9e-18
facet p7 p4 p12 p1 distance= -7.8e-18
facet p7 p4 p12 p8 distance= 2.7e-18

These points either have a maximum or minimum x-coordinate, or
they maximize the determinant for k coordinates.  Trial points
are first selected from points that maximize a coordinate.

The min and max coordinates for each dimension are:
  0:    -0.185     0.379  difference= 0.564
  1:     0.651     0.912  difference= 0.261
  2:     0.055     0.055  difference= 1.388e-17
  3:         0     0.912  difference= 0.912

If the input should be full dimensional, you have several options that
may determine an initial simplex:
  - use 'QJ'  to joggle the input and make it full dimensional
  - use 'QbB' to scale the points to the unit cube
  - use 'QR0' to randomly rotate the input for different maximum points
  - use 'Qs'  to search all points for the initial simplex
  - use 'En'  to specify a maximum roundoff error less than 1.8e-15.
  - trace execution with 'T3' to see the determinant for each point.

If the input is lower dimensional:
  - use 'QJ' to joggle the input and make it full dimensional
  - use 'Qbk:0Bk:0' to delete coordinate k from the input.  You should
    pick the coordinate with the least range.  The hull will have the
    correct topology.
  - determine the flat containing the points, rotate the points
    into a coordinate plane, and delete the other coordinates.
  - add one or more points to make the input full dimensional.

I tried to add a noise as suggested by chatGPT but it didn't solve it.

        if params_voronoi["compute_voronoi"]["features"]:
            noise = np.random.normal(scale=1e-6, size=scaled_data_interest_f.shape)
            volume_voronoi[key]["features"] = get_volume_voronoi(
                scaled_data_interest_f + noise,
                len(features),tol=params_voronoi["tol"], isFilter=params_voronoi["isFilter"]
            )

But when you check only interest results, they are in fact too condesed in some regions. We could try to remove close points from the scaled_data_interest_f.

import numpy as np
from scipy.spatial import distance_matrix

def remove_close_points(data, threshold=1e-10):
    """
    Remove points that are too close to each other based on a threshold distance.

    Parameters:
    - data (np.ndarray): The input data array.
    - threshold (float): The minimum allowable distance between points.

    Returns:
    - np.ndarray: The data with close points removed.
    """
    # Compute the distance matrix
    dist_matrix = distance_matrix(data, data)

    # Identify points to keep
    keep_indices = []
    for i in range(len(data)):
        # Check if the point is too close to any previously kept point
        if all(dist_matrix[i, j] >= threshold for j in keep_indices):
            keep_indices.append(i)

    return data[keep_indices]
yassersami commented 1 month ago

01_features_2d