camlspotter / learnHA

Learns a hybrid automaton model from trajectories of Cyber-Physical Systems.
GNU General Public License v3.0
0 stars 0 forks source link

The test script fails out of the box. #1

Open camlspotter opened 1 month ago

camlspotter commented 1 month ago
$ ./run_tests 
*
optimization finished, #iter = 10
nu = 0.102758
obj = -6.447543, rho = 1.504222
nSV = 12, nBSV = 9
*
optimization finished, #iter = 28
nu = 0.062941
obj = -4.147741, rho = 0.746432
nSV = 9, nBSV = 5
*
optimization finished, #iter = 82
nu = 0.665830
obj = -65.652223, rho = -0.564208
nSV = 68, nBSV = 64
Total nSV = 77
Accuracy = 38.2886% (46047/120263) (classification)
/home/jun/hal/learnHA/infer_ha/test_svm.py:42: UserWarning: No data for colormapping provided via 'c'. Parameters 'cmap' will be ignored
  plt.scatter(X[idx, 0], X[idx, 1], c=color, cmap=plt.cm.coolwarm, edgecolor='k', s=20)
/home/jun/hal/learnHA/infer_ha/test_svm.py:46: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()
test_runLearnHA_bball_withAnnotate (tests.test_run.TestLearnHA) ... Running test runLearnHA module with Bouncing Ball model with type annotation
ERROR
test_runLearnHA_osci_withAnnotate (tests.test_run.TestLearnHA) ... Running test runLearnHA module with Oscillator model with type annotation
stepsize = 0.01
Number of segments = 14
len of drop =  0
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/numpy/matrixlib/defmatrix.py:69: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
  return matrix(data, dtype=dtype, copy=False)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/numpy/matrixlib/defmatrix.py:69: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
  return matrix(data, dtype=dtype, copy=False)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
Total Clusters after DTW algorithm =  2
Computing Mode Invariant done!
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:36: ResourceWarning: unclosed file <_io.TextIOWrapper name='outputs/data_scale' mode='r' encoding='UTF-8'>
  for i, line in enumerate(open(data_file_name)):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:61: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_y = scipy.frombuffer(prob_y, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:62: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_x = scipy.frombuffer(prob_x, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:63: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  col_idx = scipy.frombuffer(col_idx, dtype='l')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:64: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  row_ptr = scipy.frombuffer(row_ptr, dtype='l')
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:171: DeprecationWarning: scipy.empty is deprecated and will be removed in SciPy 2.0.0, use numpy.empty instead
  x_space = prob.x_space = scipy.empty((x.nnz + x.shape[0]), dtype=svm_node)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:173: DeprecationWarning: scipy.arange is deprecated and will be removed in SciPy 2.0.0, use numpy.arange instead
  prob.rowptr[1:] += scipy.arange(1, x.shape[0] + 1)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/commonutil.py:95: DeprecationWarning: scipy.asarray is deprecated and will be removed in SciPy 2.0.0, use numpy.asarray instead
  return evaluations_scipy(scipy.asarray(ty), scipy.asarray(pv))
Guard Accuracy is  50.0
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:36: ResourceWarning: unclosed file <_io.TextIOWrapper name='outputs/data_scale' mode='r' encoding='UTF-8'>
  for i, line in enumerate(open(data_file_name)):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:61: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_y = scipy.frombuffer(prob_y, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:62: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_x = scipy.frombuffer(prob_x, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:63: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  col_idx = scipy.frombuffer(col_idx, dtype='l')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:64: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  row_ptr = scipy.frombuffer(row_ptr, dtype='l')
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:171: DeprecationWarning: scipy.empty is deprecated and will be removed in SciPy 2.0.0, use numpy.empty instead
  x_space = prob.x_space = scipy.empty((x.nnz + x.shape[0]), dtype=svm_node)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:173: DeprecationWarning: scipy.arange is deprecated and will be removed in SciPy 2.0.0, use numpy.arange instead
  prob.rowptr[1:] += scipy.arange(1, x.shape[0] + 1)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/commonutil.py:95: DeprecationWarning: scipy.asarray is deprecated and will be removed in SciPy 2.0.0, use numpy.asarray instead
  return evaluations_scipy(scipy.asarray(ty), scipy.asarray(pv))
Guard Accuracy is  100.0
False
ok
test_runLearnHA_osci_withoutAnnotate (tests.test_run.TestLearnHA) ... Running test runLearnHA module
stepsize = 0.01
Number of segments = 14
len of drop =  0
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/numpy/matrixlib/defmatrix.py:69: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
  return matrix(data, dtype=dtype, copy=False)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/numpy/matrixlib/defmatrix.py:69: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
  return matrix(data, dtype=dtype, copy=False)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/sklearn/utils/validation.py:585: FutureWarning: np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. Please convert to a numpy array with np.asarray. For more information see: https://numpy.org/doc/stable/reference/generated/numpy.matrix.html
  warnings.warn(
Total Clusters after DTW algorithm =  2
Computing Mode Invariant done!
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:36: ResourceWarning: unclosed file <_io.TextIOWrapper name='outputs/data_scale' mode='r' encoding='UTF-8'>
  for i, line in enumerate(open(data_file_name)):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:61: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_y = scipy.frombuffer(prob_y, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:62: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_x = scipy.frombuffer(prob_x, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:63: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  col_idx = scipy.frombuffer(col_idx, dtype='l')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:64: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  row_ptr = scipy.frombuffer(row_ptr, dtype='l')
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:171: DeprecationWarning: scipy.empty is deprecated and will be removed in SciPy 2.0.0, use numpy.empty instead
  x_space = prob.x_space = scipy.empty((x.nnz + x.shape[0]), dtype=svm_node)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:173: DeprecationWarning: scipy.arange is deprecated and will be removed in SciPy 2.0.0, use numpy.arange instead
  prob.rowptr[1:] += scipy.arange(1, x.shape[0] + 1)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/commonutil.py:95: DeprecationWarning: scipy.asarray is deprecated and will be removed in SciPy 2.0.0, use numpy.asarray instead
  return evaluations_scipy(scipy.asarray(ty), scipy.asarray(pv))
Guard Accuracy is  50.0
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:36: ResourceWarning: unclosed file <_io.TextIOWrapper name='outputs/data_scale' mode='r' encoding='UTF-8'>
  for i, line in enumerate(open(data_file_name)):
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:61: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_y = scipy.frombuffer(prob_y, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:62: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  prob_x = scipy.frombuffer(prob_x, dtype='d')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:63: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  col_idx = scipy.frombuffer(col_idx, dtype='l')
/home/jun/hal/learnHA/infer_ha/libsvm/commonutil.py:64: DeprecationWarning: scipy.frombuffer is deprecated and will be removed in SciPy 2.0.0, use numpy.frombuffer instead
  row_ptr = scipy.frombuffer(row_ptr, dtype='l')
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:171: DeprecationWarning: scipy.empty is deprecated and will be removed in SciPy 2.0.0, use numpy.empty instead
  x_space = prob.x_space = scipy.empty((x.nnz + x.shape[0]), dtype=svm_node)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/svm.py:173: DeprecationWarning: scipy.arange is deprecated and will be removed in SciPy 2.0.0, use numpy.arange instead
  prob.rowptr[1:] += scipy.arange(1, x.shape[0] + 1)
/home/jun/.local/share/virtualenvs/learnHA-l4uw3VnU/lib/python3.8/site-packages/libsvm/commonutil.py:95: DeprecationWarning: scipy.asarray is deprecated and will be removed in SciPy 2.0.0, use numpy.asarray instead
  return evaluations_scipy(scipy.asarray(ty), scipy.asarray(pv))
Guard Accuracy is  100.0
False
ok
test_commandline_parser (tests.tests_utils.test_menu.TestLearnHA) ... Running test commandline_parser module
usage: run.py [-h] -i INPUT_FILENAME [-o OUTPUT_FILENAME] [-c {1,2,3}] [-d ODE_DEGREE] [-m MODES] [-b GUARD_DEGREE]
              [--segmentation-error-tol SEGMENTATION_ERROR_TOL]
              [--segmentation-fine-error-tol SEGMENTATION_FINE_ERROR_TOL] [--threshold-distance THRESHOLD_DISTANCE]
              [--threshold-correlation THRESHOLD_CORRELATION] [--dbscan-eps-dist DBSCAN_EPS_DIST]
              [--dbscan-min-samples DBSCAN_MIN_SAMPLES] --size-input-variable SIZE_INPUT_VARIABLE
              --size-output-variable SIZE_OUTPUT_VARIABLE [--variable-types VARIABLE_TYPES] [--pool-values POOL_VALUES]
              [--constant-value CONSTANT_VALUE] [--ode-speedup ODE_SPEEDUP] [--is-invariant {0,1,2}]
              [--stepsize STEPSIZE] [--filter-last-segment {0,1}] [--lmm-step-size {2,3,4,5,6}]

Learns HA model from input--output trajectories

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_FILENAME, --input-filename INPUT_FILENAME
                        input FileName containing trajectories
  -o OUTPUT_FILENAME, --output-filename OUTPUT_FILENAME
                        output FileName with the learned HA model. Set to out.txt by default
  -c {1,2,3}, --clustering-method {1,2,3}
                        Clustering Algorithm. Options are: 1: DTW (default) 2: DBSCAN 3: piecelinear
  -d ODE_DEGREE, --ode-degree ODE_DEGREE
                        Degree of polynomial in ODE. Set to 1 by default
  -m MODES, --modes MODES
                        Number of modes. Used only in piecelinear clustering algorithm. Set to 1 by default
  -b GUARD_DEGREE, --guard-degree GUARD_DEGREE
                        Degree of polynomial inequalities for Guards. Set to 1 by default
  --segmentation-error-tol SEGMENTATION_ERROR_TOL
                        Maximal relative-difference (FwdBwd) error tolerated during segmentation. Set to 0.01 by
                        default
  --segmentation-fine-error-tol SEGMENTATION_FINE_ERROR_TOL
                        Maximal relative-difference (Bwd) fine-error tolerated during segmentation. Set to 0.01 by
                        default
  --threshold-distance THRESHOLD_DISTANCE
                        Maximal threshold for distance in DTW clustering algorithm. Set to 0.1 by default
  --threshold-correlation THRESHOLD_CORRELATION
                        Maximal threshold for correlation value in DTW clustering algorithm. Set to 0.8 by default
  --dbscan-eps-dist DBSCAN_EPS_DIST
                        Maximal threshold for distance in DBSCAN clustering algorithm. Set to 0.01 by default
  --dbscan-min-samples DBSCAN_MIN_SAMPLES
                        Maximal threshold for min-samples in DBSCAN clustering algorithm. Set to 2 by default
  --size-input-variable SIZE_INPUT_VARIABLE
                        Number of input variables in the trajectories
  --size-output-variable SIZE_OUTPUT_VARIABLE
                        Number of output variables in the trajectories
  --variable-types VARIABLE_TYPES
                        Type Annotation for variables. Options are t1: continuous variables, t2: constant pool of
                        values, t3: constant assignment. Syntax: --variable-types "x0=t1, x1=t2, x2=t3"
  --pool-values POOL_VALUES
                        set the values of type=t2. Syntax: --pool-values "x1={10, 20, 30, 40}"
  --constant-value CONSTANT_VALUE
                        set the reset value of type=t3. Syntax: --constant-value "x1=0 & x2=47.7"
  --ode-speedup ODE_SPEEDUP
                        Maximum number of segments to include for ODE computation. Set to 10 by default
  --is-invariant {0,1,2}
                        Options are: 0/1/2. Values 0 (default) and 1 computes invariant and 2 disable computation
  --stepsize STEPSIZE   Fixed sampling time step-size of the input trajectories. Set to 0.01 by default
  --filter-last-segment {0,1}
                        1 to enable and 0 to disable (default) filtering out the last segment from a trajectory during
                        segmentation
  --lmm-step-size {2,3,4,5,6}
                        Options are: 2/3/4/5/6. Higher values computes more accurate derivatives. 5 is set default
ok

======================================================================
ERROR: test_runLearnHA_bball_withAnnotate (tests.test_run.TestLearnHA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jun/hal/learnHA/tests/test_run.py", line 171, in test_runLearnHA_bball_withAnnotate
    list_of_trajectories, stepsize, system_dimension = parse_trajectories(input_filename)
  File "/home/jun/hal/learnHA/utils/parse_parameters.py", line 230, in parse_trajectories
    with open(input_filename, 'r') as file:
FileNotFoundError: [Errno 2] No such file or directory: 'data/test_data/simu_bball_4.txt'

----------------------------------------------------------------------
Ran 4 tests in 2.937s

FAILED (errors=1)
camlspotter commented 1 month ago

FileNotFoundError: [Errno 2] No such file or directory: 'data/test_data/simu_bball_4.txt'

The data file does not exist.

camlspotter commented 1 month ago

FileNotFoundError: [Errno 2] No such file or directory: 'data/test_data/simu_bball_4.txt'

The data file does not exist.

https://github.com/camlspotter/learnHA/issues/2

camlspotter commented 1 month ago

List of tests:

camlspotter commented 1 month ago

test_menu.py is a test to print its help message. The output is expected.

camlspotter commented 1 month ago

For libsvm, see #3