search

researchart / rose6icse

12 stars 71 forks source link

ARIsTEO #188

Closed timm closed 4 years ago

timm commented 4 years ago

https://github.com/researchart/rose6icse/tree/master/submissions/reusable/ARIsTEO

applying for AVAILABLE and REUSABLE

azhe825 commented 4 years ago

@timm: I can't assess reusable since I don't have Matlab licenses. What can I do?

timm commented 4 years ago

@azhe825 we are getting this problem a lot: not everyone has access to all tools.

please assess for available

claudiomenghi commented 4 years ago

my github id is claudiomenghi . The licence is available at https://github.com/SNTSVV/ARIsTEO since our TTO delivered the licence to us only yesterday (they had to do a thorough check).

claudiomenghi commented 4 years ago

We are actually applying for reusable and available. If I remember well in the CfP (which I guess as been changed) it was specified that reusable implied available, so when reusable was assessed available was automatically evaluated.

azhe825 commented 4 years ago

@claudiomenghi

  1. As for availability, it is suggested to get a DOI or a permanent link for the code. E.g. at zenodo.org you can (1) release one version of your repo, then (2) upload that zip file to zenodo for a DOI.
  2. As for reusability, I am assuming that I will need a paid version of Matlab to assess it. Correct me if I am wrong.
claudiomenghi commented 4 years ago

Yes, if you are a member of a University usually they have agreements with Matlab, so you can probably download it for free.

azhe825 commented 4 years ago

OK. Trying that.

azhe825 commented 4 years ago

@claudiomenghi I got the following error message when trying to run the pendulum example after setup_staliro:

Error using systemsimulator (line 343)
External input specified in the Configuration Parameters dialog box on the Data Import/Export page (or as an option to SIM command) to a model
containing no root level input ports

Error in abstract (line 65)
    [hs, ~, sigData] = systemsimulator(model, InitialState, UPoint, TotSimTime, input_range, cp_array, aristeo_options);

Error in aristeo (line 322)
                    [data, abstractedmodel,X0, idOptions]=abstract(model,init_cond,input_range,cp_array,TotSimTime,aristeo_options);

Error in test (line 39)
[resultsaristeo,inputaristeo] = aristeo(model, init_cond, input_range, cp_array, phi, preds, TotSimTime, opt);
arennax commented 4 years ago

@claudiomenghi

I was trying to run setup_staliro but got following errors (using Matlab 2019b), not sure if some dependencies were missing:

>> setup_staliro
Warning: Xcode is installed, but its license has not been accepted. Run Xcode and accept its license
agreement.

> In setup_polarity (line 20)
  In setup_staliro (line 51) 
Error using mex
No supported compiler was found. For options, visit https://www.mathworks.com/support/compilers.

Error in setup_polarity (line 20)
    mex mx_polarity.c cache.c lex.c parse.c rewrt.c polarity.c

Error in setup_staliro (line 51)
setup_polarity(skip_mex)
timm commented 4 years ago

@azhe825 @arennax : can we at least assigned a badge of "available" here? that does not need demonstrably provable executables

claudiomenghi commented 4 years ago

@arennax S-Taliro (which is used by ARIsTEO) uses mex which requires that C is installed on your machine and is correctly configured with by Matlab. The error (https://www.mathworks.com/support/compilers) report you a link that show how to install the c compiler with matlab

claudiomenghi commented 4 years ago

@azhe825 (1) did add the folder ARIsTEO and all of its subfolders on your classpath (right click on the folder > add to path > selected folder and subfolders) (2) Are you in the folder ARIsTEO/Tutorial when you run AristeoTutorial?

If you want you can try to run another model.

For example, you can run staliro_demo_autotrans_01.m (This script sets up all the parameters of ARIsTEO: model to be veried property etc.) Then you can run [resultsaristeo,inputaristeo] = aristeo(model, init_cond, input_range, cp_array, phi, preds, TotSimTime, opt);

To have a description of all the parameters, options etc you can type help aristeo. There are a huge number of parameters and options you can enable and disable.

claudiomenghi commented 4 years ago

@timm @azhe825 We made our software available also on Zenodo DOI 10.5281/zenodo.3676337

@All Please let me know if you have any additional problems in reusing the software. Thanks, Claudio.

azhe825 commented 4 years ago

@claudiomenghi

  1. the AristeoTutorial requires Matlab 2018b (not 2018a as indicated in the repo) to run.
  2. the following error message appears when running AristeoTutorial.m: 
    
Undefined function 'resample' for input arguments of type 'double'.

Error in abstract (line 95) YT=resample(YT,hs.T,1/staliro_opt.SampTime);

Error in aristeo (line 322) [data, abstractedmodel,X0, idOptions]=abstract(model,init_cond,input_range,cp_array,TotSimTime,aristeo_options);

Error in AristeoTutorial (line 16) [resultsaristeo,inputaristeo] = aristeo(model, init_cond, input_range, cp_array, phi, preds, TotSimTime, opt);


and  animation in simulink played correctly until the above error occurs:
![Screen Shot 2020-02-20 at 10 22 45 AM](https://user-images.githubusercontent.com/13929197/74948977-12f90e80-53cb-11ea-95cd-e2c53b639cbb.png)
arennax commented 4 years ago

@arennax S-Taliro (which is used by ARIsTEO) uses mex which requires that C is installed on your machine and is correctly configured with by Matlab. The error (https://www.mathworks.com/support/compilers) report you a link that show how to install the c compiler with matlab

I see, so for macOS, to compile mex file, ARIsTEO requires Xcode (around 7.8 GB) or Intel Parallel Studio XE (around 12 GB) as one of the dependencies, is that correct?

claudiomenghi commented 4 years ago

@azhe825 can you install the signal processing toolbox? Click on "Home > Adds On > Signal Processing Toolbox"

claudiomenghi commented 4 years ago

@arennax Any compatible compiler reported in the webpage (https://www.mathworks.com/support/compilers). Otherwise, I guess that it is also possible to compile the c files of S-Taliro outside Matlab (but I never tried it)

arennax commented 4 years ago

@claudiomenghi webpage (https://www.mathworks.com/support/compilers) only provides Xcode and Intel Parallel Studio XE as options for macOS, maybe it's a good idea to add them as prerequisite in readme file to reduce confusion. (although most of the users may have them already)

claudiomenghi commented 4 years ago

@arennax Yes sure, I am adding them right now. I will also add the Signal Processing Toolbox to add it "Home > Adds On > Signal Processing Toolbox"

claudiomenghi commented 4 years ago

Regarding the version, the software have been tested on Matlab 2018a, 2018b, 2010b and 2016a. Since the file of the tutorial (of the model) has been saved with the newest version 2018a the tutorial can only be executed with this version. However, ARIsTEO has been tested with all these versions of Matlab.

azhe825 commented 4 years ago

@claudiomenghi

% Considers the default initial conditions of the model init_cond = [];

% Sets the input_range. We assume that the user can apply a momentun in the range [-0.5 0.5] input_range = [-2 2];

% Sets the number of control points. We assume the inputs has 100 control points cp_array = 100;

% Defines of the property of interest. The pendulum must remain below the horizontal line. The ARIsTEO testing framework searches for a test input that showing that the pendulum does not remain below the horizontal line. phi='[]_[0,10000] (a/\b)'; preds(1).str = 'a'; preds(1).A = [1]; preds(1).b = [1.5]; preds(2).str = 'b'; preds(2).A = [-1]; preds(2).b = [1.5];

% Sets the simulation Time TotSimTime=10;

% Creates the options of ARIsTEO opt=aristeo_options();

% Sets the number of refinements rounds opt.n_refinement_rounds=20;

%Sets the abstraction algorithm and its parameters opt.fals_at_zero=0; opt.abstraction_algorithm='ss'; opt.nx=2; opt.optim_params.n_tests=10; opt.dispinfo=1;

% Runs ARIsTEO [resultsaristeo,inputaristeo] = aristeo(model, init_cond, input_range, cp_array, phi, preds, TotSimTime, opt);

Error message:

Error using systemsimulator (line 343) External input specified in the Configuration Parameters dialog box on the Data Import/Export page (or as an option to SIM command) to a model containing no root level input ports

Error in abstract (line 65) [hs, ~, sigData] = systemsimulator(model, InitialState, UPoint, TotSimTime, input_range, cp_array, aristeo_options);

Error in aristeo (line 322) [data, abstractedmodel,X0, idOptions]=abstract(model,init_cond,input_range,cp_array,TotSimTime,aristeo_options);

claudiomenghi commented 4 years ago

Yes, my fault model='pendulum'; I renamed the name of the model. Thanks!

azhe825 commented 4 years ago

@claudiomenghi Great! Now it works. I will grant the reusable and available badge once you have added the dependency of (Xcode for mac, toolboxes for tutorial) information to your repo and changed the model name in the example.

Also, at least I can confirm that the tutorial requires Matlab version newer than 2018b to execute, 2018a is not enough.

claudiomenghi commented 4 years ago

I have applied the suggested changes. Thanks a lot for your help!

minkull commented 4 years ago

@azhe825 please could you confirm whether your recommendation is available+reusable based on the modifications? @arennax please could you update us on your views on available and reusable?

azhe825 commented 4 years ago

@claudiomenghi I can see the changes in INSTALL.md. Can you also update the README.md?

claudiomenghi commented 4 years ago

@azhe825 Updated. Thanks again!

azhe825 commented 4 years ago

@minkull @timm Reusable and available passed.

arennax commented 4 years ago

@minkull @timm available and reusable tests are passed (with macOS Catalina 10.15.3)

claudiomenghi commented 4 years ago

@minkull @minkull @timm Thanks for the great work and for organizing this initiative.

minkull commented 4 years ago

Reusable and available awarded!

Many thanks for all your work, @azhe825 @arennax @claudiomenghi