TKlerx / SADL

Sequence-based Anomaly Detection Library
GNU General Public License v3.0
3 stars 9 forks source link

Learning a PDRTA from positive samples #30

Open armour23 opened 6 years ago

armour23 commented 6 years ago

Hi

I am a PhD student researching on learning a timed, probabilistic automata from positive samples. I would like to try using SADL's RTI+ implementation but am having errors building the project using Eclipse Oxygen. One of the errors is "Plugin execution not covered by lifecycle configuration".

May I know if there is a specific build of Eclipse I should use to build SADL correctly? Are there specific settings / other needed plugins to note?

Thanks! Teck Ping

TKlerx commented 6 years ago

Hi Teck Ping,

did you try running mvn install without eclipse? Most probably, you are missing a m2e plugin for a specific lifecycle. Can you provide more details? If you look at the pom.xml eclipse usually recommends the plugins you need to install. Btw: Which java version do you use? And did you try the jar uploaded as release? Additionally, you should be able to download SADL adding the following to your pom.xml (at the right place):

...
    <repositories>
        <repository>
            <id>SADL-mvn-repo</id>
            <url>https://raw.github.com/TKlerx/SADL/mvn-repo/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
        <!-- <repository> <id>edwardraff-repo</id> <url>http://www.edwardraff.com/maven-repo/</url> 
            </repository> -->
    </repositories>
...

Best, Timo

armour23 commented 6 years ago

Hi Timo

Many thanks for your reply. On my Windows 7 system, I now try to use maven to build the project without Eclipse. I am using the latest version of Java (version 10).

Using a command prompt, I changed directory to "\SADL-dev" and ran "mvn test". It appears that RTI+ test failed as the following (with more details but omitted) was recorded in the file "sadl.modellearner.RtiPlusTest" within the "surefire-reports" folder.

Test set: sadl.modellearner.RtiPlusTest

Tests run: 5, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 48.655 sec <<< FAILURE! - in sadl.modellearner.RtiPlusTest testSerialization(sadl.modellearner.RtiPlusTest) Time elapsed: 3.803 sec <<< FAILURE! java.lang.AssertionError: PDRTAs for files 5 are not equal expected:<// minTimeDelay=1

Possible to download SADL without needing to build it? The XML snippet provided in your earlier post appears in the POM.XML file of the "PDTTA-core" project. Should I use this file to download SADL? Is the maven command "mvn package"? I tried this but it stopped because of the RTI+ test failure.

Thanks! Teck Ping

TKlerx commented 6 years ago

Hi Teck Ping,

the issue came with java 9 and still persists in java 8. With java 8, you should be able to build the jar file. On the other hand, you can download the released jar from https://github.com/TKlerx/SADL/releases Snapshots are available via maven or downloaded from https://github.com/TKlerx/SADL/tree/mvn-repo/de/upb/timok/PDTTA-core/0.0.2-SNAPSHOT Or you run maven and skip the tests using mvn install -Dmaven.test.skip=true

Best, Timo

armour23 commented 6 years ago

Hi Timo

Many thanks for the follow-up. My current use case is to provide some timed strings and to use RTI+ to construct a PDRTA. Is it possible to provide a minimal working example?

I managed to download the JAR file. In a command prompt, I tried to run without success:

java -jar PDTTA-core-0.0.1.jar train rti -sig 0.05 -hist 4

However, I know I am missing the filename parameter and am not sure what is the needed input file format.

Thanks! Teck Ping

TKlerx commented 6 years ago

Hi Teck Ping,

the command line actually has a bug (and I found that the command line usage is very crappy; even though it was designed to be very usable). We actually used SADL more as a library than from the command line. In the TimedInput class, you find the supported file formats and a sample file in the folder testResources. Our proposed format is one sequence per line as ([event]_1, time interval_1) ([event]_2, time interval_2) ...

The command to run would be train rti -in SADL\PDTTA-core\testResources\pdrta\test_1.inp -sig 0.05 -hist MAD. I still need to fix the bug. Will hopefully do this in the evening.

Best, Timo

TKlerx commented 6 years ago

Hi Teck Ping,

the patched jar can be found here. The command line call will create a sadl_train_out.model that can be used for testing using

test -trainSeqs SADL\PDTTA-core\testResources\pdrta\test_1.inp -testSeqs SADL\PDTTA-core\testResources\pdrta\test_1.inp -model SADL\PDTTA-core\sadl_train_out.model

What type of output do you expect from the train method? Testing seems to work not properly for the PDRTA. Somehow, since java9, there is a bug with the seralization. I tried to fix this but I did not succeed so far.

Hope this helps, Timo

armour23 commented 6 years ago

Hi Timo

Many thanks for working on this. I will revert if I get results.

Best Regards Teck Ping

armour23 commented 5 years ago

Hi Timo

It's been some time, hope you've been going well the past year. I am currently exploring this tool again, and I saved the file "test_1.inp" together with the patched jar file and ran this command:

> java -jar PDTTA-core-0.0.2-20180419.205047-1.jar train rti -in test_1.inp -sig 0.05 -hist MAD

I got the following error, please let me know if you have ideas on how to solve it, thanks!

Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory at sadl.run.SADL.(SADL.java:46) Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 1 more