Deploying the core (master branch and demo-brussels tag)

fp7-netide / Engine

App Engine to enable Network App programs to be executed, systematically tested, and refined on a variety of concrete SDN platforms

Eclipse Public License 1.0

10 stars 11 forks source link

Deploying the core (master branch and demo-brussels tag) #71

Closed ElisaRojas closed 8 years ago

ElisaRojas commented 8 years ago

Hi,

I'm preparing a backup VM with the most updated code of the Engine. Right now, I have successfully installed Ryu backend, shim and Python core (everything worked), also compiled the ODL shim (but not tested yet); however, I got stuck with the Java core.

If I use the demo-brussels tag, I can add the repo, but when trying to install the core, I get this error:

karaf@root()> feature:install netide-core
Error executing command: Can't install feature netide-core/0.0.0:     
Could not start bundle mvn:eu.netide.core/core.management/1.0.0.0-SNAPSHOT in feature(s) netide-core-management-0.0.0: Unresolved constraint in bundle core.management [69]: Unable to resolve 69.0: missing requirement [69.0] osgi.wiring.package; (&(osgi.wiring.package=org.apache.felix.service.command)(version>=0.16.0)(!(version>=1.0.0)))

After searching in Google, I decided to change the core.management bundle pom.xml and write: org.apache.felix.service.command;version="[0.16.0,1.0.0)", (to look for specific versions of the package) and then, the feature:install netide-core command does not raise any errors, but gets stuck forever :disappointed:

If I use the master branch instead (I know it is not fully working, but I just wanted to have a try), I see a long error trace when installing the core:

karaf@root()> feature:install netide-core
java.lang.ArrayIndexOutOfBoundsException: 42752
    at aQute.bnd.osgi.Clazz.classConstRef(Clazz.java:1862)
    at aQute.bnd.osgi.Clazz.crawl(Clazz.java:1174)
    at aQute.bnd.osgi.Clazz.doCode(Clazz.java:1134)
    at aQute.bnd.osgi.Clazz.doAttribute(Clazz.java:945)
    at aQute.bnd.osgi.Clazz.doAttributes(Clazz.java:910)
    at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:741)
    at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:494)
    at aQute.bnd.osgi.Clazz.parseClassFileWithCollector(Clazz.java:483)
    at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:473)
    at aQute.bnd.osgi.Analyzer.analyzeJar(Analyzer.java:2177)
    at aQute.bnd.osgi.Analyzer.analyzeBundleClasspath(Analyzer.java:2083)
    at aQute.bnd.osgi.Analyzer.analyze(Analyzer.java:138)
    at aQute.bnd.osgi.Analyzer.calcManifest(Analyzer.java:616)
    at org.ops4j.pax.swissbox.bnd.BndUtils.createBundle(BndUtils.java:161)
    at org.ops4j.pax.url.wrap.internal.Connection.getInputStream(Connection.java:83)
    at java.net.URL.openStream(URL.java:1038)
    at org.apache.karaf.features.internal.BundleManager.getInputStreamForBundle(BundleManager.java:185)
    at org.apache.karaf.features.internal.BundleManager.doInstallBundleIfNeeded(BundleManager.java:95)
    at org.apache.karaf.features.internal.BundleManager.installBundleIfNeeded(BundleManager.java:89)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.doInstallFeature(FeaturesServiceImpl.java:556)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatureDependency(FeaturesServiceImpl.java:613)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.doInstallFeature(FeaturesServiceImpl.java:547)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:419)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:400)
    at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:378)
    at Proxy2eb7271f_a3f2_4b1e_a53c_eaa7c58d1e21.installFeature(Unknown Source)
    at org.apache.karaf.features.command.InstallFeatureCommand.doExecute(InstallFeatureCommand.java:67)
    at org.apache.karaf.features.command.FeaturesCommandSupport.doExecute(FeaturesCommandSupport.java:38)
    at org.apache.karaf.shell.console.AbstractAction.execute(AbstractAction.java:33)
    at org.apache.karaf.shell.console.OsgiCommandSupport.execute(OsgiCommandSupport.java:39)
    at org.apache.karaf.shell.commands.basic.AbstractCommand.execute(AbstractCommand.java:33)
    at Proxy12928ac4_c793_4da7_a9c1_e1afa60992d7.execute(Unknown Source)
    at Proxy12928ac4_c793_4da7_a9c1_e1afa60992d7.execute(Unknown Source)
    at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:78)
    at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:477)
    at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:403)
    at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)
    at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183)
    at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120)
    at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:92)
    at org.apache.karaf.shell.console.impl.jline.ConsoleImpl.run(ConsoleImpl.java:208)
    at org.apache.karaf.shell.console.impl.jline.LocalConsoleManager$2$1$1.run(LocalConsoleManager.java:109)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.apache.karaf.jaas.modules.JaasHelper.doAs(JaasHelper.java:57)
    at org.apache.karaf.shell.console.impl.jline.LocalConsoleManager$2$1.run(LocalConsoleManager.java:102)

and more errors when connecting the Ryu shim:

karaf@root()> Exception in thread "ZeroMQBasedConnector Receive Loop" java.lang.NoClassDefFoundError: com/google/common/hash/Funnel
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.findClass(BundleWiringImpl.java:2279)
    at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1501)
    at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.projectfloodlight.openflow.protocol.ver10.OFMatchV1Ver10.(OFMatchV1Ver10.java:2301)
    at org.projectfloodlight.openflow.protocol.ver10.OFFactoryVer10.(OFFactoryVer10.java:460)
    at org.projectfloodlight.openflow.protocol.OFFactories$GenericReader.readFrom(OFFactories.java:57)
    at org.projectfloodlight.openflow.protocol.OFFactories$GenericReader.readFrom(OFFactories.java:49)
    at eu.netide.lib.netip.NetIPUtils.toOpenFlowMessage(NetIPUtils.java:129)
    at eu.netide.lib.netip.NetIPUtils.ConcretizeMessage(NetIPUtils.java:43)
    at eu.netide.lib.netip.NetIPConverter.parseConcreteMessage(NetIPConverter.java:18)
    at eu.netide.core.connectivity.ShimManager.OnDataReceived(ShimManager.java:47)
    at eu.netide.core.connectivity.ZeroMQBasedConnector.run(ZeroMQBasedConnector.java:110)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.google.common.hash.Funnel not found by org.onosproject.openflowj [65]
    at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1532)
    at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
    at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 17 more

and of course, the combo Ryu backend + Java core + Java shim does not work :(

These errors (particularly the ones for the demo-brussels tag) might be too obvious, but I've been several hours trying and recompiling with different changes in different pom.xml files and the only thing I was able to do is what I said before. So I would really appreciate if @schwabe or @amarsico can give any hints.

Also, @schwabe , does this tag work for you? Maybe we could use another more stable version? I guess the org.apache.felix.service.command package is needed for the custom commands inside Karaf, did you install something else or am I missing something there?

Any help at all would be really appreciated, because we're running out of time, unfortunately :(

P.S: I'm using Maven 3.3.3 and Karaf 3.0.3, just in case it matters.

amarsico commented 8 years ago

Hi Elisa, I had the same issue when trying the core. It depends on the Karaf version. Please try Karaf 3.0.5.

Best regards, Antonio

Il giorno mar 23 feb 2016 alle 01:03 Elisa Rojas notifications@github.com ha scritto:

Hi,

I'm preparing a backup VM with the most updated code of the Engine. Right now, I have successfully installed Ryu backend, shim and Python core (everything worked), also compiled the ODL shim (but not tested yet); however, I got stuck with the Java core.

If I use the demo-brussels tag, I can add the repo, but when trying to install the core, I get this error: karaf@root()> feature:install netide-core Error executing command: Can't install feature netide-core/0.0.0: Could not start bundle mvn:eu.netide.core/core.management/1.0.0.0-SNAPSHOT in feature(s) netide-core-management-0.0.0: Unresolved constraint in bundle core.management [69]: Unable to resolve 69.0: missing requirement [69.0] osgi.wiring.package; (&(osgi.wiring.package=org.apache.felix.service.command)(version>=0.16.0)(!(version>=1.0.0)))

After searching in Google, I decided to change the core.management bundle pom.xml and write: org.apache.felix.service.command;version="[0.16.0,1.0.0)", and then, the feature:install netide-core command does not raise any errors, but get stuck forever [image: :disappointed:]

If I use the master branch instead (I know it is not fully working, but I just wanted to have a try), I see a long error trace when installing the core: karaf@root()> feature:install netide-core java.lang.ArrayIndexOutOfBoundsException: 42752 at aQute.bnd.osgi.Clazz.classConstRef(Clazz.java:1862) at aQute.bnd.osgi.Clazz.crawl(Clazz.java:1174) at aQute.bnd.osgi.Clazz.doCode(Clazz.java:1134) at aQute.bnd.osgi.Clazz.doAttribute(Clazz.java:945) at aQute.bnd.osgi.Clazz.doAttributes(Clazz.java:910) at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:741) at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:494) at aQute.bnd.osgi.Clazz.parseClassFileWithCollector(Clazz.java:483) at aQute.bnd.osgi.Clazz.parseClassFile(Clazz.java:473) at aQute.bnd.osgi.Analyzer.analyzeJar(Analyzer.java:2177) at aQute.bnd.osgi.Analyzer.analyzeBundleClasspath(Analyzer.java:2083) at aQute.bnd.osgi.Analyzer.analyze(Analyzer.java:138) at aQute.bnd.osgi.Analyzer.calcManifest(Analyzer.java:616) at org.ops4j.pax.swissbox.bnd.BndUtils.createBundle(BndUtils.java:161) at org.ops4j.pax.url.wrap.internal.Connection.getInputStream(Connection.java:83) at java.net.URL.openStream(URL.java:1038) at org.apache.karaf.features.internal.BundleManager.getInputStreamForBundle(BundleManager.java:185) at org.apache.karaf.features.internal.BundleManager.doInstallBundleIfNeeded(BundleManager.java:95) at org.apache.karaf.features.internal.BundleManager.installBundleIfNeeded(BundleManager.java:89) at org.apache.karaf.features.internal.FeaturesServiceImpl.doInstallFeature(FeaturesServiceImpl.java:556) at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatureDependency(FeaturesServiceImpl.java:613) at org.apache.karaf.features.internal.FeaturesServiceImpl.doInstallFeature(FeaturesServiceImpl.java:547) at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeatures(FeaturesServiceImpl.java:419) at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:400) at org.apache.karaf.features.internal.FeaturesServiceImpl.installFeature(FeaturesServiceImpl.java:378) at Proxy2eb7271f_a3f2_4b1e_a53c_eaa7c58d1e21.installFeature(Unknown Source) at org.apache.karaf.features.command.InstallFeatureCommand.doExecute(InstallFeatureCommand.java:67) at org.apache.karaf.features.command.FeaturesCommandSupport.doExecute(FeaturesCommandSupport.java:38) at org.apache.karaf.shell.console.AbstractAction.execute(AbstractAction.java:33) at org.apache.karaf.shell.console.OsgiCommandSupport.execute(OsgiCommandSupport.java:39) at org.apache.karaf.shell.commands.basic.AbstractCommand.execute(AbstractCommand.java:33) at Proxy12928ac4_c793_4da7_a9c1_e1afa60992d7.execute(Unknown Source) at Proxy12928ac4_c793_4da7_a9c1_e1afa60992d7.execute(Unknown Source) at org.apache.felix.gogo.runtime.CommandProxy.execute(CommandProxy.java:78) at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:477) at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:403) at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108) at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183) at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120) at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:92) at org.apache.karaf.shell.console.impl.jline.ConsoleImpl.run(ConsoleImpl.java:208) at org.apache.karaf.shell.console.impl.jline.LocalConsoleManager$2$1$1.run(LocalConsoleManager.java:109) at java.security.AccessController.doPrivileged(Native Method) at org.apache.karaf.jaas.modules.JaasHelper.doAs(JaasHelper.java:57) at org.apache.karaf.shell.console.impl.jline.LocalConsoleManager$2$1.run(LocalConsoleManager.java:102)

and more errors when connecting the Ryu shim: karaf@root()> Exception in thread "ZeroMQBasedConnector Receive Loop" java.lang.NoClassDefFoundError: com/google/common/hash/Funnel at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:760) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.findClass(BundleWiringImpl.java:2279) at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1501) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.projectfloodlight.openflow.protocol.ver10.OFMatchV1Ver10.(OFMatchV1Ver10.java:2301) at org.projectfloodlight.openflow.protocol.ver10.OFFactoryVer10.(OFFactoryVer10.java:460) at org.projectfloodlight.openflow.protocol.OFFactories$GenericReader.readFrom(OFFactories.java:57) at org.projectfloodlight.openflow.protocol.OFFactories$GenericReader.readFrom(OFFactories.java:49) at eu.netide.lib.netip.NetIPUtils.toOpenFlowMessage(NetIPUtils.java:129) at eu.netide.lib.netip.NetIPUtils.ConcretizeMessage(NetIPUtils.java:43) at eu.netide.lib.netip.NetIPConverter.parseConcreteMessage(NetIPConverter.java:18) at eu.netide.core.connectivity.ShimManager.OnDataReceived(ShimManager.java:47) at eu.netide.core.connectivity.ZeroMQBasedConnector.run(ZeroMQBasedConnector.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: com.google.common.hash.Funnel not found by org.onosproject.openflowj [65] at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1532) at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75) at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 17 more and of course, the combo Ryu backend + Java core + Java shim does not work :(

These errors (particularly the ones for the demo-brussels tag) might be too obvious, but I've been several hours trying and recompiling with different changes in different pom.xml files and the only thing I was able to do is what I said before. So I would really appreciate if @schwabe https://github.com/schwabe or @amarsico https://github.com/amarsico can give any hints.

Also, @schwabe https://github.com/schwabe , does this tag work for you? Maybe we could use another more stable version? I guess the org.apache.felix.service.command package is needed for the custom commands inside Karaf, did you install something else or am I missing something there?

Any help at all would be really appreciated, because we're running out of time, unfortunately :(

P.S: I'm using Maven 3.3.3 and Karaf 3.0.3, just in case it matters.

— Reply to this email directly or view it on GitHub https://github.com/fp7-netide/Engine/issues/71.

ElisaRojas commented 8 years ago

Thank you @amarsico ! That was it :smile: Solved with Karaf 3.0.5.

Now, my problem is that composition is not working, I'm using the CompositionSpecification.xml file located in the specification folder and the custom topology provided by Roberto. If I check the log, I see:

2016-02-23 11:55:58,924 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.

@schwabe : Should I use a different topology? @amarsico : Have you tried this scenario and works for you?

Thank you!

schwabe commented 8 years ago

@ElisaRojas the configuration error mean that the composition xml has not yet been loaded or not all required backends have connected. (use netide:loadcomposition in the shell to load composition and netide:listModules to show the backends.)

Also after @amarsico changes to the pom files I increased the version number to 1.0.1.0 to make sure not have a mix of old and new packages.

amarsico commented 8 years ago

@ElisaRojas no, I didn't try the CORE composition. I used for my tests the python CORE developed by @doriguzzi.

ElisaRojas commented 8 years ago

Hi @schwabe . I had done both (loaded the composition file and check the backends) and everything was fine. However, I've done it again, just in case, with this specific order:

Run Java core
Load composition file with netide:loadcomposition
Start Ryu shim
Start Ryu backend
List backends with netide:listmodules (and it shows correctly the 1 backend and 2 modules)
Start Mininet

It says that the handshake has completed and the Ryu shim is receiving the PacketIn events from Mininet, however, all pings are dropped (no response).

Maybe I have to change the order in which I deploy elements? When I tried this in December I don't remember any specific order and it worked.

Thank you!

schwabe commented 8 years ago

I don't think that there is a specific order other than starting the core before the backends (and maybe shim). At least in my test I only started the core first and the other components more less random.

I almost never stop/start mininet

ElisaRojas commented 8 years ago

So, any idea of what could it be? I still obtain the same error:

2016-02-23 11:55:58,924 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.

I'm using CompositionSpecification.xml file. Ryu shim and backend from master branch, and Java core from demo-brussels tag with Karaf 3.0.5 and everything compiled with Maven 3.3.3.

I've also tried loading the file with the emulator tool instead of the command, and I still have the same problem (and the file seems to have been sent correctly).

(P.S: Maybe if you tell me what specific components and files you're using, we can start debugging)

schwabe commented 8 years ago

@ElisaRojas THere should be other messages in the load, indicating what modules should be loaded.

If you post more log or the output of netide:listModules and the composition xml, I can take a look

ElisaRojas commented 8 years ago

@schwabe What do you mean with "in the load", in the Java core console after loading the composition file? in its log? or somewhere else?

Please tell me your configuration for debugging on my side. I'm also pasting all the different outputs I have below:

Java core log:display

2016-02-23 14:54:57,643 | ERROR | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:58,668 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:58,669 | INFO  | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=336, totalLen=42, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -29, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -29, 10, 0, 0, 3, 0, 0, 0, 0, 0, 0, 10, 0, 0, 10])]
2016-02-23 14:54:58,669 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:58,964 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:58,965 | INFO  | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=8,ModuleId=0,TransactionId=0,DatapathId=1],Type=ECHO_REQUEST,OFMessage=OFEchoRequestVer10(xid=0, data=[])]
2016-02-23 14:54:58,965 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:58,965 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:58,966 | INFO  | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=8,ModuleId=0,TransactionId=0,DatapathId=10],Type=ECHO_REQUEST,OFMessage=OFEchoRequestVer10(xid=0, data=[])]
2016-02-23 14:54:58,966 | ERROR | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:58,966 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:58,966 | INFO  | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=8,ModuleId=0,TransactionId=0,DatapathId=3],Type=ECHO_REQUEST,OFMessage=OFEchoRequestVer10(xid=0, data=[])]
2016-02-23 14:54:58,966 | ERROR | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:58,966 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:58,966 | INFO  | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=8,ModuleId=0,TransactionId=0,DatapathId=2],Type=ECHO_REQUEST,OFMessage=OFEchoRequestVer10(xid=0, data=[])]
2016-02-23 14:54:58,966 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:59,376 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:59,376 | INFO  | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=146,ModuleId=0,TransactionId=0,DatapathId=1],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=423, totalLen=342, inPort=2, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, -46, -41, 102, -109, 86, 115, 8, 0, 69, 16, 1, 72, 0, 0, 0, 0, -128, 17, 57, -106, 0, 0, 0, 0, -1, -1, -1, -1, 0, 68, 0, 67, 1, 52, -86, -66, 1, 1, 6, 0, -46, -41, 118, 78, 0, 39, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -46, -41, 102, -109, 86, 115, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])]
2016-02-23 14:54:59,376 | ERROR | pool-6-thread-9  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.
2016-02-23 14:54:59,651 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 14:54:59,651 | INFO  | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=337, totalLen=42, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -29, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -29, 10, 0, 0, 3, 0, 0, 0, 0, 0, 0, 10, 0, 0, 10])]
2016-02-23 14:54:59,651 | ERROR | pool-6-thread-11 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Could not handle incoming message due to configuration error.

netide:listmodules

karaf@root()> netide:listmodules
   Id                 Name              Backend Last message (s ago)
   36         SimpleSwitch     backend-ryu-9945                    -
   24     backend-ryu-9945     backend-ryu-9945                 3.59
   58             Firewall     backend-ryu-9945                    -

Ryu backend

---- Backend: Ryu ----
loading app ryu-backend.py
loading app tests/simple_switch.py
loading app tests/firewall.py
loading app ryu.controller.ofp_handler
instantiating app tests/firewall.py of Firewall
instantiating app tests/simple_switch.py of SimpleSwitch
instantiating app ryu-backend.py of RyuBackend
Backend supported protocols: {17: [1, 3, 4], 18: []}
RYU Backend initiated: backend-ryu-9945
Connecting to Core on 127.0.0.1:5555...
instantiating app ryu.controller.ofp_handler of OFPHandler
Received ack from Core: ['\x02\x05\x00\x10\x00\x00\x00\x00\x00\x00\x00\x18\x00\x00\x00\x00\x00\x00\x00\x00backend-ryu-9945']
Received ack from Core: ['\x02\x05\x00\x08\x00\x00\x00\x00\x00\x00\x00:\x00\x00\x00\x00\x00\x00\x00\x00Firewall']
Received ack from Core: ['\x02\x05\x00\x0c\x00\x00\x00\x00\x00\x00\x00$\x00\x00\x00\x00\x00\x00\x00\x00SimpleSwitch']
Starting the handshake process...
Waiting for the Handshake to be completed...
Waiting for the Handshake to be completed...
Waiting for the Handshake to be completed...
Waiting for the Handshake to be completed...
Received Message header: (3, 1, 2, 0, 24, 0)
Message type: NETIDE_HELLO
Message xid: 0
Received Message body: 11:01
Handshake completed!!!

Ryu shim

Sending to Core: Message header: (3, 17, 8, 0, 0, 1)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 10)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 2)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 3)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Received from Core: Message header: (3, 6, 0, 0, 24, 0)
Received from Core: Message body: 
Sending to Core: Message header: (3, 17, 8, 0, 0, 3)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 10)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 1)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Sending to Core: Message header: (3, 17, 8, 0, 0, 2)
Sending to Core: Message body: 01:02:00:08:00:00:00:00
Received from Core: Message header: (3, 6, 0, 0, 24, 0)
Received from Core: Message body:

Mininet

mininet> pingall
*** Ping: testing ping reachability
alice -> X X X 
bob -> X X X 
charlie -> X X X 
www -> X X X 
*** Results: 100% dropped (0/12 received)

pablomuri commented 8 years ago

@ElisaRojas Have you tried this? https://github.com/fp7-netide/Engine/issues/42#issuecomment-155400783

schwabe commented 8 years ago

@ElisaRojas the messages about the composition should be before the log line you posted. That message menas that the compositionmanager has not seen all modules that are listed in the xml file, which is missing in your post. They should be right after loading a composition specification.

ElisaRojas commented 8 years ago

@schwabe Ok, thank you, at the same time you've replied, I just realised that there are two files called "CompositionSpecification.xml", but they are different: Ryu: https://github.com/fp7-netide/Engine/blob/master/ryu-backend/tests/CompositionSpecification.xml Core: https://github.com/fp7-netide/Engine/blob/master/core/specification/CompositionSpecification.xml

So I tried again specifically using the one from Ryu (which was the one that I tried the Python core with and the one that fits the SW+FW combination), and now I get this error instead:

der [Version=VERSION_1_2,Type=OPENFLOW,Length=146,ModuleId=0,TransactionId=0,DatapathId=1],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=265, totalLen=342, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 6, -57, -85, 65, 62, 102, 8, 0, 69, 16, 1, 72, 0, 0, 0, 0, -128, 17, 57, -106, 0, 0, 0, 0, -1, -1, -1, -1, 0, 68, 0, 67, 1, 52, 62, -102, 1, 1, 6, 0, 73, 124, 11, 95, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, -57, -85, 65, 62, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])]
2016-02-23 15:18:53,985 | ERROR | pool-6-thread-1  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | An exception occurred while handling shim message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)[:1.8.0_66]
    at java.util.ArrayList.get(ArrayList.java:429)[:1.8.0_66]
    at eu.netide.core.caos.execution.SequentialFlowExecutor.executeFlow(SequentialFlowExecutor.java:47)
    at eu.netide.core.caos.CompositionManager.OnShimMessage(CompositionManager.java:131)
    at Proxy3ef65629_65b3_4ae5_b4ff_cd7bee3e0426.OnShimMessage(Unknown Source)
    at Proxy2cb493ee_d19f_4d67_8191_cffb78b71359.OnShimMessage(Unknown Source)
    at eu.netide.core.connectivity.ShimManager.lambda$OnDataReceived$4(ShimManager.java:61)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2016-02-23 15:18:54,738 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-23 15:18:54,738 | INFO  | pool-6-thread-7  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=2],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=268, totalLen=42, inPort=2, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -30, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -30, 10, 0, 0, 2, 0, 0, 0, 0, 0, 0, 10, 0, 0, 1])]
2016-02-23 15:18:54,739 | ERROR | pool-6-thread-7  | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | An exception occurred while handling shim message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)[:1.8.0_66]
    at java.util.ArrayList.get(ArrayList.java:429)[:1.8.0_66]
    at eu.netide.core.caos.execution.SequentialFlowExecutor.executeFlow(SequentialFlowExecutor.java:47)
    at eu.netide.core.caos.CompositionManager.OnShimMessage(CompositionManager.java:131)
    at Proxy3ef65629_65b3_4ae5_b4ff_cd7bee3e0426.OnShimMessage(Unknown Source)
    at Proxy2cb493ee_d19f_4d67_8191_cffb78b71359.OnShimMessage(Unknown Source)
    at eu.netide.core.connectivity.ShimManager.lambda$OnDataReceived$4(ShimManager.java:61)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]

@schwabe : Can we talk in Slack? Maybe it's faster there... @pablomuri : I already tried that and it worked with a previous version of the code... or do you mean something specific of that post?

schwabe commented 8 years ago

Yeah, we can talk in slack, just message me there.

@ElisaRojas the ryu specification is invalid xml. The java parser is more strict. 0x1 is not a valid xml integer for the java parser.

ElisaRojas commented 8 years ago

@schwabe Ok, that was it, thank you! I tried with the original "DpidPartitionExample.xml" example and it works, but it works the way it is told in this file, which is different from the one of Roberto :smiley: I'll try to generate an equivalent file, with the behavior of Roberto's xml and try to upload it later, and then I'll close issue. Thanks!

ElisaRojas commented 8 years ago

Ok, so I've prepared another composition file that follows Roberto's example, it is core/specification/DpidPartitionABCW.xml and tested with the Java core, to compare it with the Python core.

The original output for the Python core is:

mininet> pingall
*** Ping: testing ping reachability
alice -> bob charlie www 
bob -> alice charlie www 
charlie -> alice bob X 
www -> alice bob charlie 
*** Results: 8% dropped (11/12 received)

(This is not exactly the behaviour of the firewall, because it opens a window for 5 seconds and then charlie can ping alice and bob as well -> As explained by @doriguzzi in Madrid, but it's the way it has to work when pinging all).

However, the output for the Java core is:

mininet> pingall
*** Ping: testing ping reachability
alice -> bob X www 
bob -> alice X www 
charlie -> X X X 
www -> alice bob X 
*** Results: 50% dropped (6/12 received)

Which means that the simple_switch.py application is working fine (the switches with datapaths IDs are alice, bob and www - @doriguzzi , correct me if I'm wrong), but not the firewall, because it is blocking all the traffic instead of distinguishing int->ext and ext->int traffic.

@schwabe : Could you please check the core with this deployment Ryu+Javacore+Ryu+Mininet= sudo mn --custom netide-topo.py --topo mytopo --controller=remote,ip=IP_ADDRESS,port=6633 (netide-topo.py can be found in the ryu-backend/tests folder) and the composition file I've uploaded to master branch (core/specification/DpidPartitionABCW.xml)? We need to check whether the firewall is correctly running or it is just me. Thank you!

doriguzzi commented 8 years ago

@ElisaRojas: right, it seems that the network element that should be controlled by the firewall module is not controlled at all.

pablomuri commented 8 years ago

@ElisaRojas I had the same issue with your configuration.

*** Ping: testing ping reachability
alice -> bob X www 
bob -> alice X www 
charlie -> X X X 
www -> alice bob X 
*** Results: 50% dropped (6/12 received)

ElisaRojas commented 8 years ago

Thanks for your feedback @pablomuri and @doriguzzi ! Have you tried the configuration @schwabe ? Any clues of what is going on or if it is easily solvable? Thank you!

ElisaRojas commented 8 years ago

Hi @schwabe ,

As you asked me in the telco, I tried just pinging a single time from Charlie to Alice and this is the error trace in the log:display:

 from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=280, totalLen=42, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -29, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -29, 10, 0, 0, 3, 0, 0, 0, 0, 0, 0, 10, 0, 0, 1])]
2016-02-24 12:52:58,107 | ERROR | ool-20-thread-22 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | An exception occurred while handling shim message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)[:1.8.0_66]
    at java.util.ArrayList.get(ArrayList.java:429)[:1.8.0_66]
    at eu.netide.core.caos.execution.SequentialFlowExecutor.executeFlow(SequentialFlowExecutor.java:47)
    at eu.netide.core.caos.CompositionManager.OnShimMessage(CompositionManager.java:131)
    at Proxya6bd1cba_8af5_4983_969b_cdd8ad9d4156.OnShimMessage(Unknown Source)
    at Proxyd468eba5_7697_4a86_8007_00838e99f952.OnShimMessage(Unknown Source)
    at eu.netide.core.connectivity.ShimManager.lambda$OnDataReceived$4(ShimManager.java:61)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2016-02-24 12:52:59,103 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-24 12:52:59,104 | INFO  | ool-20-thread-23 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=281, totalLen=42, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -29, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -29, 10, 0, 0, 3, 0, 0, 0, 0, 0, 0, 10, 0, 0, 1])]
2016-02-24 12:52:59,104 | ERROR | ool-20-thread-23 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | An exception occurred while handling shim message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)[:1.8.0_66]
    at java.util.ArrayList.get(ArrayList.java:429)[:1.8.0_66]
    at eu.netide.core.caos.execution.SequentialFlowExecutor.executeFlow(SequentialFlowExecutor.java:47)
    at eu.netide.core.caos.CompositionManager.OnShimMessage(CompositionManager.java:131)
    at Proxya6bd1cba_8af5_4983_969b_cdd8ad9d4156.OnShimMessage(Unknown Source)
    at Proxyd468eba5_7697_4a86_8007_00838e99f952.OnShimMessage(Unknown Source)
    at eu.netide.core.connectivity.ShimManager.lambda$OnDataReceived$4(ShimManager.java:61)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]
2016-02-24 12:53:00,104 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'shim'.
2016-02-24 12:53:00,104 | INFO  | ool-20-thread-22 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=60,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=282, totalLen=42, inPort=1, reason=NO_MATCH, data=[-1, -1, -1, -1, -1, -1, 0, 0, 0, 0, 0, -29, 8, 6, 0, 1, 8, 0, 6, 4, 0, 1, 0, 0, 0, 0, 0, -29, 10, 0, 0, 3, 0, 0, 0, 0, 0, 0, 10, 0, 0, 1])]
2016-02-24 12:53:00,106 | ERROR | ool-20-thread-22 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | An exception occurred while handling shim message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
    at java.util.ArrayList.rangeCheck(ArrayList.java:653)[:1.8.0_66]
    at java.util.ArrayList.get(ArrayList.java:429)[:1.8.0_66]
    at eu.netide.core.caos.execution.SequentialFlowExecutor.executeFlow(SequentialFlowExecutor.java:47)
    at eu.netide.core.caos.CompositionManager.OnShimMessage(CompositionManager.java:131)
    at Proxya6bd1cba_8af5_4983_969b_cdd8ad9d4156.OnShimMessage(Unknown Source)
    at Proxyd468eba5_7697_4a86_8007_00838e99f952.OnShimMessage(Unknown Source)
    at eu.netide.core.connectivity.ShimManager.lambda$OnDataReceived$4(ShimManager.java:61)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)[:1.8.0_66]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_66]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_66]
    at java.lang.Thread.run(Thread.java:745)[:1.8.0_66]

schwabe commented 8 years ago

I am not getting that error.

When I try the compositon the firewall module the firewall module gets the packet ins and just does not repsond to them:

eceived Message body: 01:0a:00:74:00:00:00:00:00:00:02:3f:00:62:00:01:00:00:00:00:00:00:00:ea:00:00:00:00:00:e3:08:00:45:00:00:54:cf:55:00:00:40:01:97:47:0a:00:00:03:0a:00:00:0a:00:00:b9:05:64:5c:00:24:43:dd:cd:56:00:00:00:00:0d:73:05:00:00:00:00:00:10:11:12:13:14:15:16:17:18:19:1a:1b:1c:1d:1e:1f:20:21:22:23:24:25:26:27:28:29:2a:2b:2c:2d:2e:2f:30:31:32:33:34:35:36:37
START calling the handler from backend <ryu.controller.ofp_event.EventOFPPacketIn object at 0x1032d9190>
FIREWALL packet in from dpid:  10  msg: version=0x1,msg_type=0xa,msg_len=0x74,xid=0x0,OFPPacketIn(buffer_id=575,data='\x00\x00\x00\x00\x00\xea\x00\x00\x00\x00\x00\xe3\x08\x00E\x00\x00T\xcfU\x00\x00@\x01\x97G\n\x00\x00\x03\n\x00\x00\n\x00\x00\xb9\x05d\\\x00$C\xdd\xcdV\x00\x00\x00\x00\rs\x05\x00\x00\x00\x00\x00\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567',in_port=1,reason=0,total_len=98)
END calling the handler from backend

So in my tests the firewall not behaving as it should.

schwabe commented 8 years ago

If I switch to the simple switch configuration on the fly (loadcomposition /Users/arne/software/netide-engine/core/specification/MinimalSwitchSpecificationNoFence.xml) the host start pinging. If I switch back to the ABCW file it stops ping.

@doriguzzi can you check why the firewall backend module does behave different when using the java core?

ElisaRojas commented 8 years ago

@schwabe You're right, the error was temporary. I tried again twice and no errors, and I get a similar log from Ryu's backend.

A log:display for datapathID=10 (everything seems correct):

2016-02-24 22:04:41,396 | INFO  | ool-20-thread-19 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | CompositionManager received message from shim: OpenFlowMessage [Header=MessageHeader [Version=VERSION_1_2,Type=OPENFLOW,Length=116,ModuleId=0,TransactionId=0,DatapathId=10],Type=PACKET_IN,OFMessage=OFPacketInVer10(xid=0, bufferId=407, totalLen=98, inPort=1, reason=NO_MATCH, data=[0, 0, 0, 0, 0, -31, 0, 0, 0, 0, 0, -29, 8, 0, 69, 0, 0, 84, 106, -96, 0, 0, 64, 1, -4, 5, 10, 0, 0, 3, 10, 0, 0, 1, 0, 0, -20, -89, 34, 3, 0, 1, -23, 26, -50, 86, 0, 0, 0, 0, 118, 15, 5, 0, 0, 0, 0, 0, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55])]
2016-02-24 22:04:41,396 | INFO  | pool-24-thread-6 | ModuleCallNodeExecutor           | 67 - core.caos - 1.0.0.0-SNAPSHOT | Sending Request to module '350'...
2016-02-24 22:04:41,396 | INFO  | pool-24-thread-6 | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Sending message 'Message [Header=MessageHeader [Version=VERSION_1_1,Type=OPENFLOW,Length=0,ModuleId=350,TransactionId=1694,DatapathId=10],Payload=[1, 10, 0, 116, 0, 0, 0, 0, 0, 0, 1, -105, 0, 98, 0, 1, 0, 0, 0, 0, 0, 0, 0, -31, 0, 0, 0, 0, 0, -29, 8, 0, 69, 0, 0, 84, 106, -96, 0, 0, 64, 1, -4, 5, 10, 0, 0, 3, 10, 0, 0, 1, 0, 0, -20, -89, 34, 3, 0, 1, -23, 26, -50, 86, 0, 0, 0, 0, 118, 15, 5, 0, 0, 0, 0, 0, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55]]' to backend 'backend-ryu-7973'.
2016-02-24 22:04:41,396 | INFO  | pool-24-thread-6 | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Sending to via relay to 'backend-ryu-7973'.
2016-02-24 22:04:41,396 | INFO  | pool-24-thread-6 | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Sent.
2016-02-24 22:04:41,396 | INFO  | pool-24-thread-6 | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Waiting for request with id '350' to complete...
2016-02-24 22:04:41,407 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'backend-ryu-7973'.
2016-02-24 22:04:41,407 | INFO  | tor Receive Loop | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from backend 'backend-ryu-7973' with moduleId '350'.
2016-02-24 22:04:41,407 | INFO  | tor Receive Loop | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data completes request (Message [Header=MessageHeader [Version=VERSION_1_2,Type=FENCE,Length=0,ModuleId=350,TransactionId=1694,DatapathId=0],Payload=[]]).
2016-02-24 22:04:41,407 | INFO  | pool-24-thread-6 | ModuleCallNodeExecutor           | 67 - core.caos - 1.0.0.0-SNAPSHOT | Request returned from module '350'.
2016-02-24 22:04:41,408 | INFO  | ool-20-thread-19 | SequentialFlowExecutor           | 67 - core.caos - 1.0.0.0-SNAPSHOT | End of sequential flow reached. Collected 0 messages.
2016-02-24 22:04:41,408 | INFO  | ool-20-thread-19 | CompositionManager               | 67 - core.caos - 1.0.0.0-SNAPSHOT | Flow execution finished.
2016-02-24 22:04:42,330 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from 'backend-ryu-7973'.
2016-02-24 22:04:42,330 | INFO  | tor Receive Loop | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Data received from backend 'backend-ryu-7973' with moduleId '940'.
2016-02-24 22:04:42,330 | INFO  | tor Receive Loop | BackendManager                   | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Received unrequested Message: 'Message [Header=MessageHeader [Version=VERSION_1_2,Type=TOPOLOGY_UPDATE,Length=0,ModuleId=940,TransactionId=0,DatapathId=0],Payload=[]]'. Relaying to shim.
2016-02-24 22:04:42,330 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Sending to via relay to 'shim'.
2016-02-24 22:04:42,330 | INFO  | tor Receive Loop | ZeroMQBasedConnector             | 66 - core.connectivity - 1.0.0.0-SNAPSHOT | Sent.

Ok, so what I see is that rules are not being installed in the switches. For example, in the case of alice ping -c 1 charlie, which only works for the Python core, I see the following table entries:

Python core

netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s11
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=3.564s, table=0, n_packets=1, n_bytes=98, idle_timeout=5, idle_age=3, ip,in_port=1,dl_src=00:00:00:00:00:e3,dl_dst=00:00:00:00:00:e1 actions=output:2
 cookie=0x0, duration=2.731s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=2, ip,in_port=1,dl_src=ff:ff:ff:ff:ff:ff,dl_dst=3e:9e:ed:60:ab:0d actions=output:2
 cookie=0x0, duration=3.568s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=3, ip,in_port=2,dl_src=00:00:00:00:00:e1,dl_dst=00:00:00:00:00:e3 actions=output:1
 cookie=0x0, duration=2.734s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=2, ip,in_port=2,dl_src=3e:9e:ed:60:ab:0d,dl_dst=ff:ff:ff:ff:ff:ff actions=output:1
 cookie=0x0, duration=4.601s, table=0, n_packets=0, n_bytes=0, idle_age=4, tcp,in_port=2,tp_src=80 actions=output:1
 cookie=0x0, duration=4.601s, table=0, n_packets=0, n_bytes=0, idle_age=4, tcp,in_port=1,nw_dst=10.0.0.10,tp_dst=80 actions=output:2
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s22
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=5.896s, table=0, n_packets=1, n_bytes=42, idle_timeout=5, idle_age=0, in_port=1,dl_dst=00:00:00:00:00:e3 actions=output:3
 cookie=0x0, duration=5.904s, table=0, n_packets=2, n_bytes=140, idle_timeout=5, idle_age=0, in_port=3,dl_dst=00:00:00:00:00:e1 actions=output:1
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s21
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=7.348s, table=0, n_packets=1, n_bytes=42, idle_timeout=5, idle_age=2, in_port=2,dl_dst=00:00:00:00:00:e3 actions=output:1
 cookie=0x0, duration=7.376s, table=0, n_packets=2, n_bytes=140, idle_timeout=5, idle_age=2, in_port=1,dl_dst=00:00:00:00:00:e1 actions=output:2
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s23
NXST_FLOW reply (xid=0x4):

Java core

netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s11
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=3.681s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=3, in_port=2,dl_dst=00:00:00:00:00:e3 actions=output:1
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s21
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=7.606s, table=0, n_packets=1, n_bytes=42, idle_timeout=5, idle_age=2, in_port=2,dl_dst=00:00:00:00:00:e3 actions=output:1
 cookie=0x0, duration=2.586s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=2, in_port=1,dl_dst=00:00:00:00:00:e1 actions=output:2
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s22
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=9.972s, table=0, n_packets=1, n_bytes=42, idle_timeout=5, idle_age=4, in_port=1,dl_dst=00:00:00:00:00:e3 actions=output:3
 cookie=0x0, duration=4.886s, table=0, n_packets=0, n_bytes=0, idle_timeout=5, idle_age=4, in_port=3,dl_dst=00:00:00:00:00:e1 actions=output:1
netide@netide-VirtualBox:~/NetIDE$ sudo ovs-ofctl dump-flows s23
NXST_FLOW reply (xid=0x4):

What I first miss are the two static table entries that s11 should have (two FlowMods sent at the beginning of the Firewall app, which do not time out). And then, entries for s21 and s22 are created correctly, but s11 still has no new dynamic flow entries (as in the Python core).

@schwabe : could you specifically check the behaviour at startup? I don't see any errors, but I don't see the table entries either... ( @doriguzzi : once again, correct me if I'm wrong about the app behaviour or whatever).

schwabe commented 8 years ago

@doriguzzi, @ElisaRojas So the firewall needs proactive flows to work?

doriguzzi commented 8 years ago

Yes, it installs some pro-active rules at start time. This behaviour should not be a problem for DPID-based composition. However, if it can't be solved in the core, we can modify the app. @ElisaRojas BTW, which version of OF are you testing?

ElisaRojas commented 8 years ago

@doriguzzi Right now I'm using the OF1.0. Are there functional differences with the OF1.3 version? (I just took a quick look and didn't see functional differences, but maybe...)

doriguzzi commented 8 years ago

In theory they provide the same functionalities. But, since they are implemented slightly differently because of the differences between OF versions, different behaviours are possible.

ElisaRojas commented 8 years ago

Ok, more news in case it helps debugging...

I just tested the combo: Ryu backend + Java core + ODL shim and it gives exactly the same output as with the Ryu shim everytime i tried. So initially, the "problem" should be between the backend and the core.

By the way, once ODL gave me this output, but just once:

*** Ping: testing ping reachability
alice -> bob X www 
bob -> alice charlie www 
charlie -> X X X 
www -> alice bob X 
*** Results: 41% dropped (7/12 received)

One more ping than expected, but still fewer than the correct ones. In any case, this was just once and I don't know if temporary or not, because I could not reproduce it any more (I always got 6/12 when repeating).

schwabe commented 8 years ago

Commit 9c5d4255f76820965020f5472e11244154620df9 should fix this issue