ViniGarcia / NIEP

MIT License
2 stars 0 forks source link

NIEP cli not started any guide to start the nodes on it #2

Open jaki300 opened 3 years ago

jaki300 commented 3 years ago

Import Executor not found Import Parse not found

ViniGarcia commented 3 years ago

Hi jaki300, thanks for the report.

It seems there is some problem with the local libraries importing, maybe some path inconsistency. I installed and executed the platform in a Ubuntu 16 sandbox here, and the error was replicated when I called the CLI.py outside the CLI folder.

This error occurs because there is a relative path defined in the CLI.py file ("path.insert(0, '../TOPO-MAN/')"). Thus if your origin execution folder is different from the CLI folder, it loses the file importing reference. I will fix these relative paths in the next few days to enable the users to call the CLI.py from everywhere. At the moment, there are two possible solutions to your problem: i) Open the CLI folder in your terminal and then run the CLI.py; ii) Change the relative importing path in the CLI.py file to make the importing available from your origin execution folder. Please, remember to execute the platform with python2.7 and superuser privileges.

Regarding the guide, we do not have this particular type of documentation yet. However, in the next few days, I will record a video to demonstrate the NIEP installing process and a simple network service instantiation and management.

For now, if you want to run a very simple sample service, here is very summarized step-by-step: -> Run the CLI.py -> Run "define" operation with the sampling service ("define /home/user/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json" -> Turn on the network service ("topoup") -> Do management operations (you can use the mininet, VNF, and SFC prompts -- type "help" in the CLI for more information) -> Turn off the network service ("topodown")

Take care to turn down the service through the NIEP platform. In another case (if you kill the process, for example), to instantiate the same service again, you will need to delete the respective VNF/VM folders in "NIEP/VEM/IMAGES" and also delete the VMs configuration through the virsh CLI or virt-manager.

As soon as I release the tutorial video, I will paste the link here.

jaki300 commented 3 years ago

HI thanks for the Answer, still require one demo to work on it.. required to perform some NFV simulation not much familiar with the topology of nfv can you guide us on this and when will we expect the video

Found this on Topoup niep> topoup libvirt: Capabilities Utils error : invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm Traceback (most recent call last): File "CLI.py", line 463, in NIEPCLI().cmdloop() File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop stop = self.onecmd(line) File "/usr/lib/python2.7/cmd.py", line 221, in onecmd return func(arg) File "CLI.py", line 89, in do_topoup self.NIEPEXE.topologyUp() File "../TOPO-MAN/Executer.py", line 187, in topologyUp VNFINSTANCE.upVNF() File "../VEM/VNF.py", line 349, in upVNF VIRT_CONNECTION.defineXML(domainXML) File "/usr/lib/python2.7/dist-packages/libvirt.py", line 3650, in defineXML if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self) libvirt.libvirtError: invalid argument: could not find capabilities for arch=x86_64 domaintype=kvm

ViniGarcia commented 3 years ago

Hi jaki300, thanks for the report.

Yes, you are right. There are some details in the SFC model that are not straightforward. Thus a guide would be most welcome. This task was included in our TODO list with priority.

Regarding the reported error, it seems to be related to the virtualization of the KVM hypervisor. There are some possibilities to be checked to solve it: i) Incorrect installation of libvirt or qemu-system ii) Virtualization support of your CPU is not active (VT-x of Intel, for example) iii) NIEP is running in a VM and the CPU does not support nested virtualization iv) You are using an X86 operational system

A simple test would be trying to instantiate a virtual machine by using the virsh CLI or the Virt-Manager GI, checking if some error occurs.

jaki300 commented 3 years ago

Thanks for your answer there is a nested virtualization issue with qemu cpu finding a way still not able to run this in qemu cpu Let me know if you have any information regarding the qemu nested virtualization enabling

tavares-tn commented 3 years ago

Hi. I'm Tavares, a vini colleague, and I think I can help. While the virtual machine is in the OFF state, run the following command on the Hyper-V physical host. This enables nested virtualization for the virtual machine.

Set-VMProcessor -VMName -ExposeVirtualizationExtensions $true

This command can help you

Em sex., 12 de fev. de 2021 às 09:51, jaki300 notifications@github.com escreveu:

Thanks for your answer there is a nested virtualization issue with qemu cpu finding a way still not able to run this in qemu cpu Let me know if you have any information regarding the qemu nested virtualization enabling

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ViniGarcia/NIEP/issues/2#issuecomment-778177409, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQYJV4JCXXDL35RS3W27PLS6UP6NANCNFSM4XMTBV4Q .

--

Atenciosamente

Tavares


Thales N. Tavares Tecnólogo em Redes de Computadores Mestrando do Programa de Pós-Graduação em Ciência da Computação (PPGCC) Universidade Federal de Santa Maria - UFSM

Skype: tavarestn Cel.: 055 984547664 Lattes: http://lattes.cnpq.br/3936087905598280

[image: Mailtrack] https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5& Remetente notificado por Mailtrack https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5& 02/12/21, 11:13:27 AM

jaki300 commented 3 years ago

Thanks for the Answer I am using eve-ng and in the lab I am using ubuntu system is running this is not working.

The hierarchy of VM is like this

1) Centos 7 host physical 2) eve-ng VM 3) ubuntu NEIP in lab

jaki300 commented 3 years ago

niep> define /root/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json niep> topoup niep> Error: moveIntf: not successfully moved to HOST01: Cannot find device "2453" Error: moveIntf: not successfully moved to HOST01: Cannot find device "2453"

Error: moveIntf: not successfully moved to HOST01: Cannot find device "2453" Error: moveIntf: not successfully moved to HOST01: Cannot find device "2453" *** gave up after 3 retries niep> define /root/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json niep> topoclean NO TOPOLOGY DEFINED niep> define /root/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json niep> topoup NO TOPOLOGY DEFINED

kvm issue resolved... root@ubuntu:~/NIEP/CLI# kvm-ok INFO: /dev/kvm exists KVM acceleration can be used

tavares-tn commented 3 years ago

Debian 7 was used for implementation

Em sex., 12 de fev. de 2021 às 11:27, jaki300 notifications@github.com escreveu:

niep> define /root/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json niep> topoclean NO TOPOLOGY DEFINED niep> define /root/NIEP/EXAMPLES/DEFINITIONS/SFCExample.json niep> topoup NO TOPOLOGY DEFINED

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ViniGarcia/NIEP/issues/2#issuecomment-778227174, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQYJVZGHZKGFWM2BTJG75DS6U3EBANCNFSM4XMTBV4Q .

--

Atenciosamente

Tavares


Thales N. Tavares Tecnólogo em Redes de Computadores Mestrando do Programa de Pós-Graduação em Ciência da Computação (PPGCC) Universidade Federal de Santa Maria - UFSM

Skype: tavarestn Cel.: 055 984547664 Lattes: http://lattes.cnpq.br/3936087905598280

ViniGarcia commented 3 years ago

Hi jaki300, thanks for the report.

It seems to be a Mininet error, particularly when the system sets up the network interfaces in the Mininet hosts. The error during the "topoup" operation undefines the topology, thus causing the "TOPOLOGY UNDEFINED" message. I will explore this Mininet error.

I am right now working on the tutorial video in the Ubuntu 16 environment. Till the end of the weekend, hopefully, I release it. Maybe, some particular doubts could be solved by using it too.

As soon as I get some clue about the Mininet error, I post it here.

PS: Thanks for the previous answers Tavares :)

ViniGarcia commented 3 years ago

Hi jaki300.

I exploited the Mininet used in the NIEP platform in the Ubuntu environment. In particular versions, such as 16.04.06, some errors occur when installing an old version of Mininet. Unfortunately, I could not identify why it occurs. Anyway, I committed a new installer.sh using the Mininet version from the standard Ubuntu repository. This version worked in all my tests here, maybe it could solve your problem.

If the error persists, please contact us.

ViniGarcia commented 3 years ago

Hi Jaki300, here is a brief tutorial of NIEP installation and usage with demo videos, hope it helps:

https://docs.google.com/presentation/d/e/2PACX-1vSuS-3_7BMIaioXhqmlGuGkvf36IZvKu8GRQnOle1gYHUOwGxcnAroJi5Y5dJQ4oGABe7uf5RN5GzBw/pub?start=false&loop=false&delayms=3000

Best regards

jaki300 commented 3 years ago

Thank you very much for the videos I will get back if face any issue, currently not able to up the top-up checking the python versioning issue.

jaki300 commented 3 years ago

thank you so much...these are soo good demo video to understand the project.One more question which OS is used for the demo..of the viedos I think its ubuntu16.04 server

ViniGarcia commented 3 years ago

Hi jaki300, Thanks for the reply, I am happy that the demo could help. :)

Yes, I executed the demo experiments using an Ubuntu 16.04 vanilla (except for the Mate interface). If you have any other doubt, please contact us.

Best regards

jaki300 commented 3 years ago

HI NIEP is running fine on the Ubuntu 16.04.7 . want to know more for the router and firewall configuration any guide for the firewall and router stuff..

ViniGarcia commented 3 years ago

Hi jaki300,

Do you mean the Click-designed router and firewall (VNFs)? Or is it about the router configurations of Mininet?

Anyway, for this answer, I will assume that you are trying to use the Click functions from VNF-REPO (please, correct me in the next post if it is not).

These both functions are static Click functions. The firewall only allows the transmission of ARP and TCP packets, dropping anything else. The router uses two Click classifiers: the first one is for network layer three, and the second one is designed for network layer four. The router, in turn, routes packets based on a static configuration (note that the MACs and IPs are hard-coded -- you can change that takes into account your executing scenario).

You are not limited to network functions in VNF-REPO. You can freely develop other functions based on the Click framework (using the DPDK module for input and output packets) -- you can get some extra information about the framework here: https://github.com/kohler/click

@lmarcuzzo can you provide some extra information to jaki300? In particular, information about the router.

jaki300 commented 3 years ago

i need create this topology that is mention in the document http://www.inf.ufrgs.br/~granville/wp-content/papercite-data/pdf/08432239.pdf And i think this i poosible with on click functions

ViniGarcia commented 3 years ago

Yes, it is possible to instantiate this topology using local NIEP modules with no problems. I will include some examples of router and firewall usage in the EXAMPLES folder.

As soon as I do it, I post it here.

jaki300 commented 3 years ago

@ViniGarcia Are you able to add the router and firewall examples

ViniGarcia commented 3 years ago

Hi jaki300,

I have been working on new features for NIEP, some of them (such as fine-grained MAC definition for the elements) will help you create your topology.

Anyway, I included an example of the router click function usage in the EXAMPLE folder and preconfigured the router click function to work with it. To set up the router example, you will need to follow the commands in next:

define /path/to/NIEP/EXAMPLES/DEFINITIONS/SFCSingleRouterExample.json topoup vnf x vnfaction replace /path/to/NIEP/VNF-REPO/router.click vnfaction start exit mininet HOST01 route add -net 192.168.122.0 netmask 255.255.255.0 gw 192.168.121.1 HOST02 route add -net 192.168.121.0 netmask 255.255.255.0 gw 192.168.122.1 HOST01 ping HOST02

Please, let me know if you get any errors in the deployment process. In the next few days, I will include a firewall example in the repository. As soon as I do it, I post it here.

Best regards

ViniGarcia commented 3 years ago

Hi Jaki300,

I updated the firewall example and create a topology for running it, the first one located in the "VNF-REPO" folder, and the second one included in the "EXAMPLE" folder. To set up this topology, you should attempt to the following commands:

define /path/to/NIEP/EXAMPLES/DEFINITIONS/SFCSingleFirewallExample.json topoup vnf x vnfaction replace /path/to/NIEP/VNF-REPO/firewall.click vnfaction start exit mininet CLIENT01 ping SERVER01 SERVER01 ping CLIENT01 CLIENT01 ping SERVER02 SERVER02 ping CLIENT01 CLIENT01 ping SERVER03 SERVER03 ping CLIENT01

ICMP communication is only allowed between SERVER01 and CLIENT01. In the other cases, firewall rules are dropping the traffic from CLIENT to SERVER. I commented on each filter of the firewall in the .click file. Thus, you can check and change the rules there. Hope it helps.

Best regards

jaki300 commented 3 years ago

Hi jaki300,

I have been working on new features for NIEP, some of them (such as fine-grained MAC definition for the elements) will help you create your topology.

Anyway, I included an example of the router click function usage in the EXAMPLE folder and preconfigured the router click function to work with it. To set up the router example, you will need to follow the commands in next:

define /path/to/NIEP/EXAMPLES/DEFINITIONS/SFCSingleRouterExample.json topoup vnf x vnfaction replace /path/to/NIEP/VNF-REPO/router.click vnfaction start exit mininet HOST01 route add -net 192.168.122.0 netmask 255.255.255.0 gw 192.168.121.1 HOST02 route add -net 192.168.121.0 netmask 255.255.255.0 gw 192.168.122.1 HOST01 ping HOST02

Please, let me know if you get any errors in the deployment process. In the next few days, I will include a firewall example in the repository. As soon as I do it, I post it here.

Best regards

FIrst of all sorry for the late reply bussy in other tasks..

i got this when running rotuter json ERROR: Invalid vnf configurations provided (DEFINE/PARSER)

jaki300 commented 3 years ago

Its working fine issue is old created topology not cleared with topoclean command Thank you soo much for giving time and sharing the doc and stuff. I wanna say one more thing can we adjust the terminal to gain its values if mistakenly go back to niep> mode and command history and bash completion attachment is good for cli users? One thing more same one more demo video on the project itself like directories and other stuff intro

ViniGarcia commented 3 years ago

Hi jaki300,

Fine! I am glad to be able to help.

Yes, there are updates scheduled for the CLI interface. Recently I have similar difficulties regarding history. I will adjust a new history schema very soon and post the update notice here. Good idea about the new tutorial video, as soon as possible, we will release a "developers tutorial" stuff.

Best regards

ViniGarcia commented 3 years ago

Hi jaki300,

I have updated the repository with an improved version of CLI.py Now the NIEP's CLI has an auto-complete system and a persistent history. I hope it helps with your work. :)

Best regards

jaki300 commented 3 years ago

HI i am creating my own topology and the issue comes in connection setup as 1host-1router-1firewall-2router-2client. 1firewall-server1,server2,server3 let me know if you can create this I am missed this with the link connections failed all time.

can we add one router in the firewall topology for demo purposes 3 different networks how they interact with each other using firewall? is this poossible

ViniGarcia commented 3 years ago

Hi jaki300,

Can you provide me a graph drawing of your topology? (you can do that by e-mail -- vfulber@inf.ufsm.br) Thus I can better understand your connection problem and try to create a generic topology that solves it.

If I understood well, it is not possible to create an interaction with different networks with the click firewall provided in the project. However, it is natural to use the firewall function connected to a router function, creating a small service chain to do both filtering and routing actions.

Please, when you send the topology image, let me know here in the GitHub topic (sometimes e-mails from new addresses are redirected to spam, so I need to check it carefully).

Best regards

jaki300 commented 3 years ago

topology picture uploaded over the email either you can solve over picture topology or below mention flow.

C1> R1> F1>S1 allowed C2 > R1> F1> S2 allowed C3 > R1> F1> S3 disallowed where C1 and C2 are in one network c3 have different network and S1, S2,S3 have different network 3 networks in total C- client, R=router, F=Firewall, S= servers

ViniGarcia commented 3 years ago

Hi jaki300,

I received your e-mail, the image made very clear the aimed topology. My feeling is that it is possible to create it with the current version of NIEP.

If I understood well, C in your description is H in the image, and S is SV. In the image S is a switch, right?

As soon as possible, I will try to create this topology and push it as part of NIEP examples. I let you know about the update on this topic.

Best regards.

jaki300 commented 3 years ago

If I understood well, C in your description is H in the image, and S is SV. In the image S is a switch, right? yes in image H is host,S is switch,FW firewall, R Router and SV is server, hopefully you will create this topo and it gives better understanding on routing and connections setup.

ViniGarcia commented 3 years ago

Hi Jaki300,

Thanks for confirming the notations. As soon as possible, I will create the topology.

Best regards

ViniGarcia commented 3 years ago

Hi Jaki300,

I updated the repository with the topology that you requested. The new environment is named ComplexTopology, and it is in the EXAMPLES/DEFINITIONS folder. The topology is structured as follows (consider # as white spaces):

H1 (HOST01 .122.01/24) ---- \ ############################################### / ---- SV1 (SERVER01 122.03/24) ###################### --------- S1 (SWITCH01) ---- FW (z) ---- S3 (SWITCH03) --------- H2 (HOST02 .122.02/24) ---- / ############################################### \ ---- SV2 (SERVER02 122.04/24)

H3 (HOST03 .123.01/24) ---- \
###################### --------- S2 (SWITCH02) ---- R1 (x) ---- SV1 (SERVER03 124.01/24) H4 (HOST04 .123.02/24) ---- /

I did a single modification in the topology sent by e-mail: insertion of S3 after FW. Once there are two servers, if there is no switch after the network function, it will be necessary to create a particular virtual network interface to communicate each of the servers, besides adjusting the traffic steering, in the firewall function. Actually, a firewall tackling the traffic steering is a very unusual and kind of weird scenario. Anyway, this scenario is possible by removing S3 and reprogramming the firewall click function.

With ComplexTopology, you can execute your scenario by adjusting the click functions with your rules and routes. For the routing function, you must adapt the HOSTs routes too.

If some problem occurs during the deployment, please let me know.

Best regards

jaki300 commented 3 years ago

HI have you run this topo in your env if yes can you share the run steps i think these are three different scenarios merged in one topo right?

jaki300 commented 3 years ago

image

ViniGarcia commented 3 years ago

Hi jaki300,

Yes, I deployed the topology in my local NIEP environment.

There are some problems with your configuration. Taking into account your topology sketch (below), HOST01 does not communicate with HOST03, but only with HOST04 (in the same network) and with SERVER03 (routing required). Another possible problem regards the router.click configuration. Did you configure the .click functions to the ComplexTopology scenario?

As soon as possible, I will post a brief tutorial about this topology in this topic.

image

ViniGarcia commented 3 years ago

When I was configuring the firewall rules I did note that your sketch and your description do not match.

Note that the sketch shows which I call "Complex Topology #1" and your description seems like "Complex Topology #2". Can you confirm which one is the aimed topology?

ComplexTopology

jaki300 commented 3 years ago

the second one "Complex Topology #2" is desired one

ViniGarcia commented 3 years ago

Hi jaki300,

I have updated the repository with both "Complex Topology #1" (now renamed to "DualTopology") and "Complex Topology #2" (renamed to "ComplexTopology"). I have also adapted the Click functions to run according to the expected behavior too. You can find these adapted NFs in EXAMPLES/RESOURCES (there, you will find a .png with a deployment guide). The topologies JSON files locate at EXAMPLES/DEFINITIONS.

Best regards

jaki300 commented 3 years ago

thank you very much for your support

ViniGarcia commented 3 years ago

You're welcome :)