search

openbaton / openstack4j-plugin

Apache License 2.0
3 stars 12 forks source link

Issue 118 #16

Closed RuthDevlaeminck closed 6 years ago

RuthDevlaeminck commented 7 years ago

This is related to NFVO Issue 118. Load the local binary file into openstack using the REST API. These changes do not support the loading of an image via URL which does not work in existing code.

RuthDevlaeminck commented 7 years ago

12 is addressed by this pull request

marcellom commented 7 years ago

Hi @RuthDVerizon, first of all thanks for the pull request. I am currently testing it and I'll let you know.

marcellom commented 7 years ago

I tested your code and I got an error in the following scenario. I uploaded a VNF package in Open Baton referring the image by a url. This is the Metadata:

name: sipp-server
provider: fokus
nfvo_version: 3.2.0
description: "description"
scripts-link: https://github.com/openbaton/vnf-scripts.git
shared: true
image:
    upload: check
    names:
        - sippimage
    link: https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
image-config:
    name: sippimage
    diskFormat: qcow2
    containerFormat: bare
    minCPU: 0
    minDisk: 0
    minRam: 0
    isPublic: true
vim_types:
    - openstack

With this metadata, I want Open Baton to check if an image called "sippimage" already exists in the VIM, if not upload it. Obviously the image doesn't exist because I want to trigger the addImage method. In the moment I upload the VNF package I got an error in the GUI with cause=null. The relevant logs of the openstack4j-plugin are the following:

Waiting for RPC requests
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Received message
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - There are 3 parameters
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Looking for method: addImage
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - -------------------------- adding image
[pool-1-thread-1] WARN org.openbaton.drivers.openstack4j.OpenStack4JDriver - Not found region 'Berlin'. Use default one...
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - ---------------------------imageV2 is : GlanceImage{id=371c060e-30cb-4a17-b79a-48ef24fec90c, name=sippimage, tags=[], imageStatus=null, containerFormat=BARE, diskFormat=QCOW2, createdAt=Mon Apr 03 16:29:27 CEST 2017, updatedAt=Mon Apr 03 16:29:27 CEST 2017, minDisk=0, minRam=0, isProtected=false, checksum=null, owner=f1644ed83f1f473c9ee729779b624766, visibility=PRIVATE, size=null, locations=null, directUrl=null, self=/v2/images/371c060e-30cb-4a17-b79a-48ef24fec90c, file=/v2/images/371c060e-30cb-4a17-b79a-48ef24fec90c/file, schema=/v2/schemas/image, architecture=null, instanceUuid=null, kernelId=null, osVersion=null, osDistro=null, ramdiskId=null, virtualSize=null}
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - image_url is https://cloud-images.ubuntu.com/releases/14.04/release/ubuntu-14.04-server-cloudimg-amd64-disk1.img
[pool-1-thread-1] INFO org.openbaton.drivers.openstack4j.OpenStack4JDriver - could not find image service endpoint
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Answer is: {}
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Reply queue is: amq.gen-1FFMgZhS-w3jFqUUbBxiLA

If I execute the same scenario, with the current code in develop branch, it works. Maybe I am not understanding the purpose of this pull request, so could you explain me what are you going to achieve? Thanks in advance.

RuthDevlaeminck commented 7 years ago

As of our testing of the develop code before submission the use of the url did not work so it does not work in our code either so if changes have been made to support issues #11 and #12 that code will need to be merged into this branch.

The purpose of this code change is to actually load an image file as part of the VNF Package. For example in the following example we include the file mini.iso which can be downloaded from http://archive.ubuntu.com/ubuntu/dists/yakkety/main/installer-amd64/current/images/netboot/mini.iso

Here is an example of the Metadata.yaml file

name: test-ubuntu
vim_types:
  - openstack
image:
    upload: check
    names:
        - test-loading-ubuntu
    imageFile : "mini.iso"
image-config:
    name: test-loading-ubuntu
    diskFormat: QCOW2
    containerFormat: BARE
    minCPU: 2
    minRam: 2048
    minDisk: 5
    isPublic: false

VNF Package will now contain vnfd.json, Metadata.yaml and mini.iso. Please note that these changes also require the changes made in the NFVO code under the branch issue-118.

marcellom commented 7 years ago

Loading the image from the VNF itself is not supported in Open Baton at the moment, with the metadata you provided I get this exception:

org.openbaton.exceptions.NotFoundException: VNFPackageManagement: For option upload=check you must define an image. Neither the image link is defined nor the image file is available. Please define at least one if you want to upload a new image

Did you change the Open Baton code too?

RuthDevlaeminck commented 7 years ago

This code is an enhancement to provide that support to include the image as part of the VNF Package. You must update the NFVO code to the issue-118 branch as well as the openstack4j-plugin to the issue-118 branch.

Please note that if there are no changes to the config files the code will work as it did previously so anyone updating one but not the other will not be affected as long as they do not use the new modification to the config files.

marcellom commented 7 years ago

Ok I didn't know about that branch. Now I tested it but I still get this error from the openstack4j-plugin.

[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Received message
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - There are 3 parameters
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Looking for method: addImage
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - -------------------------- adding image
[pool-1-thread-1] WARN org.openbaton.drivers.openstack4j.OpenStack4JDriver - Not found region 'Berlin'. Use default one...
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - ---------------------------imageV2 is : GlanceImage{id=ebdb856d-5098-4e48-a91f-c2792172a204, name=sippimage-mini, tags=[], imageStatus=null, containerFormat=BARE, diskFormat=QCOW2, createdAt=Mon Apr 03 19:33:45 CEST 2017, updatedAt=Mon Apr 03 19:33:45 CEST 2017, minDisk=0, minRam=0, isProtected=false, checksum=null, owner=f1644ed83f1f473c9ee729779b624766, visibility=PRIVATE, size=null, locations=null, directUrl=null, self=/v2/images/ebdb856d-5098-4e48-a91f-c2792172a204, file=/v2/images/ebdb856d-5098-4e48-a91f-c2792172a204/file, schema=/v2/schemas/image, architecture=null, instanceUuid=null, kernelId=null, osVersion=null, osDistro=null, ramdiskId=null, virtualSize=null}
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - image_url is mini.iso
[pool-1-thread-1] INFO org.openbaton.drivers.openstack4j.OpenStack4JDriver - could not find image service endpoint
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Answer is: {}
[pool-1-thread-1] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Reply queue is: amq.gen-QVV66OhSpJ4cDQjDORUCNw

And this is my metadata:

name: sipp-server
provider: fokus
nfvo_version: 3.2.0
description: "sipp-server description"
scripts-link: https://github.com/openbaton/vnf-scripts.git
shared: true
image:
    upload: check
    names:
        - sippimage-mini
    imageFile: "mini.iso"
image-config:
    name: sippimage-mini
    diskFormat: qcow2
    containerFormat: bare
    minCPU: 0
    minDisk: 0
    minRam: 0
    isPublic: true
vim_types:
    - openstack

Another question, why you didn't implement this method https://github.com/openbaton/openstack4j-plugin/blob/issue-118/src/main/java/org/openbaton/drivers/openstack4j/OpenStack4JDriver.java#L1022 instead of the one for the image_url?

RuthDevlaeminck commented 7 years ago

@marcellom The big issue that was faced in trying to load the images was the issue of running out of stack space. By extracting the image file from the tar file and loading it using a URL we avoid having to process the image within the program.

marcellom commented 7 years ago

@RuthDVerizon Is this feature working? Let me know so I can test it and let you know.

RuthDevlaeminck commented 7 years ago

@marcellom this feature is ready for testing. If you have any questions please let me know.

marcellom commented 7 years ago

Sorry for the late reply. I tested again your pull request. The image file gets uploaded in Open Stack successfully, however there are still some problems:

RuthDevlaeminck commented 7 years ago

Marcello,

Thank you for testing again. I am attempting to work with the develop code but I currently am unable to get NFVO to launch successfully (I have opened issue #178 with additional details). I also noticed that when I go back to commit 0cb94772af9b4938518c13028c687b8e32723bfe in NFVO and attempt to run it with openstack plugin provided in the bootstrap I get the following error upon trying to upload a vnf package (have previously tested this exact tar using older openbaton code successfully).

[pool-1-thread-3] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Reply queue is: amq.gen-kTnYgPFwz-icrqyJUDb7xg [pool-1-thread-9] ERROR org.openbaton.drivers.openstack4j.OpenStack4JDriver - null java.lang.NullPointerException at org.openbaton.drivers.openstack4j.Utils.getSubnet(Utils.java:97) at org.openbaton.drivers.openstack4j.OpenStack4JDriver.getSubnetById(OpenStack4JDriver.java:419) at org.openbaton.drivers.openstack4j.OpenStack4JDriver.listNetworks(OpenStack4JDriver.java:391) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.openbaton.plugin.PluginListener.executeMethod(PluginListener.java:208) at org.openbaton.plugin.PluginListener.run(PluginListener.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [pool-1-thread-9] ERROR org.openbaton.drivers.openstack4j.OpenStack4JDriver - null org.openbaton.exceptions.VimDriverException at org.openbaton.drivers.openstack4j.OpenStack4JDriver.getSubnetById(OpenStack4JDriver.java:422) at org.openbaton.drivers.openstack4j.OpenStack4JDriver.listNetworks(OpenStack4JDriver.java:391) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.openbaton.plugin.PluginListener.executeMethod(PluginListener.java:208) at org.openbaton.plugin.PluginListener.run(PluginListener.java:124) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) [pool-1-thread-9] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Answer is: { "exception": { "stackTrace": [], "suppressedExceptions": [] } } [pool-1-thread-9] DEBUG org.openbaton.drivers.openstack4j.OpenStack4JDriver - Reply queue is: amq.gen-gl5iMgPe0c8qpPUoi9teDQ

The code I have been running successfully is the code in the issue 118 branches for NFVO and openstack plugin.

I will look into the version of the openstack api, thank you for letting me know you are planning to support both.

Sincerely, Ruth

From: Marcello Monachesi notifications@github.com Reply-To: openbaton/openstack4j-plugin reply@reply.github.com Date: Tuesday, July 18, 2017 at 6:18 AM To: openbaton/openstack4j-plugin openstack4j-plugin@noreply.github.com Cc: "Devlaeminck, Ruth Elizabeth" Ruth.Devlaeminck@VerizonWireless.com, Mention mention@noreply.github.com Subject: [E] Re: [openbaton/openstack4j-plugin] Issue 118 (#16)

Sorry for the late reply. I tested again your pull request. The image file gets uploaded in Open Stack successfully, however there are still some problems:

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openbaton_openstack4j-2Dplugin_pull_16-23issuecomment-2D316033778&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=EY5CqGX4EVtYvAbrHGifQxQZmBksxNEgyue2WryUc-yn7NYHkuIeDEM1bZHkxUyX&m=ce-nWsD4OBF3_bR5rBH2gOLSGeegjtjpVHWcr9V_x7w&s=f4InwdDzdVcHNKTKt9oh39BHZKXMLvFFLd_fAlzXp7o&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AZCi7XGkjQX0-5FL4Q1ZdyBPli4lVUsRddks5sPJRwgaJpZM4MsSPF&d=DwMFaQ&c=udBTRvFvXC5Dhqg7UHpJlPps3mZ3LRxpb6__0PomBTQ&r=EY5CqGX4EVtYvAbrHGifQxQZmBksxNEgyue2WryUc-yn7NYHkuIeDEM1bZHkxUyX&m=ce-nWsD4OBF3_bR5rBH2gOLSGeegjtjpVHWcr9V_x7w&s=bKEw6tX9jNxyTiIaE5PNUZ1fEP33eMfPkARXWeWawg4&e=.

lorenzotomasini commented 7 years ago

Hi @RuthDVerizon

the error you are facing is a bug in the openstack plugin due to a missing subnet in a network (mainly missing the rights to see the subnet). It was already fixed in the current openstack-plugin.

RuthDevlaeminck commented 6 years ago

Per discussion held at Verizon a different approach will be used to upload images, closing this pull request.