search

fog / fog

The Ruby cloud services library.
http://fog.github.io
MIT License
4.32k stars 1.47k forks source link

Flavours in OpenNebula not applied #3205

Closed pmartins21 closed 9 years ago

pmartins21 commented 10 years ago

We're currently investigating how we can implement fog to deploy VMs into our OpenNebula-Cloud. After some test I found a problem with I'cant fix or find a solution for it.

When I create a VM threw the OpenNebula provider, the vm still is deployed with 1 VCPU and 128MB of memory. I have tested this with the flavours of the template and also override the flavours of the template. In the code snippets below there is a example how I did these tests.

irb(main):001:0> require './fog.rb' => true irb(main):002:0> irb(main):003:0* irb(main):004:0* irb(main):005:0* con = Fog::Compute.new( irb(main):006:1* { irb(main):007:2* :provider => 'OpenNebula', irb(main):008:2* :opennebula_username => 'user', irb(main):009:2* :opennebula_password => 'xxxxxxx', irb(main):010:2* :opennebula_endpoint => 'http://ip.ip.ip.ip:2633/RPC2' irb(main):011:2> } irb(main):012:1> ) => #<Fog::Compute::OpenNebula::Real:8497580 @client=#<OpenNebula::Client:0x00000000a63768 @one_auth="oneadmin:ibda", @one_endpoint="http://ip.ip.ip.ip:2633/RPC2", @server=#<XMLRPC::Client:0x00000000a61008 @http_header_extra={"accept-encoding"=>"identity"}, @http_last_response=nil, @cookie=nil, @host="ip.ip.ip.ip", @path="/RPC2", @proxy_host=nil, @proxy_port=nil, @use_ssl=false, @timeout=30, @port=2633, @password=nil, @user=nil, @auth=nil, @http=#, @parser=#<OpenNebula::NokogiriStreamParser:0x000000012ad8b0 @parser_class=OpenNebula::NokogiriStreamParser::NokogiriParser>, @create=nil>>> irb(main):013:0> newvm = con.servers.new => <Fog::Compute::OpenNebula::Server id=nil, template_str=nil, name=nil, uuid=nil, state=nil, status=nil, ip=nil, mac=nil, vcpu=nil, cpu=nil, memory=nil, user=nil, gid=nil, group=nil, onevm_object=nil, flavor=nil

irb(main):014:0> newvm.flavor = con.flavors.get 19 => <Fog::Compute::OpenNebula::Flavor id="19", name="one-hook", content="CPU=\"1\"\nDISK=[\n CACHE=\"none\",\n DEV_PREFIX=\"vd\",\n DRIVER=\"qcow2\",\n IMAGE=\"one-hook-test-1\",\n IMAGE_UNAME=\"oneadmin\" ]\nGRAPHICS=[\n KEYMAP=\"de\",\n LISTEN=\"0.0.0.0\",\n TYPE=\"VNC\" ]\nMEMORY=\"1024\"\nOS=[\n ARCH=\"x86_64\",\n BOOT=\"network\" ]\nVCPU=\"2\"", cpu="1", vcpu="2", memory="1024", sched_requirements=nil, sched_rank=nil, sched_ds_requirements=nil, sched_ds_rank=nil, disk={"CACHE"=>"none", "DEV_PREFIX"=>"vd", "DRIVER"=>"qcow2", "IMAGE"=>"one-hook-test-1", "IMAGE_UNAME"=>"oneadmin"}, nic=[], os={"ARCH"=>"x86_64", "BOOT"=>"network"}, graphics={"KEYMAP"=>"de", "LISTEN"=>"0.0.0.0", "TYPE"=>"VNC"}, raw=nil

irb(main):015:0> newvm.flavor.vcpu = 2 => 2 irb(main):016:0> newvm.flavor.memory = 2048 => 2048 irb(main):017:0> network = client.networks.get 1 NameError: undefined local variable or method client' for main:Object from (irb):17 from /usr/bin/irb:12:in

' irb(main):018:0> network = con.networks.get 1 => <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

irb(main):019:0> nic = con.interfaces.new({ :vnet => network, :model => "virtio"}) => <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

, model="virtio", name=nil, mac=nil

irb(main):020:0> newvm.flavor.nic = [ nic ] => [ <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

, model="virtio", name=nil, mac=nil ] irb(main):021:0> newvm.name = "FooBarVM" => "FooBarVM" irb(main):022:0> newvm.save => <Fog::Compute::OpenNebula::Server id=141, template_str=nil, name="FooBarVM", uuid=141, state="LCM_INIT", status=1, ip="192.168.30.3", mac="02:00:c0:a8:1e:03", vcpu=nil, cpu="1", memory="128", user="oneadmin", gid=0, group="oneadmin", onevm_object=#<OpenNebula::VirtualMachine:0x00000002071708 @xml=[#<Nokogiri::XML::Element:0xf75c60 name="VM" children=[#<Nokogiri::XML::Element:0xf756e8 name="ID" children=[#<Nokogiri::XML::Text:0xf78578 "141">]>, #<Nokogiri::XML::Element:0xf78488 name="UID" children=[#<Nokogiri::XML::Text:0xf78460 "0">]>, #<Nokogiri::XML::Element:0xf78370 name="GID" children=[#<Nokogiri::XML::Text:0xf782d0 "0">]>, #<Nokogiri::XML::Element:0xf78244 name="UNAME" children=[#<Nokogiri::XML::Text:0xf7821c "oneadmin">]>, #<Nokogiri::XML::Element:0xf780c8 name="GNAME" children=[#<Nokogiri::XML::Text:0xf78050 "oneadmin">]>, #<Nokogiri::XML::Element:0xf74e64 name="NAME" children=[#<Nokogiri::XML::Text:0xf77fc4 "FooBarVM">]>, #<Nokogiri::XML::Element:0xf77e48 name="PERMISSIONS" children=[#<Nokogiri::XML::Element:0xf77de4 name="OWNER_U" children=[#<Nokogiri::XML::Text:0xf77dbc "1">]>, #<Nokogiri::XML::Element:0xf77bf0 name="OWNER_M" children=[#<Nokogiri::XML::Text:0xf77b78 "1">]>, #<Nokogiri::XML::Element:0xf77ab0 name="OWNER_A" children=[#<Nokogiri::XML::Text:0xf77970 "0">]>, #<Nokogiri::XML::Element:0xf7781c name="GROUP_U" children=[#<Nokogiri::XML::Text:0xf77704 "0">]>, #<Nokogiri::XML::Element:0xf7754c name="GROUP_M" children=[#<Nokogiri::XML::Text:0xf77524 "0">]>, #<Nokogiri::XML::Element:0xf773e4 name="GROUP_A" children=[#<Nokogiri::XML::Text:0xf77330 "0">]>, #<Nokogiri::XML::Element:0xf772b8 name="OTHER_U" children=[#<Nokogiri::XML::Text:0xf77290 "0">]>, #<Nokogiri::XML::Element:0xf771f0 name="OTHER_M" children=[#<Nokogiri::XML::Text:0xf77150 "0">]>, #<Nokogiri::XML::Element:0xf77100 name="OTHER_A" children=[#<Nokogiri::XML::Text:0xf770c4 "0">]>]>, #<Nokogiri::XML::Element:0xf77038 name="LAST_POLL" children=[#<Nokogiri::XML::Text:0xf76ffc "0">]>, #<Nokogiri::XML::Element:0xf76f70 name="STATE" children=[#<Nokogiri::XML::Text:0xf76f20 "1">]>, #<Nokogiri::XML::Element:0xf76e6c name="LCM_STATE" children=[#<Nokogiri::XML::Text:0xf76e44 "0">]>, #<Nokogiri::XML::Element:0xf76cf0 name="RESCHED" children=[#<Nokogiri::XML::Text:0xf76c50 "0">]>, #<Nokogiri::XML::Element:0xf76bec name="STIME" children=[#<Nokogiri::XML::Text:0xf76bb0 "1413542116">]>, #<Nokogiri::XML::Element:0xf76b10 name="ETIME" children=[#<Nokogiri::XML::Text:0xf76ae8 "0">]>, #, #<Nokogiri::XML::Element:0x10bc86c name="MEMORY" children=[#<Nokogiri::XML::Text:0x10bc844 "0">]>, #<Nokogiri::XML::Element:0x10bc7f4 name="CPU" children=[#<Nokogiri::XML::Text:0x10bc7b8 "0">]>, #<Nokogiri::XML::Element:0x10bc754 name="NET_TX" children=[#<Nokogiri::XML::Text:0x10bc72c "0">]>, #<Nokogiri::XML::Element:0x10bc6c8 name="NET_RX" children=[#<Nokogiri::XML::Text:0x10bc6a0 "0">]>, #<Nokogiri::XML::Element:0x10bc63c name="TEMPLATE" children=[#<Nokogiri::XML::Element:0x10bc5d8 name="AUTOMATIC_REQUIREMENTS" children=[#<Nokogiri::XML::CDATA:0x10bc5b0 "CLUSTER_ID = 100 & !(PUBLIC_CLOUD = YES)">]>, #<Nokogiri::XML::Element:0x10bc560 name="CPU" children=[#<Nokogiri::XML::CDATA:0x10bc524 "0.2">]>, #<Nokogiri::XML::Element:0x10bc498 name="DISK" children=[#<Nokogiri::XML::Element:0x10bc3d0 name="CACHE" children=[#<Nokogiri::XML::CDATA:0x10bc3a8 "none">]>, #<Nokogiri::XML::Element:0x10bc330 name="CLONE" children=[#<Nokogiri::XML::CDATA:0x10bc2e0 "YES">]>, #<Nokogiri::XML::Element:0x10bc290 name="CLONE_TARGET" children=[#<Nokogiri::XML::CDATA:0x10bc254 "SYSTEM">]>, #<Nokogiri::XML::Element:0x10bc1f0 name="DATASTORE" children=[#<Nokogiri::XML::CDATA:0x10bc128 "SYNO-IMAGE">]>, #<Nokogiri::XML::Element:0x10bc060 name="DATASTORE_ID" children=[#<Nokogiri::XML::0x10bc038 "106">]>, #<Nokogiri::XML::Element:0x10bbfe8 name="DEV_PREFIX" children=[#<Nokogiri::XML::CDATA:0x10bbfc0 "vd">]>, #<Nokogiri::XML::Element:0x10bbf70 name="DISK_ID" children=[#<Nokogiri::XML::CDATA:0x10bbf48 "0">]>, #<Nokogiri::XML::Element:0x10bbea8 name="DRIVER" children=[#<Nokogiri::XML::CDATA:0x10bbe30 "qcow2">]>, #<Nokogiri::XML::Element:0x10bbdcc name="IMAGE" children=[#<Nokogiri::XML::CDATA:0x10bbda4 "one-hook-test-1">]>, #<Nokogiri::XML::Element:0x10bbd18 name="IMAGE_ID" children=[#<Nokogiri::XML::CDATA:0x10bbcf0 "16">]>, #<Nokogiri::XML::Element:0x10bbc8c name="IMAGE_UNAME" children=[#<Nokogiri::XML::CDATA:0x10bbc64 "oneadmin">]>, #<Nokogiri::XML::Element:0x10bbc14 name="LN_TARGET" children=[#<Nokogiri::XML::CDATA:0x10bbbec "NONE">]>, #<Nokogiri::XML::Element:0x10bbb9c name="READONLY" children=[#<Nokogiri::XML::CDATA:0x10bbb74 "NO">]>, #<Nokogiri::XML::Element:0x10bbb24 name="SAVE" children=[#<Nokogiri::XML::CDATA:0x10bbafc "NO">]>, #<Nokogiri::XML::Element:0x10bbaac name="SIZE" children=[#<Nokogiri::XML::CDATA:0x10bba84 "30720">]>, #<Nokogiri::XML::Element:0x10bb980 name="SOURCE" children=[#<Nokogiri::XML::CDATA:0x10bb908 "/var/lib/one//datastores/106/1e3d99c3f01fd22e12f453339abf0389">]>, #<Nokogiri::XML::Element:0x10bb82c name="TARGET" children=[#<Nokogiri::XML::CDATA:0x10bb73c "vda">]>, #<Nokogiri::XML::Element:0x10bb6b0 name="TM_MAD" children=[#<Nokogiri::XML::CDATA:0x10bb624 "shared">]>, #<Nokogiri::XML::Element:0x10bb584 name="TYPE" children=[#<Nokogiri::XML::CDATA:0x10bb55c "FILE">]>]>, #<Nokogiri::XML::Element:0x10bb4bc name="GRAPHICS" children=[#<Nokogiri::XML::Element:0x10bb444 name="KEYMAP" children=[#<Nokogiri::XML::CDATA:0x10bb41c "de">]>, #<Nokogiri::XML::Element:0x10bb3cc name="LISTEN" children=[#<Nokogiri::XML::CDATA:0x10bb3a4 "0.0.0.0">]>, #<Nokogiri::XML::Element:0x10bb354 name="PORT" children=[#<Nokogiri::XML::CDATA:0x10bb32c "6041">]>, #<Nokogiri::XML::Element:0x10bb2c8 name="TYPE" children=[#<Nokogiri::XML::CDATA:0x10bb2a0 "VNC">]>]>, #<Nokogiri::XML::Element:0x10bb200 name="MEMORY" children=[#<Nokogiri::XML::CDATA:0x10bb1d8 "128">]>, #<Nokogiri::XML::Element:0x10bb188 name="NIC" children=[#<Nokogiri::XML::Element:0x10bb0d4 name="BRIDGE" children=[#<Nokogiri::XML::CDATA:0x10bb0ac "vmnet0">]>, #<Nokogiri::XML::Element:0x10baeb8 name="CLUSTER_ID" children=[#<Nokogiri::XML::CDATA:0x10bae90 "100">]>, #<Nokogiri::XML::Element:0x10baddc name="IP" children=[#<Nokogiri::XML::CDATA:0x10badb4 "192.168.30.3">]>, #<Nokogiri::XML::Element:0x10bad64 name="IP6_LINK" children=[#<Nokogiri::XML::CDATA:0x10bad28 "fe80::400:c0ff:fea8:1e03">]>, #<Nokogiri::XML::Element:0x10bac9c name="MAC" children=[#<Nokogiri::XML::CDATA:0x10bac74 "02:00:c0:a8:1e:03">]>, #<Nokogiri::XML::Element:0x10bac24 name="MODEL" children=[#<Nokogiri::XML::CDATA:0x10babe8 "virtio">]>, #<Nokogiri::XML::Element:0x10bab98 name="NETWORK" children=[#<Nokogiri::XML::CDATA:0x10bab70 "LAB LAN">]>, #<Nokogiri::XML::Element:0x10bab0c name="NETWORK_ID" children=[#<Nokogiri::XML::CDATA:0x10baae4 "1">]>, #<Nokogiri::XML::Element:0x10baa44 name="NIC_ID" children=[#<Nokogiri::XML::CDATA:0x10ba9e0 "0">]>, #<Nokogiri::XML::Element:0x10ba968 name="VLAN" children=[#<Nokogiri::XML::CDATA:0x10ba918 "NO">]>]>, #<Nokogiri::XML::Element:0xf40830 name="OS" children=[#<Nokogiri::XML::Element:0xf4077c name="ARCH" children=[#<Nokogiri::XML::CDATA:0xf40704 "x86_64">]>, #<Nokogiri::XML::Element:0xf406a0 name="BOOT" children=[#<Nokogiri::XML::CDATA:0xf40664 "network">]>]>, #<Nokogiri::XML::Element:0xf4054c name="VCPU" children=[#<Nokogiri::XML::CDATA:0xf40510 "1">]>, #<Nokogiri::XML::Element:0xf40484 name="VMID" children=[#<Nokogiri::XML::CDATA:0xf4045c "141">]>]>, #, #]>], @client=#<OpenNebula::Client:0x00000000a63768 @one_auth="oneadmin:ibda", @one_endpoint="http://192.168.30.22:2633/RPC2", @server=#<XMLRPC::Client:0x00000000a61008 @http_header_extra={"accept-encoding"=>"identity"}, @http_last_response=#<Net::HTTPOK 200 OK readbody=true>, @cookie=nil, @host="192.168.30.22", @path="/RPC2", @proxy_host=nil, @proxy_port=nil, @use_ssl=false, @timeout=30, @port=2633, @password=nil, @user=nil, @auth=nil, @http=#, @parser=#<OpenNebula::NokogiriStreamParser:0x000000012ad8b0 @parser_class=OpenNebula::NokogiriStreamParser::NokogiriParser>, @create=#<XMLRPC::Create:0x000000012ce470 @writer=#>>>, @pe_id=141, @name="FooBarVM">, flavor= <Fog::Compute::OpenNebula::Flavor id="19", name="one-hook", content="CPU=\"1\"\nDISK=[\n CACHE=\"none\",\n DEV_PREFIX=\"vd\",\n DRIVER=\"qcow2\",\n IMAGE=\"one-hook-test-1\",\n IMAGE_UNAME=\"oneadmin\" ]\nGRAPHICS=[\n KEYMAP=\"de\",\n LISTEN=\"0.0.0.0\",\n TYPE=\"VNC\" ]\nMEMORY=\"1024\"\nOS=[\n ARCH=\"x86_64\",\n BOOT=\"network\" ]\nVCPU=\"2\"", cpu="1", vcpu=2, memory=2048, sched_requirements=nil, sched_rank=nil, sched_ds_requirements=nil, sched_ds_rank=nil, disk={"CACHE"=>"none", "DEV_PREFIX"=>"vd", "DRIVER"=>"qcow2", "IMAGE"=>"one-hook-test-1", "IMAGE_UNAME"=>"oneadmin"}, nic=[ <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan="" , model="virtio", name=nil, mac=nil ], os={"ARCH"=>"x86_64", "BOOT"=>"network"}, graphics={"KEYMAP"=>"de", "LISTEN"=>"0.0.0.0", "TYPE"=>"VNC"}, raw=nil

Please let me know if I'm doing something wrong or if this is a current bug.

Thanks a lot in advance and for all the work of this great project.

geemus commented 10 years ago

Unfortunately I'm not the most familiar with opennebula in particular. I would expect for an operation like this, however, that the flavor id is the only thing likely to be paid attention to (such that if you want different values than the flavor, you should pick a different flavor, instead of trying to modify the flavor). Does that help/make sense?

pmartins21 commented 10 years ago

I didn't exactly understand your suggestion. The OpenNebula API provides you facts/variables in template which can be overridden. Afterwards you are able to deploy VM's with your created template. I'm not sure but I guess in one of the modified fog version of NETWAYS( https://github.com/NETWAYS/fog) this was already working.

Thanks in advance.

geemus commented 10 years ago

Hmm. One thing I was just noticing. I don't think the save actually will try to update the local copy after the fact. Maybe try reloading the server and see if the attributes are then closer to what you expect? If so we could probably change things around to do this automatically.

Otherwise...

Thanks for linking to the NETWAYS stuff, unfortunately I don't see any obvious/related changes to this particular issue.

It looks like the allocate tries to build an arg based on the flavor here: https://github.com/fog/fog/blob/master/lib/fog/opennebula/requests/compute/vm_allocate.rb#L16

The string in question is then generated here: https://github.com/fog/fog/blob/master/lib/fog/opennebula/models/compute/flavor.rb#L28

I don't know enough about what should actually be done here to know what might be going wrong there, but hopefully that can point you in the right direction to figure something out.

pmartins21 commented 10 years ago

I will try troubleshooting that this weekend and will give you feedback on monday.

Thanks.

ghost commented 10 years ago

In our test environment we installed a OpenNebula V4.8 instance for testing the "fog to OpenNebula" conversation. At the client side we installed several versions of fog: 1.21 (netways), 1.23 (official) and 1.24 (official).

With all of them we tried to create a virtual machine defined by the OpenNebula template with template ID "19". The creation of the new virtual machine was done within a interactive ruby shell.

We logged all console output and traced the network between client (fog) and server (OpenNebula).

Our analysis shows the following results with all tested fog versions:

  1. Client <-> Server communication via XMLRPC

1.1  Query vm template ( client-> server)

POST /RPC2 HTTP/1.1
User-Agent: XMLRPC::Client (Ruby 1.9.3)
Content-Type: text/xml; charset=utf-8
Content-Length: 294
Connection: close
Accept-Encoding: identity
Accept: */*
Host: 192.168.30.22:2633
<?xml version="1.0" encoding="UTF-8"?>

<methodCall>
   <methodName>one.templatepool.info</methodName>
   <params>
      <param>
         <value>
            <string>username:password</string>
         </value>
      </param>
      <param>
         <value>
            <i4>-2</i4>
         </value>
      </param>
      <param>
         <value>
            <i4>19</i4>
         </value>
      </param>
      <param>
         <value>
            <i4>19</i4>
         </value>
      </param>
   </params>
</methodCall>

1.2 OpenNebula response (server -> client)

HTTP/1.1 200 OK
Content-type: text/xml; charset=utf-8
Content-length: 1858
Connection: close
Date: Mon, 27 Oct 2014 12:28:39 UTC
Server: Xmlrpc-c_Abyss/1.33.6
<?xml version="1.0" encoding="UTF-8"?>

<methodResponse>
<params>
<param><value><array><data>
<value><boolean>1</boolean></value>
<value><string><VMTEMPLATE_POOL><VMTEMPLATE>
  <ID>19</ID>
  <UID>0</UID>
  <GID>0</GID>
  <UNAME>oneadmin</UNAME>
  <GNAME>oneadmin</GNAME>
  <NAME>one-hook</NAME>
  <PERMISSIONS>
    <OWNER_U>1</OWNER_U>
    <OWNER_M>1</OWNER_M>
    <OWNER_A>0</OWNER_A>
    <GROUP_U>0</GROUP_U>
    <GROUP_M>0</GROUP_M>
    <GROUP_A>0</GROUP_A>
    <OTHER_U>0</OTHER_U>
    <OTHER_M>0</OTHER_M>
    <OTHER_A>0</OTHER_A>
  </PERMISSIONS>
  <REGTIME>1412677728</REGTIME>
  <TEMPLATE>
    <CPU><![CDATA[1]]></CPU>
    <DISK>
      <CACHE><![CDATA[none]]></CACHE>
      <DEV_PREFIX><![CDATA[vd]]></DEV_PREFIX>
      <DRIVER><![CDATA[qcow2]]></DRIVER>
      <IMAGE><![CDATA[one-hook-test-1]]></IMAGE>
      <IMAGE_UNAME><![CDATA[oneadmin]]></IMAGE_UNAME>
    </DISK>
    <GRAPHICS>
      <KEYMAP><![CDATA[de]]></KEYMAP>
      <LISTEN><![CDATA[0.0.0.0]]></LISTEN>
      <TYPE><![CDATA[VNC]]></TYPE>
    </GRAPHICS>
    <MEMORY><![CDATA[1024]]></MEMORY>
    <OS>
      <ARCH><![CDATA[x86_64]]></ARCH>
      <BOOT><![CDATA[network]]></BOOT>
    </OS>
    <VCPU><![CDATA[2]]></VCPU>
  </TEMPLATE>
</VMTEMPLATE></VMTEMPLATE_POOL></string></value>
<value><i4>0</i4></value>
</data></array></value></param>
</params>
</methodResponse>

As you can see, the values for RAM (MEMORY) and CPU (VCPU) are like this:

MEMORY[CDATA[1024]] = 1024 MB = 1GB RAM

VCPU[CDATA[2]] = 2VCPU = 2 CPU CORES 

1.3 Allocate the new virtual machine (client -> server)

POST /RPC2 HTTP/1.1
User-Agent: XMLRPC::Client (Ruby 1.9.3)
Content-Type: text/xml; charset=utf-8
Content-Length: 548
Connection: close
Accept-Encoding: identity
Accept: */*
Host: 192.168.30.22:2633
<?xml version="1.0" encoding="UTF-8"?>

<methodCall>
   <methodName>one.vm.allocate</methodName>
   <params>
      <param>
         <value>
            <string>username:password</string>
         </value>
      </param>
      <param>
         <value>
            <string>CPU=0.2

VCPU=1

MEMORY=128

DISK=["CACHE"="none", "DEV_PREFIX"="vd", "DRIVER"="qcow2", "IMAGE"="one-hook-test-1", "IMAGE_UNAME"="oneadmin"]

NIC=[MODEL="virtio",NETWORK_ID="1"]

OS=["ARCH"="x86_64", "BOOT"="network"]

GRAPHICS=["KEYMAP"="de", "LISTEN"="0.0.0.0", "TYPE"="VNC"]

NAME=fog-test</string>
         </value>
      </param>
      <param>
         <value>
            <boolean>0</boolean>
         </value>
      </param>
   </params>
</methodCall>

As you can see, the values for RAM and CPU differs from the values in the template:

VCPU = 1 = 1 CPU CORE MEMORY = 128 = 128 MB RAM

The new VM is allocated with these (wrong) values. There is no difference if we changed the template or adjust the fog commands / vm.flavours in the direct ruby-calls  e.g. "newvm.memory = 1024".

If you are interested in the CLI output or the PCAP traces please let us know.

We could run such test (for example with a special / patched fog version) in our testlab with nearly no delay.

Please let us know if we should try some things.

Best regards.

geemus commented 10 years ago

Could you add some notes on what fog code you are doing that results in those invocations as well? Just want to make sure I have all the info to be on the same page before I head down a potentially dead end.

ghost commented 9 years ago

irb(main):003:0* require './fog.rb' => true irb(main):004:0> con = Fog::Compute.new( irb(main):005:1* { irb(main):006:2* :provider => 'OpenNebula', irb(main):007:2* :opennebula_username => 'xxx', irb(main):008:2* :opennebula_password => 'xxx', irb(main):009:2* :opennebula_endpoint => 'http://192.168.30.22:2633/RPC2' irb(main):010:2> } irb(main):011:1> ) => #<Fog::Compute::OpenNebula::Real:16297940 @client=#<OpenNebula::Client:0x00000002078e18 @one_auth="xxx:xxx", @one_endpoint="http://192.168.30.22:2633/RPC2", @server=#<XMLRPC::Client:0x000000020764b0 @http_header_extra={"accept-encoding"=>"identity"}, @http_last_response=nil, @cookie=nil, @host="192.168.30.22", @path="/RPC2", @proxy_host=nil, @proxy_port=nil, @use_ssl=false, @timeout=30, @port=2633, @password=nil, @user=nil, @auth=nil, @http=#<Net::HTTP 192.168.30.22:2633 open=false>, @parser=#<OpenNebula::NokogiriStreamParser:0x00000001e52be8 @parser_class=OpenNebula::NokogiriStreamParser::NokogiriParser>, @create=nil>>> irb(main):012:0> newvm = con.servers.new => <Fog::Compute::OpenNebula::Server id=nil, template_str=nil, name=nil, uuid=nil, state=nil, status=nil, ip=nil, mac=nil, vcpu=nil, cpu=nil, memory=nil, user=nil, gid=nil, group=nil, onevm_object=nil, flavor=nil

irb(main):013:0> newvm.flavor = con.flavors.get 19 => <Fog::Compute::OpenNebula::Flavor id="19", name="one-hook", content="CPU=\"1\"\nDISK=[\n CACHE=\"none\",\n DEV_PREFIX=\"vd\",\n DRIVER=\"qcow2\",\n IMAGE=\"one-hook-test-1\",\n IMAGE_UNAME=\"oneadmin\" ]\nGRAPHICS=[\n KEYMAP=\"de\",\n LISTEN=\"0.0.0.0\",\n TYPE=\"VNC\" ]\nMEMORY=\"1024\"\nOS=[\n ARCH=\"x86_64\",\n BOOT=\"network\" ]\nVCPU=\"2\"", cpu="1", vcpu="2", memory="1024", sched_requirements=nil, sched_rank=nil, sched_ds_requirements=nil, sched_ds_rank=nil, disk={"CACHE"=>"none", "DEV_PREFIX"=>"vd", "DRIVER"=>"qcow2", "IMAGE"=>"one-hook-test-1", "IMAGE_UNAME"=>"oneadmin"}, nic=[], os={"ARCH"=>"x86_64", "BOOT"=>"network"}, graphics={"KEYMAP"=>"de", "LISTEN"=>"0.0.0.0", "TYPE"=>"VNC"}, raw=nil

irb(main):014:0> newvm.name = "fog-test" => "fog-test" irb(main):015:0> con.networks.get 1 => <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

irb(main):016:0> network = con.networks.get 1 => <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

irb(main):017:0> nic = con.interfaces.new({ :vnet => network, :model =>"virtio"}) => <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

, model="virtio", name=nil, mac=nil

irb(main):018:0> newvm.flavor.nic = [ nic ] => [ <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan=""

, model="virtio", name=nil, mac=nil ] irb(main):019:0> newvm.save => <Fog::Compute::OpenNebula::Server id=151, template_str=nil, name="fog-test", uuid=151, state="LCM_INIT", status=1, ip="192.168.30.7", mac="02:00:c0:a8:1e:07", vcpu=nil, cpu="1", memory="128", user="oneadmin", gid=0, group="oneadmin", onevm_object=#<OpenNebula::VirtualMachine:0x000000030b4548 @xml=[#<Nokogiri::XML::Element:0x16f0af8 name="VM" children=[#<Nokogiri::XML::Element:0x16f0788 name="ID" children=[#<Nokogiri::XML::Text:0x16efd9c "151">]>, #<Nokogiri::XML::Element:0x16efd4c name="UID" children=[#<Nokogiri::XML::Text:0x16efd24 "0">]>, #<Nokogiri::XML::Element:0x16efcd4 name="GID" children=[#<Nokogiri::XML::Text:0x16efcac "0">]>, #<Nokogiri::XML::Element:0x16efc48 name="UNAME" children=[#<Nokogiri::XML::Text:0x16efbe4 "oneadmin">]>, #<Nokogiri::XML::Element:0x16efb94 name="GNAME" children=[#<Nokogiri::XML::Text:0x16efb58 "oneadmin">]>, #<Nokogiri::XML::Element:0x16f015c name="NAME" children=[#<Nokogiri::XML::Text:0x16efaf4 "fog-test">]>, #<Nokogiri::XML::Element:0x16efa90 name="PERMISSIONS" children=[#<Nokogiri::XML::Element:0x16efa2c name="OWNER_U" children=[#<Nokogiri::XML::Text:0x16ef9f0 "1">]>, #<Nokogiri::XML::Element:0x198c640 name="OWNER_M" children=[#<Nokogiri::XML::Text:0x198c618 "1">]>, #<Nokogiri::XML::Element:0x198c5c8 name="OWNER_A" children=[#<Nokogiri::XML::Text:0x198c5a0 "0">]>, #<Nokogiri::XML::Element:0x198c550 name="GROUP_U" children=[#<Nokogiri::XML::Text:0x198c528 "0">]>, #<Nokogiri::XML::Element:0x198c4d8 name="GROUP_M" children=[#<Nokogiri::XML::Text:0x198c4b0 "0">]>, #<Nokogiri::XML::Element:0x198c460 name="GROUP_A" children=[#<Nokogiri::XML::Text:0x198c438 "0">]>, #<Nokogiri::XML::Element:0x198c3d4 name="OTHER_U" children=[#<Nokogiri::XML::Text:0x198c398 "0">]>, #<Nokogiri::XML::Element:0x198c320 name="OTHER_M" children=[#<Nokogiri::XML::Text:0x198c2f8 "0">]>, #<Nokogiri::XML::Element:0x198c2a8 name="OTHER_A" children=[#<Nokogiri::XML::Text:0x198c280 "0">]>]>, #<Nokogiri::XML::Element:0x198c1f4 name="LAST_POLL" children=[#<Nokogiri::XML::Text:0x198c1cc "0">]>, #<Nokogiri::XML::Element:0x198c154 name="STATE" children=[#<Nokogiri::XML::Text:0x198c12c "1">]>, #<Nokogiri::XML::Element:0x198c0c8 name="LCM_STATE" children=[#<Nokogiri::XML::Text:0x198c0a0 "0">]>, #<Nokogiri::XML::Element:0x198c03c name="RESCHED" children=[#<Nokogiri::XML::Text:0x198c014 "0">]>, #<Nokogiri::XML::Element:0x198bfc4 name="STIME" children=[#<Nokogiri::XML::Text:0x198bf9c "1414412941">]>, #<Nokogiri::XML::Element:0x198bf4c name="ETIME" children=[#<Nokogiri::XML::Text:0x198bf10 "0">]>, #, #<Nokogiri::XML::Element:0x198bdf8 name="MEMORY" children=[#<Nokogiri::XML::Text:0x198bdd0 "0">]>, #<Nokogiri::XML::Element:0x198bd80 name="CPU" children=[#<Nokogiri::XML::Text:0x198bd58 "0">]>, #<Nokogiri::XML::Element:0x198bcf4 name="NET_TX" children=[#<Nokogiri::XML::Text:0x198bccc "0">]>, #<Nokogiri::XML::Element:0x198bc7c name="NET_RX" children=[#<Nokogiri::XML::Text:0x198bc54 "0">]>, #<Nokogiri::XML::Element:0x198bbdc name="TEMPLATE" children=[#<Nokogiri::XML::Element:0x198bb78 name="AUTOMATIC_REQUIREMENTS" children=[#<Nokogiri::XML::CDATA:0x198bb50 "CLUSTER_ID = 100 & !(PUBLIC_CLOUD = YES)">]>, #<Nokogiri::XML::Element:0x198bb00 name="CPU" children=[#<Nokogiri::XML::CDATA:0x198bad8 "0.2">]>, #<Nokogiri::XML::Element:0x198ba88 name="DISK" children=[#<Nokogiri::XML::Element:0x198ba10 name="CACHE" children=[#<Nokogiri::XML::CDATA:0x198b9e8 "none">]>, #<Nokogiri::XML::Element:0x198b984 name="CLONE" children=[#<Nokogiri::XML::CDATA:0x198b948 "YES">]>, #<Nokogiri::XML::Element:0x198b8f8 name="CLONE_TARGET" children=[#<Nokogiri::XML::CDATA:0x198b8d0 "SYSTEM">]>, #<Nokogiri::XML::Element:0x198b880 name="DATASTORE" children=[#<Nokogiri::XML::CDATA:0x198b858 "SYNO-IMAGE">]>, #<Nokogiri::XML::Element:0x198b7f4 name="DATASTORE_ID" children=[#<Nokogiri::XML::CDATA:0x198b7cc "106">]>, #<Nokogiri::XML::Element:0x198b77c name="DEV_PREFIX" children=[#<Nokogiri::XML::CDATA:0x198b754 "vd">]>, #<Nokogiri::XML::Element:0x198b704 name="DISK_ID" children=[#<Nokogiri::XML::CDATA:0x198b6dc "0">]>, #<Nokogiri::XML::Element:0x198b68c name="DRIVER" children=[#<Nokogiri::XML::CDATA:0x198b664 "qcow2">]>, #<Nokogiri::XML::Element:0x198b614 name="IMAGE" children=[#<Nokogiri::XML::CDATA:0x198b5ec "one-hook-test-1">]>, #<Nokogiri::XML::Element:0x198b588 name="IMAGE_ID" children=[#<Nokogiri::XML::CDATA:0x198b560 "16">]>, #<Nokogiri::XML::Element:0x198b510 name="IMAGE_UNAME" children=[#<Nokogiri::XML::CDATA:0x198b4e8 "oneadmin">]>, #<Nokogiri::XML::Element:0x198b498 name="LN_TARGET" children=[#<Nokogiri::XML::CDATA:0x198b470 "NONE">]>, #<Nokogiri::XML::Element:0x198b40c name="READONLY" children=[#<Nokogiri::XML::CDATA:0x198b3e4 "NO">]>, #<Nokogiri::XML::Element:0x198b394 name="SAVE" children=[#<Nokogiri::XML::CDATA:0x198b36c "NO">]>, #<Nokogiri::XML::Element:0x198b31c name="SIZE" children=[#<Nokogiri::XML::CDATA:0x198b2f4 "30720">]>, #<Nokogiri::XML::Element:0x198b290 name="SOURCE" children=[#<Nokogiri::XML::CDATA:0x198b254 "/var/lib/one//datastores/106/1e3d99c3f01fd22e12f453339abf0389">]>, #<Nokogiri::XML::Element:0x198b204 name="TARGET" children=[#<Nokogiri::XML::CDATA:0x198b1c8 "vda">]>, #<Nokogiri::XML::Element:0x198b13c name="TM_MAD" children=[#<Nokogiri::XML::CDATA:0x198b0d8 "shared">]>, #<Nokogiri::XML::Element:0x198b04c name="TYPE" children=[#<Nokogiri::XML::CDATA:0x198b024 "FILE">]>]>, #<Nokogiri::XML::Element:0x198af0c name="GRAPHICS" children=[#<Nokogiri::XML::Element:0x198ada4 name="KEYMAP" children=[#<Nokogiri::XML::CDATA:0x198ad18 "de">]>, #<Nokogiri::XML::Element:0x198abc4 name="LISTEN" children=[#<Nokogiri::XML::CDATA:0x198ab38 "0.0.0.0">]>, #<Nokogiri::XML::Element:0x198aa20 name="PORT" children=[#<Nokogiri::XML::CDATA:0x198a994 "6051">]>, #<Nokogiri::XML::Element:0x198a890 name="TYPE" children=[#<Nokogiri::XML::CDATA:0x198a818 "VNC">]>]>, #<Nokogiri::XML::Element:0x198a700 name="MEMORY" children=[#<Nokogiri::XML::CDATA:0x198a69c "128">]>, #<Nokogiri::XML::Element:0x198f188 name="NIC" children=[#<Nokogiri::XML::Element:0x198f0e8 name="AR_ID" children=[#<Nokogiri::XML::CDATA:0x198f070 "0">]>, #<Nokogiri::XML::Element:0x198efbc name="BRIDGE" children=[#<Nokogiri::XML::CDATA:0x198ef94 "vmnet0">]>, #<Nokogiri::XML::Element:0x198eeb8 name="CLUSTER_ID" children=[#<Nokogiri::XML::CDATA:0x198ee90 "100">]>, #<Nokogiri::XML::Element:0x198ee40 name="IP" children=[#<Nokogiri::XML::CDATA:0x198ee18 "192.168.30.7">]>, #<Nokogiri::XML::Element:0x198edc8 name="MAC" children=[#<Nokogiri::XML::CDATA:0x198ed8c "02:00:c0:a8:1e:07">]>, #<Nokogiri::XML::Element:0x198ed3c name="MODEL" children=[#<Nokogiri::XML::CDATA:0x198ed14 "virtio">]>, #<Nokogiri::XML::Element:0x198ecc4 name="NETWORK" children=[#<Nokogiri::XML::CDATA:0x198ec9c "LAB LAN">]>, #<Nokogiri::XML::Element:0x198ec4c name="NETWORK_ID" children=[#<Nokogiri::XML::CDATA:0x198ec24 "1">]>, #<Nokogiri::XML::Element:0x198ebd4 name="NIC_ID" children=[#<Nokogiri::XML::CDATA:0x198eb84 "0">]>, #<Nokogiri::XML::Element:0x198e88c name="VLAN" children=[#<Nokogiri::XML::CDATA:0x198e814 "NO">]>]>, #<Nokogiri::XML::Element:0x198e6e8 name="OS" children=[#<Nokogiri::XML::Element:0x198e670 name="ARCH" children=[#<Nokogiri::XML::CDATA:0x198e634 "x86_64">]>, #<Nokogiri::XML::Element:0x198e5a8 name="BOOT" children=[#<Nokogiri::XML::CDATA:0x198e530 "network">]>]>, #<Nokogiri::XML::Element:0x198e454 name="VCPU" children=[#<Nokogiri::XML::CDATA:0x198e3dc "1">]>, #<Nokogiri::XML::Element:0x198e2c4 name="VMID" children=[#<Nokogiri::XML::CDATA:0x198e288 "151">]>]>, #, #]>], @client=#<OpenNebula::Client:0x00000002078e18 @one_auth="xxx", @one_endpoint="http://192.168.30.22:2633/RPC2", @server=#<XMLRPC::Client:0x000000020764b0 @http_header_extra={"accept-encoding"=>"identity"}, @http_last_response=#<Net::HTTPOK 200 OK readbody=true>, @cookie=nil, @host="192.168.30.22", @path="/RPC2", @proxy_host=nil, @proxy_port=nil, @use_ssl=false, @timeout=30, @port=2633, @password=nil, @user=nil, @auth=nil, @http=#<Net::HTTP 192.168.30.22:2633 open=false>, @parser=#<OpenNebula::NokogiriStreamParser:0x00000001e52be8 @parser_class=OpenNebula::NokogiriStreamParser::NokogiriParser>, @create=#<XMLRPC::Create:0x000000021d7a20 @writer=#>>>, @pe_id=151, @name="fog-test">, flavor= <Fog::Compute::OpenNebula::Flavor id="19", name="one-hook", content="CPU=\"1\"\nDISK=[\n CACHE=\"none\",\n DEV_PREFIX=\"vd\",\n DRIVER=\"qcow2\",\n IMAGE=\"one-hook-test-1\",\n IMAGE_UNAME=\"oneadmin\" ]\nGRAPHICS=[\n KEYMAP=\"de\",\n LISTEN=\"0.0.0.0\",\n TYPE=\"VNC\" ]\nMEMORY=\"1024\"\nOS=[\n ARCH=\"x86_64\",\n BOOT=\"network\" ]\nVCPU=\"2\"", cpu="1", vcpu="2", memory="1024", sched_requirements=nil, sched_rank=nil, sched_ds_requirements=nil, sched_ds_rank=nil, disk={"CACHE"=>"none", "DEV_PREFIX"=>"vd", "DRIVER"=>"qcow2", "IMAGE"=>"one-hook-test-1", "IMAGE_UNAME"=>"oneadmin"}, nic=[ <Fog::Compute::OpenNebula::Interface id=nil, vnet= <Fog::Compute::OpenNebula::Network id="1", name="LAB LAN", uid="0", gid="0", description="", vlan="" , model="virtio", name=nil, mac=nil ], os={"ARCH"=>"x86_64", "BOOT"=>"network"}, graphics={"KEYMAP"=>"de", "LISTEN"=>"0.0.0.0", "TYPE"=>"VNC"}, raw=nil

irb(main):020:0>

geemus commented 9 years ago

It looks like the nic stuff passed through correctly there, but that maybe there were not any calls to adjust CPU or Memory. Apologies if I'm missing something there though.

pmartins21 commented 9 years ago

You're right.

geemus commented 9 years ago

Sorry for all the back and forth, tricky to debug without access to a system of this type. Does the output you pasted still deviate from your expectations then? If so, how? Thanks!

pmartins21 commented 9 years ago

I believe this is very hard to debug without any system. If it's easier for you I'll try to provide you an testing environment? I guess the output looks good but the call which is made to the OpenNebula API seems to be wrong. semakra described this problem quiet good which each call which is made to OpenNebula. Let me know if you need further assistance.

Thanks!

geemus commented 9 years ago

I've attempted to summon some extra help from another opennebula contributor, so hopefully they will have enough additional context to get closer.

andrewjamesbrown commented 9 years ago

Hi @pmartins21, PR https://github.com/fog/fog/pull/3255 should fix this for you. I ran into the same issue and fixed it with the PR. Note that there is some additional checking to ensure VCPU >= CPU, and sets them equal if not, in case you are trying to do that.

geemus commented 9 years ago

@andrewjamesbrown great, thanks for checking this out. I suspected they might be related, but wasn't certain. I'm going to close this then, but @pmartins21 please reopen if it doesn't fix things for you. I'm hoping to get a release out with these and other fixes soon also. Thanks!

plribeiro3000 commented 9 years ago

@pmartins21 Can you confirm it is fixed?

plribeiro3000 commented 9 years ago

Closing as stated by @geemus.

pmartins21 commented 9 years ago

Hi,

sorry for my delayed answer. I checked it out and can confirm that the problem is fixed. Only one thing left, as @andrewjamesbrown already described the CPU and VCPU should be equal if modified in the template. Is this already known issue?

Great job guys!

Thanks a lot!

geemus commented 9 years ago

I was not aware of the further issue. Could you open up a new issue with details and we can perhaps work from there? Thanks!