"Affinity is required for this operation" on VM creation

plribeiro3000 commented 8 years ago

chef/chef-provisioning-fog#123

Is effected by this change.

fog version (1.35.0 -> 1.34.0) fog-core (1.35.0 -> 1.32.1)

Need to figure out what changes for Xen happened in these and see what may have caused this issue.

original issue opened by @jjasghar at fog/fog#3860.

jjasghar commented 8 years ago

Awesome thanks @plribeiro3000 i googled for fog-xen, not fog-xenserver, my mistake in the original post.

fernandes commented 8 years ago

check my comment https://github.com/fog/fog-xenserver/issues/53#issuecomment-170398499

affinity was introduced on XenServer 4.0, and it was optional, as documentation states: "a host which the VM has some affinity for (or NULL). This is used as a hint to the start call when it decides where to run the VM. Implementations are free to ignore this field."

but seems this changed on XenServer 6.5 and its a required field, including when you are using a standalone host with no pool setup.

I think there's no other solution, just specify an affinity :smile:

plribeiro3000 commented 8 years ago

Yeap @fernandes. I guess you will need to work it out in chef-provisiong-fog. Is that something you can do?

fernandes commented 8 years ago

will be a little bit hard for me on next 15 days... if someone want touch chef-provisiong-fog I can help (because I never used this gem).

but the idea is

affinity_server = conn.servers.first
vm.affinity = affinity_server.id

just came into mind, not sure if this code will run smoothly heheheh

plribeiro3000 commented 8 years ago

Oh sure. I guess my last message was kind of confusing. =)

By you i wanted to say anyone. =)

But i agree, the snippet you pasted should do the trick. :wink:

I wont be able to tackle it as well. @jjasghar Is that something you could do it yourself?

fernandes commented 8 years ago

hahahha :+1:

@jjasghar if you need any help, ping me

jjasghar commented 8 years ago

@plribeiro3000 and @fernandes unfortunately i don't have access to Xen at all. :(

fernandes commented 8 years ago

@jjasghar modify on a branch and ask someone to test :sunglasses:

jjasghar commented 8 years ago

True, but that would require me to check this out and then push up the branch and create the PR.

The irony of me typing out all this which is less then the changes in the PR doesn't miss on me.

I only reported this because I've been attempting to get support updated and documented in chef-provisioning-fog for Xen server. Finding someone to own this other then me would help get this resolved faster, where as i have, and probably never will actually use Xen.

fernandes commented 8 years ago

@jjasghar hahahah never say never :stuck_out_tongue_winking_eye:

ok, no problem... I just suggested because you were on this issue, we can't help much here because this is a change on XenServer, not on fog-xenserver, just point this issue to who is facing this xenserver problem and we can help...

I do really appreciate all your efforts (specially not being a problem that affects you) on trying to solve this problem for everybody! :+1:

jjasghar commented 8 years ago

:+1: :rocket: :cake: :+1:

fernandes commented 8 years ago

:smile:

fabio-santos-movile commented 8 years ago

This bug has any solution?

fernandes commented 8 years ago

@fabio-santos-movile this is not a bug, this is an API change on XenServer 6.5, now affinity is required

fabio-santos-movile commented 8 years ago

@fernandes do you know how is the default set for no affinity? We talk about host affinity for vm/template. Right?

fernandes commented 8 years ago

@fabio-santos-movile yeah, we are talking about the same

According to documentation

host ref affinity [read/write] a host which the VM has some affinity for (or NULL). This is used as a hint to the start call when it decides where to run the VM. Implementations are free to ignore this field.

So you need to set VM affinity field with a host reference

jjasghar commented 8 years ago

@fabio-santos-movile any chance i can convince you to to put this patch in? I have no access to Xen and my downstream chef-provisioning-fog is hurt by this at the moment.

fabio-santos-movile commented 8 years ago

@fernandes why not set the default value to NULL?

I Think, when this set is null, the pool will do a RR and provisioning the vm where has more resource free

plribeiro3000 commented 8 years ago

@fabio-santos-movile Can you confirm it works like this? Last time i checked it actually need an existent Host uuid.

fabio-santos-movile commented 8 years ago

@plribeiro3000 i don't know how. I'm not a ruby developer. Can you show me the code? I tried

:affinity  => "NULL",

But does not work.

plribeiro3000 commented 8 years ago

try :affinity => nil. =)

fabio-santos-movile commented 8 years ago

        vm = connection.servers.new :name => config[:vm_name],
                                    :affinity => nil,                                                                                                                   
                                    :template_name => config[:vm_template]

        vm.save :auto_start => false

Creating VM my-hostname...
Using template CentOS_6_x64 [uuid: 84392427-dbd4-81b0-c6ee-aa4845e1d33c]...
ERROR: ArgumentError: affinity is required for this operation

plribeiro3000 commented 8 years ago

@fabio-santos-movile Yeap. I've crossed this when i rewrote the codebase of fog-xenserver. I guess this is something to be changed in the other library instead. fog-xenserver seems good to me.

jojoger commented 6 years ago

Hello,

Unfortunately The affinity setting in fog-xenserver is buggy even if used as intended. Even if I set a valid UUID one of the Pool's hosts:

connection.servers.create   :name           => 'my_test_kitchen_vm',                 
                        :affinity => UUIDTools::UUID.parse('512d80bb-b227-4b65-b77c-f9cadba9b983'),                                                                        
                      :template_name  => 'ol73'

I get an error from the connection:

/opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/xen_server/connection.rb:35:in `eval': (eval):1: syntax error, unexpected tIDENTIFIER, expecting ')' (SyntaxError)                         
9-3c10-def0aceed035', 512d80bb-b227-4b65-b77c-f9cadba9b983)                          
                              ^           
(eval):1: syntax error, unexpected tIDENTIFIER, expecting end-of-input               
0aceed035', 512d80bb-b227-4b65-b77c-f9cadba9b983)                                    
                              ^           
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/xen_server/connection.rb:35:in `request'                                            
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/compute/xen_server/requests/get_record.rb:10:in `get_record_by_ref'                 
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/compute/xen_server/requests/get_record.rb:6:in `get_record'                         
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/compute/xen_server/models/collection.rb:12:in `get'                                 
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/associations/one_identity.rb:20:in `affinity'                                      
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/attributes.rb:173:in `block in missing_attributes'                                 
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/attributes.rb:172:in `each'                                                        
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/attributes.rb:172:in `missing_attributes'                                          
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/attributes.rb:154:in `requires'                                                    
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/compute/xen_server/models/instance_methods.rb:15:in `require_creation_attributes'   
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-xenserver-0.3.0/lib/fog/compute/xen_server/models/instance_methods.rb:23:in `save'                          
        from /opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/fog-core-1.45.0/lib/fog/core/collection.rb:50:in `create'                                                       
        from test.rb:67:in `<main>'

So my guess is that this code was never really tested? My workaround as I don't need affinity is to uncomment/overwrite require_before_save :name, :affinity in fog-xenserver-0.3.0/lib/fog/compute/xen_server/models/server.rb with require_before_save :name and set in the server create method :affinity to 'NULL:

connection.servers.create   :name           => 'my_test_kitchen_vm',                 
                        :affinity => 'NULL',
                      :template_name  => 'ol73'

Best Regards, Jonathan

plribeiro3000 commented 6 years ago

affinity should be a host reference, not a uuid.

another thing i noted is that when it translated the parameters to call the request, it got the uuid you've passed and passed as is, not as a string. I wonder what UUIDTools::UUID.parse does return.

Can you try passing a host reference instead?

jojoger commented 6 years ago

thank you for your fast reply! With a host reference it works. For everyone else facing those problems, here's how it's done:

host = connection.hosts.all.find { |h| h.name == "<NAMEOFHOST>" }
connection.servers.create   :name           => 'my_test_kitchen_vm',
                        :affinity => host,
                      :template_name  => 'ol73'

plribeiro3000 commented 6 years ago

Awesome. I was writing a step by step of the stack trace to help you understand. Since i'm almost done i will paste it here once i finish. =)

plribeiro3000 commented 6 years ago

Ok, So following the stack trace and the logic that is happening when you call create on the collection:

It does initialize a new object, save it and return it (See here)

def create(attributes = {})
  object = new(attributes)
  object.save
  object
end

The object initialization save the attributes using setter's if existent otherwise saving on the @attributes hash (See here)

def merge_attributes(new_attributes = {})
  new_attributes.each_pair do |key, value|
    next if self.class.ignored_attributes.include?(key)
    if self.class.aliases[key]
      send("#{self.class.aliases[key]}=", value)
    elsif self.respond_to?("#{key}=", true)
      send("#{key}=", value)
    else
      attributes[key] = value
    end
  end
  self
end

affinity is defined as an association so it has a setter and getter defined (See here)

  has_one_identity      :affinity,          :hosts

affinity= is called by merge_attributes. It expects the identity of the association or the actual association (See here)

def create_setter
  model.class_eval <<-EOS, __FILE__, __LINE__
    def #{name}=(new_#{name})
      associations[:#{name}] = new_#{name}.respond_to?(:identity) ? new_#{name}.identity : new_#{name}
    end
  EOS
end

It saves the value you've passed under associations[:affinity]
It tries to save the server (See here)

def save(extra_params = {})
  require_creation_attributes
  attrs = all_associations_and_attributes.reject { |_key, value| value.nil? }
  ref = service.send("create_#{provider_class.downcase}", attrs, extra_params)
  merge_attributes collection.get(ref).attributes
  true
end

It checks for the required attibutes (See here)

def require_creation_attributes
  requires *self.class.instance_variable_get("@require_before_save")
end

It requires a name and affinity (See here)

  require_before_save :name, :affinity

requires check if the attributes are missing (See here)

def requires(*args)
  missing = missing_attributes(args)
  if missing.length == 1
    raise(ArgumentError, "#{missing.first} is required for this operation")
  elsif missing.any?
    raise(ArgumentError, "#{missing[0...-1].join(", ")} and #{missing[-1]} are required for this operation")
  end
end

missing_attributes call the methods and check if some of them are blank (See here)

def missing_attributes(args)
  missing = []
  ([:service] | args).each do |arg|
    missing << arg unless send("#{arg}") || attributes.key?(arg)
  end
  missing
end

affinity is called. (See here)

def create_getter
  model.class_eval <<-EOS, __FILE__, __LINE__
    def #{name}
      return nil if associations[:#{name}].nil
      service.send(self.class.associations[:#{name}]).get(associations[:#{name}])
    end
  EOS
end

It calls get on the collection of the previous defined association (See here)

def get(ref)
  data = service.get_record(ref, model.provider_class)
  new(data)
rescue Fog::XenServer::NotFound, Fog::XenServer::RequestFailed
    nil
end

It calls the request get_record (See here)

def get_record(ref, provider_class, options = {})
  get_record_by_ref(ref, provider_class, options)
end

It calls the request get_record_by_ref (See here)

def get_record_by_ref(ref, provider_class, options = {})
  @connection.request({:parser => Fog::Parsers::XenServer::Base.new, :method => "#{provider_class}.get_record"}, ref).merge(:reference => ref)
end

Conclusion: You do need to pass an Reference or the actual fog-xenserver object as affinity. :wink:

fog / fog-xenserver

"Affinity is required for this operation" on VM creation #57