tumblr / collins

groovy kind of love
tumblr.github.com/collins
Apache License 2.0
571 stars 99 forks source link

Question: Problem with import lshw #450

Closed onimsha closed 8 years ago

onimsha commented 8 years ago

Scenario:

  1. Install a VM, configure IP Address for all NICs
  2. Deploy Collins, use ipAddresses.strict=false. Not define any POOLs
  3. Create asset "tt-0001"
  4. Import LSHW XML for "tt-0001"
curl --basic -u blake:admin:first --data-urlencode 'CHASSIS_TAG=tt-0001' --data-urlencode 'lldp=null' --data-urlencode "lshw@/Users/tuan.co/tt-0001-lshw.xml" --data-urlencode 'attribute=NODECLASS;dns' http://192.168.100.100:9080/api/asset/tt-0001

Following error happened:

{"status":"error","data":{"message":"Error updating asset: Could not find capacity for network interface for Virtio network device"}}%

Is it true that we can only import LSHW of a machine without IP Information?

byxorna commented 8 years ago

No. That error means your lshw report does not have any capacity for the network interface stated above. Collins needs to know how large your nics are. Ive seen this happen because lshw doesnt support the type of interface you are trying to report (strange driver perhaps), or issues reading out of sysfs for the capacity attribute (-1 sometimes). Ive solved this by postprocessing the lshw xml with nokogiri and injecting/fixing the capacity attribute where appropriate.

onimsha commented 8 years ago

Indeed. Currently i'm using OpenStack with VirtIO network card, that's why lshw is unable to get the capacity information. Thank you for explaination.

onimsha commented 8 years ago

EDIT: I know why already, Collins requires to have valid LLDP XML file for update the asset. We can ignore this question.

@byxorna a after added the capacity for network interfaces, now i have another error

curl --basic -u blake:admin:first --data-urlencode 'CHASSIS_TAG=tt-0001' --data-urlencode 'lldp=null' --data-urlencode "lshw@/Users/tuan.co/tt-0001-lshw.xml" --data-urlencode 'attribute=NODECLASS;dns' http://192.168.100.100:9080/api/asset/tt-0001
{"status":"error","data":{"message":"Error updating asset: Content is not allowed in prolog."}}%

Do you have any idea what happened? I did not find any Content keyword in the lshw xml.

thanks

onimsha commented 8 years ago

One more question is: Is LLDP XML mandatory for updating a server node? from the curl, without the lldp parameter, collins does not accept the asset update operation.

    message Error updating asset: lldp parameter not specified
roymarantz commented 8 years ago

It depends on the state of the asset. In

collins https://opengrok.ewr01.tumblr.net/opengrok/xref/collins//app https://opengrok.ewr01.tumblr.net/opengrok/xref/collins/app//collins https://opengrok.ewr01.tumblr.net/opengrok/xref/collins/app/collins// models https://opengrok.ewr01.tumblr.net/opengrok/xref/collins/app/collins/models/ /AssetLifecycle.scala https://opengrok.ewr01.tumblr.net/opengrok/xref/collins/app/collins/models/AssetLifecycle.scala line 210 is a routine that makes sure an Incomplete state server gets "lshw", "lldp", "CHASSIS_TAG" tags set on update which I believe is the source of your message.

Roy

On Fri, Jun 17, 2016 at 6:08 AM, Alex Co notifications@github.com wrote:

One more question is: Is LLDP XML mandatory for updating a server node? from the curl, without the lldp parameter, collins does not accept the asset update operation.

message Error updating asset: lldp parameter not specified

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/tumblr/collins/issues/450#issuecomment-226732280, or mute the thread https://github.com/notifications/unsubscribe/AF2LSp3edsSBDs6qn2R6BcjDiaqcgbDYks5qMnIrgaJpZM4I3LGu .

onimsha commented 8 years ago

@roymarantz got it. Since we don't have LLDP data, i think we can just simple move server to other status than "Incomplete".

Another thing is the "Managed" tag. As far as i can see, those "Managed" tags can only being imported by using LSHW and LLDP XML. Any attempt to update individual Managed tags will failed, with the message ( below example is for DISK_TYPE )

exception  Error processing request: Error updating asset: Attribute DISK_TYPE is restricted

Do we have to enforce this? I mean, is there any possibility that we have a feature that can allow updating those "Managed" tags? Like "updateManagedtags = true"

byxorna commented 8 years ago

@onimsha another thing you can do is submit a blank LLDP report, or synthesize one. Same for LSHW; you can totally postprocess an LSHW blob to inject whatever information you collect. We do that when lshw fails to discover certain attributes we care about (we populate the xml from /sys to fill in the gaps)

To move an asset from Incomplete->New automatically, yes you need to submit an LLDP report and LSHW report.

william-richard commented 8 years ago

@onimsha to answer your question about managed tags - they are usually protected/derrived attributes, so there isn't any way to modify them directly. In some edge cases, they don't quite act like real attributes - they're sort of special.

For attributes like DISK_TYPE I think you may need to just upload a fresh, synthesized LSHW report as @byxorna suggested. Collins has a very tight coupling on LLDP and LSHW, which is really nice when you are using them, but makes things a bit difficult when you're not.

onimsha commented 8 years ago

Thanks all. I have better visibility about collins now.