bb-Ricardo / netbox-sync

Sync objects from VMware or redfish sources to NetBox
MIT License
282 stars 65 forks source link

Error during sync for 'assigned_object_id' on ip-addresses #361

Closed AnuragYewale15 closed 7 months ago

AnuragYewale15 commented 9 months ago

I'm trying to run the netbox sync but I'm facing this error

2023-12-29 05:31:57,147 - ERROR: NetBox returned: POST /api/ipam/ip-addresses/ Bad Request 2023-12-29 05:31:57,147 - ERROR: NetBox returned body: {'assigned_object_id': ['This field cannot be null.']} 2023-12-29 05:31:57,147 - ERROR: Request Failed for IP address.

could you please help me with it. Am I missing out on something

AnuragYewale15 commented 9 months ago

Hi @bb-Ricardo , Could you please help me with this?

bb-Ricardo commented 9 months ago

Hi @AnuragYewale15,

Did you get any other error messages during your run? Can you post them here please?

Thank you

AnuragYewale15 commented 9 months ago

Hi @bb-Ricardo , This is the first error which I encounter which is repeated multiple times after that. Before that I got this in DEBUG

DEBUG: Unable to find a PTR record for ...: Domain name not found

and after few minute I got this

INFO: Creating new NetBox 'IP address' object '.../' with data: {'tags': [{'name': 'NetBox-synced'}, {'name': 'Source: vcenter-dallas'}], 'address': '.../', 'assigned_object_type': 'virtualization.vminterface', 'dns_name': '*****'} 2024-01-04 07:25:36,991 - ERROR: NetBox returned: POST /api/virtualization/interfaces/ Bad Request 2024-01-04 07:25:36,991 - ERROR: NetBox returned body: {'virtual_machine': ['This field is required.']} 2024-01-04 07:25:36,991 - ERROR: Request Failed for virtual machine interface.

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo , Any update on this?

bb-Ricardo commented 8 months ago

Sorry, forgot to answer you. I would need the full log output with -l DEBUG2 in order to be able to debug it. Can you please send it over?

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo ,

2024-01-16 09:32:37,544 - DEBUG2: Received HTTP Status 400. 2024-01-16 09:32:37,544 - ERROR: NetBox returned: POST /api/ipam/ip-addresses/ Bad Request 2024-01-16 09:32:37,544 - ERROR: NetBox returned body: {'assigned_object_id': ['This field cannot be null.']} 2024-01-16 09:32:37,544 - ERROR: Request Failed for IP address. Used data: {'tags': [{'name': 'NetBox-synced'}, {'name': 'Source: vcenter-dallas'}], 'address': '.../*', 'assigned_object_type': 'virtualization.vminterface', 'dns_name': '****'}

Due to confidentiality reason I can't share the full logs .Is there any other way to mask the logs and then provide it to you?

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo any update on this?

bb-Ricardo commented 8 months ago

Hi @AnuragYewale15,

Nope. It's very difficult to work with only a few lines of log data in order to figure put the issue.

But I might have an idea where the issue could stem from.

Question: is the IP which causes this issue assigned to more then one interface?

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo ,

No in our environment we have one IP dedicate to one interface.

We have approximately 1200 VMs and even after the sync we see only 84 VMs.

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo , Any update??

bb-Ricardo commented 8 months ago

Hi,

Currently time is very tight with family and everything. This is a hobby project and I get to it once I have some spare time. Will let you know when a commit has been pushed.

bb-Ricardo commented 8 months ago

Hey @AnuragYewale15

I just pushed a new commit to the development branch. Can you test it out and see if the error still occures?

AnuragYewale15 commented 8 months ago

Sure , I will check

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo , I checked and I'm still facing the same error

bb-Ricardo commented 8 months ago

Sorry, then I'm not able to help. Without logs this is impossible to debug.

bb-Ricardo commented 8 months ago

I finally have seen this issue. Seems a dependency issue while adding IP addresses

bb-Ricardo commented 8 months ago

Hey @AnuragYewale15,

I pushed another commit to the development branch. Can you please have a look?

If it works for you then you can go ahead and close this issue. Thank you.

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo , Now the IP address issue has been fixed I'm now able see the IPs in the UI, but now exactly same issue I'm facing with the virtual machine

INFO: Creating new NetBox 'virtual machine' object '' with data: {'tags': [{'name': 'NetBox-synced'}, {'name': 'Source: dallas'}], 'name': '', 'cluster': 3, 'status': 'active', 'memory': 4096, 'vcpus': 2, 'site': 1, 'device': 1, 'platform': 2} 2024-02-02 07:27:49,885 - ERROR: NetBox returned: POST /api/virtualization/virtual-machines/ Bad Request 2024-02-02 07:27:49,885 - ERROR: NetBox returned body: {'device': ['The selected device (.com (IBM SoftLayer)) is not assigned to this cluster (Non-Production).']} 2024-02-02 07:27:49,885 - ERROR: Request Failed for virtual machine. Used data: {'tags': [{'name': 'NetBox-synced'}, {'name': 'Source: dallas'}], 'name': '', 'cluster': 3, 'status': 'active', 'memory': 4096, 'vcpus': 2, 'site': 1, 'device': 1, 'platform': 2}

bb-Ricardo commented 8 months ago

Hi,

It seams you are having a couple of issues but only posting them one by one and this is quite exhausting to support.

Is this the only issue which is currently showing up as ERROR?

It seems you need to adapt your cluster site relation or host site relation and fix the expression so the host and the VM are tracked being on the same site.

Try switching off track_vm_host and see if the sync works then.

AnuragYewale15 commented 8 months ago

Hi @bb-Ricardo , The Error which I mentioned is redundant for all VMs, VLANs, etc. Your recent commit solve the issue related to IP that is (ERROR: NetBox returned: POST /api/ipam/ip-addresses/ Bad Request) this one and now I'm facing the same for virtual machines that is (ERROR: NetBox returned: POST /api/virtualization/virtual-machines/ Bad Request) this

So it would be very help if you can correlate these errors and see if the changes you made to resolve IP address issue can be applied for VMs and VLans also.

Till then I will work with what you have suggested and will let you know if I'm facing any other issue

Thank You!

bb-Ricardo commented 8 months ago

Sorr, but I can't further assist without any substantial log files.

AnuragYewale15 commented 7 months ago

Hi @bb-Ricardo , I tried switching off track_vm_host and it work for 1 datacenter with few VMs still missing and for other datacenters I'm getting no site relation error with is further leading to missing of VMs and other data from the remaining datacenter . Are there any other changes in settings.ini which I'm missing ?

Thanks

AnuragYewale15 commented 7 months ago

Also @bb-Ricardo what is the format to add cluster_site_relation variable. For example my settings.ini consist [source/vcenter-PQR] enabled = True type = vmware host_fqdn = ** validate_tls_certs = False username = ** password =** permitted_subnets = 0.0.0.0/0 collect_hardware_asset_tag = True skip_vm_templates = True skip_vm_comments = False track_vm_host = False sync_vm_interface_mtu = True dns_name_lookup = True custom_dns_servers = ..,, , .,,, set_primary_ip = always ip_tenant_inheritance_order = device, prefix disable_vlan_sync = False cluster_site_relation = vcenter-PQR.* = PQR [source/vcenter-ABC] enabled = True type = vmware host_fqdn =** validate_tls_certs = False username = ** password =** permitted_subnets = 0.0.0.0/0 collect_hardware_asset_tag = True skip_vm_templates = True skip_vm_comments = False track_vm_host = False sync_vm_interface_mtu = True dns_name_lookup = True custom_dns_servers = .,,, .,,, set_primary_ip = always ip_tenant_inheritance_order = device, prefix disable_vlan_sync = False cluster_site_relation = vcenter-ABC.* = ABC

AnuragYewale15 commented 7 months ago

Hi @bb-Ricardo

Any update you can help me with?

bb-Ricardo commented 7 months ago

closing this issue as the original issue has been solved.