search

OCSInventory-NG / OCSInventory-ocsreports

Webconsole for OCS Inventory NG
https://www.ocsinventory-ng.org
GNU General Public License v2.0
230 stars 150 forks source link

[BUG] AUTO_DUPLICATE_LVL not working for some few computers #1078

Closed pchamo closed 2 years ago

pchamo commented 3 years ago

OCS Inventory version Version : 2.8

Describe the bug After updating to the last ocs server some duplicates started to appear. Even sometimes I get some computers tripled. We are getting some few duplicated daily, so is extra work to process those duplicates manually. I have in my ocs server:

-r-xr-xr-x 1 root root 11184 Sep 11 03:48 /usr/local/share/perl5/Apache/Ocsinventory/Server/Duplicate.pm I see that the file Duplicate.pm included in ocs server 2.8 is the same one included in previous versions except for these lines:

  # workaround agent old_deviceid bug
  if(!$result->{CONTENT}->{OLD_DEVICEID} and $result->{CONTENT}->{DOWNLOAD}->{HISTORY}->{OLD_DEVICEID})
  {
    $result->{CONTENT}->{OLD_DEVICEID} = $result->{CONTENT}->{DOWNLOAD}->{HISTORY}->{OLD_DEVICEID};
  }

Could you please explain to me what those lines do exactly?

Screenshots

My setup: http://prntscr.com/vogkx2 A sample o f my duplicates: http://prntscr.com/vogk56

Log file (optional) Mon Nov 23 09:16:50 2020;11845;320;MANIZALES-119-2020-11-23-03-07-31;192.168.xxx.235;OCS-NG_WINDOWS_AGENT_v2.8.0.0;session;end Mon Nov 23 09:16:50 2020;11845;517;MANIZALES-119-2020-11-23-03-07-31;192.168.xxx.235;OCS-NG_WINDOWS_AGENT_v2.8.0.0;duplicate;replacing_error Mon Nov 23 09:16:51 2020;11845;101;MANIZALES-119-2020-11-23-03-07-31;192.168.xxx.235;OCS-NG_WINDOWS_AGENT_v2.8.0.0;inventory;transmitted

pchamo commented 3 years ago

The same computer registered again and again: http://prntscr.com/vuoql4

Lea9250 commented 3 years ago

Hi @pchamo, Is this issue still relevant ? I'll advise you to update to the latest server version if possible.

Regards, Léa

pchamo commented 3 years ago

Hi @Lea9250 I updated to 2.9.1 last night and the issue is still present. And what's worse I don't see the 'Merge all duplicates' anymore :(

pchamo commented 3 years ago

But I see that the update did do something. I had an extrange duplicate of a computer that appeared alone. That one is not present anymore.

Lea9250 commented 3 years ago

Hi @pchamo, The merge all button has been removed with 2.9.1 version, due to some changes in the duplicate feature.

I'll advise to check if the device IDs for these computers are different (SELECT DEVICEID, NAME FROM hardware WHERE NAME LIKE 'your_duplicate_computer') This would explain why the auto merge didn't apply. How is your AUTO_DUPLICATE_LVL set in OCS configuration ?

Thanks, Léa

pchamo commented 3 years ago

Hi @Lea9250, My AUTO_DUPLICATE_LVL setup is by hostname + macaddress. At this moment I have no duplicates. I'm updating the agent for the duplicates and I will monitor and check. I let you know. Thanks.

pchamo commented 3 years ago

I have a duplicate for sgeologico-772, the query returns:

DEVICEID NAME SGEOLOGICO-772-2021-08-09-08-24-28 SGEOLOGICO-772 SGEOLOGICO-772-2021-08-09-11-20-42 SGEOLOGICO-772

Same model, ipaddress, macaddress, name, os and osversion. ¿Why are deviceid different? Thanks.

pchamo commented 3 years ago

Manizales-129 has 3 network adapters UP, one Ethernet and two Hyper-V, and appears three times under MAC address + Serial number duplicates: https://prnt.sc/1lz1c75

BG-GEO-006316 has 1 ethernet ipv4 adapter UP, and 1 virtual adapter DOWN, and appears twice under MAC address + Serial number duplicates, similar to manizales-129. What surprise me is one of the duplicates corresponds to the virtual adapter that is DOWN: https://prnt.sc/1lz9oh5

Manizales-130 appears three times and has 2 adapters, one UP and one DOWN, similar to bg-geo-006316: https://prnt.sc/1lzb90x.

Well, it seems those duplicates are gone. Good job. Thanks.

Lea9250 commented 3 years ago

Hi again @pchamo, Just to confirm, if I understood well this is now resolved ? If so, I'll close this issue.

Regards, Léa

pchamo commented 3 years ago

Hi @Lea9250,

It seems to me that the fix for duplicates in 2.9.1 is working somehow. Some duplicates are not appearing anymore (for instance manizales-129, or bg-geo-006316), but manizales-130 is still appearing. And today I have other few duplicates (5 more). I think I need to do things like to merge or to delete the affected computers and to see how it goes. I think I need some days to see how it goes. I let you know next week.

Thanks.

pchamo commented 3 years ago

Hi @Lea9250,

I'm registering 30 to 40 duplicates a day. Look at this for instance: https://prnt.sc/1qc8hfb. And this is the network info for those 2 duplicates: https://prnt.sc/1qc8rfe and https://prnt.sc/1qc8tu5

Lea9250 commented 3 years ago

Hi @pchamo, I'm guessing these ones do have different deviceids too. Can I ask how has the OCS agent been installed on these machines ? Sounds more like an agent configuration/installation issue to me, as the auto_duplicate_lvl and duplicate feature seems to be working.

pchamo commented 3 years ago

Hi @Lea9250,

The agents were installed with a cmd script using psexec: psexec \computer -s \domain-controller\msi\instalar-agente-ocs.cmd

and the cmd script used is: \domain-controller\msi\ocsagent-28.exe /nosplash /s /server=http://ocs-server.mydomain/ocsinventory /now /debug

I'm noticing some duplicates appear after rebooting the users computers. About manizales-99 (one of the ones with many duplicates), I deleted the program data ocsinventory folder and reinstalled the agent and is not appearing as duplicate so far.

I think before ocs server 2.8 all was working right, I didn't have duplicates. My duplicates come from ocs 2.8. My agents are 2.4. I updated some of them to 2.8 but I see no difference.

In summary, some duplicates don't appear again, but I'm having some new duplicates each day. And yes, the duplicates have different deviceids.

Thank you.

pchamo commented 3 years ago

Hi @Lea9250,

Here the /var/log/ocsinventory-server/activity.log file for the last two duplicates registered today:

Tue Aug 31 14:41:22 2021;13470;326;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;MANIZALES-46-2021-02-26-19-59-56
Tue Aug 31 14:41:22 2021;13470;513;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;old deviceid
Tue Aug 31 14:41:22 2021;13470;517;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;replacing_error
Tue Aug 31 15:00:11 2021;28108;517;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;replacing_error
Tue Aug 31 16:01:30 2021;13621;517;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;replacing_error
Tue Aug 31 16:06:44 2021;5533;517;MANIZALES-46-2021-08-31-14-40-24;192.168.1.62;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;replacing_error
Tue Aug 31 19:53:10 2021;5533;326;MANIZALES-113-2021-08-31-20-03-21;192.168.1.138;OCS-NG_WINDOWS_AGENT_v2.8.0.0;duplicate;MANIZALES-113-2020-11-17-13-48-37
Tue Aug 31 19:53:10 2021;5533;513;MANIZALES-113-2021-08-31-20-03-21;192.168.1.138;OCS-NG_WINDOWS_AGENT_v2.8.0.0;duplicate;old deviceid
Tue Aug 31 19:53:10 2021;5533;517;MANIZALES-113-2021-08-31-20-03-21;192.168.1.138;OCS-NG_WINDOWS_AGENT_v2.8.0.0;duplicate;replacing_error
Tue Aug 31 20:35:11 2021;30013;517;MANIZALES-113-2021-08-31-20-03-21;192.168.1.138;OCS-NG_WINDOWS_AGENT_v2.8.0.0;duplicate;replacing_error

What does it mean replacing_error? the log file is not verbose enough :(

Lea9250 commented 2 years ago

Hi @pchamo,

Would it be possible for you to share the OCS agent logs of at least one of the duplicated computers ? There might be some useful informations as to why the deviceids keep changing. Make sure to set the debug to 1 in your ocsinventory.ini file to get the most out of the logs.

Regards, Léa

pchamo commented 2 years ago

Hi @Lea9250, I have PROLOG_FREQ = 1 hour and automerge for Hostname + Serial number but I don't have any at this moment because I just deleted those duplicates. It will take some hours to have some of those duplicates generated. I mean, not every hour again and again but at certain moments, I haven't realized exactly when that ocurrs.

Lea9250 commented 2 years ago

Hi @pchamo,

Which version of mysql are you using on your server ?

Thanks, Léa

charleneauger commented 2 years ago

Hi @pchamo ,

Can you apply this PR on your server ? https://github.com/OCSInventory-NG/OCSInventory-Server/pull/358 It add more logs on duplicates merging. After applied, reload apache.

Regards, Charlene

pchamo commented 2 years ago

Computer log after applying the PR:

OCSInventory.log

pchamo commented 2 years ago

Hi @charleneauger, here the activity.log after applying the PR:

Sun Nov  7 02:03:09 2021;27387;100;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;prolog;accepted
Sun Nov  7 02:03:09 2021;27387;311;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;session;started
Sun Nov  7 02:04:46 2021;27387;318;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;session;missing
Sun Nov  7 02:04:46 2021;27387;114;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;no_session
Sun Nov  7 02:04:46 2021;27387;104;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;incoming
Sun Nov  7 02:04:46 2021;27387;113;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;u:securitycenter
Sun Nov  7 02:04:46 2021;27387;113;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;u:checkfiles
Sun Nov  7 02:04:46 2021;27387;113;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;u:officepack
Sun Nov  7 02:04:46 2021;27387;113;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;u:winusers
Sun Nov  7 02:04:46 2021;27387;113;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;u:sharedfolders
Sun Nov  7 02:04:47 2021;27387;320;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;session;end
Sun Nov  7 02:04:47 2021;27387;301;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;error to delete hardware_id from bitlockerstatus
Sun Nov  7 02:04:47 2021;27387;517;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;replacing_error
Sun Nov  7 02:04:47 2021;27387;301;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;error to delete hardware_id from bitlockerstatus
Sun Nov  7 02:04:47 2021;27387;517;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;replacing_error
Sun Nov  7 02:04:47 2021;27387;301;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;error to delete hardware_id from bitlockerstatus
Sun Nov  7 02:04:47 2021;27387;517;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;duplicate;replacing_error
Sun Nov  7 02:04:47 2021;27387;101;PC-029-2021-11-05-23-07-18;ip-address;OCS-AGENT_v2.4;inventory;transmitted
pchamo commented 2 years ago

Duplicate image: duplicate

pchamo commented 2 years ago

Hi @Lea9250, mysql server version: mysql-community-server-5.7.35-1.el7.x86_64. Thanks.

pchamo commented 2 years ago

I realized that I had the bitlocker plugin installed incorrectly. I just fixed that. I'm going to see if that fixes my duplicates issue.

charleneauger commented 2 years ago

Hi @pchamo ,

Good news ! I await your reply :-)

Regards, Charlene

pchamo commented 2 years ago

There are no duplicates anymore. Good job! Thank you very much. How important is to have logs with more debug info. My issue was due bitlocker. I installed it (and didn't remember about it) but the plugin didn't appear in the plugins list and I never copied the bitlockerstatus.vbs file in the windows computers. I would like to reproduce the issue. Anyhow I would say that it's a bug, wouldn't you?

pchamo commented 2 years ago

I tried the firewallrules plugin and same result. I was testing the plugin and then I decided to disable it from the ocs web console, leaving the firewallrules plugin files in the disk in /usr/share/ocsinventory-reports/ocsreports/extensions/ and now I have some few duplicates. and these error messages:

Mon Nov 15 22:50:40 2021;10024;301;COMPUTER-914-2021-11-15-12-41-35;172.20.2.24;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;error to delete hardware_id from firewallrules

Mon Nov 15 22:50:40 2021;10024;517;COMPUTER-914-2021-11-15-12-41-35;172.20.2.24;OCS-NG_WINDOWS_AGENT_v2.4.0.0;duplicate;replacing_error

So, I have to uninstall the plugin completely, thanks.

charleneauger commented 2 years ago

Hi ! Good news, I close this issue now :)

Best regards, Charlene

pchamo commented 2 years ago

It's good to know how to fix the issue, but, the bug is there.