How to increase the show versions function performance?

[KevinTang8888]

Hi I have a large amount of devices 500+ config backup by Oxidzed. If I check the device config versions difference. it took about 30-40 minutes to display them on the screen.

I went through the Git garbage collection process. But it does not improve the performance much.

https://github.com/ytti/oxidized/blob/master/docs/Outputs.md#output-git

When you use git to store your configurations, the size of your repository will grow over time. This growth can lead to performance issues. To resolve these issues, you should perform a Git garbage collection on your repository.

I check the oxidized memory and CPU usage, still plenty free.

Any other config/setting that can improve versions difference performance?

Thanks.

[KevinTang8888]

just did the gc process again. But still very slow to display the versions difference. :-(

$ git gc Enumerating objects: 1, done. Counting objects: 100% (1/1), done. Writing objects: 100% (1/1), done. Total 1 (delta 0), reused 1 (delta 0), pack-reused 0 Removing duplicate objects: 100% (256/256), done. Checking connectivity: 252720, done. $

[yoakim]

Does your entire oxidized instance freeze when trying to show the versions? In that case i might have the same problem.

[KevinTang8888]

yes, if you click any other devices, then no respond.

[KevinTang8888]

if I check the memory and CPU, plenty resource available.

[yoakim]

Yea, i copied the repo to my local machine and git log -- <hostname> worked flawlessly, while taking very long time on the actual oxidized server. I then mounted the remote storage where the production oxidized git is located and it went very slow again. This means that the storage is the culprit for us in this specific case.

[KevinTang8888]

thanks, that confirmed the oxidized server function problem, not the server hardware resource (memory/cpu) issue.

[robertcheramy]

just did the gc process again. But still very slow to display the versions difference. :-(

$ git gc Enumerating objects: 1, done. Counting objects: 100% (1/1), done.

Are you sure you are running git gc on the right place? If you have only one object, this can't be the repository of 500+ devices.

Here is a git gc on a relative small oxidized instance:

oxidized.git$ git gc
Enumerating objects: 340, done.
Counting objects: 100% (340/340), done.
Delta compression using up to 8 threads
Compressing objects: 100% (329/329), done.
Writing objects: 100% (340/340), done.
Building bitmaps: 100% (85/85), done.
Total 340 (delta 207), reused 0 (delta 0), pack-reused 0

[yoakim]

thanks, that confirmed the oxidized server function problem, not the server hardware resource (memory/cpu) issue.

Our case is 100% outside oxidized, our remote storage is an Azure blob storage, which probably isn't ideal for this kind of data. We are looking into migrating the repo to a "normal" file system.

[KevinTang8888]

the issue has been fixed now after I rebuilt Oxidized