Library Stats API Changes

[pchang388]

First, thank you for Tdarr! It's great and I use it myself.

I am unsure where to create this since it is not technically a bug but hope it fits here.

Describe the issue I help maintain a small project, tdarr-exporter, that gathers stats from Tdarr and makes them available as Prometheus metrics. I noticed around this version: 2.24.01 [11th August 2024] that the API behavior has changed. Below are my findings/assumptions so may not be an accurate way to use the API.

I used to be able to get per library and general transcode, video, audio stats from the /api/v2/cruddb endpoint using payload:

{
    "data": {
        "collection": "StatisticsJSONDB",
        "mode": "getById",
        "docID": "statistics",
        "obj": {}
    }
}

Within the response there is a pies section that used to have all the per library general stats and was fairly quick in responding:

{
    ...
    ...
    "pies": [
        [
            "lib_name",
            "lib_id",
            2313,
        ]
    ]
],

Now the response is always an empty pie section:

{
    ...
    ...
    "pies": [],
    ...
}

Based on my poking around the UI calls this endpoint for Library stats: /api/v2/stats/get-pies. This endpoint is not in the API documentation, I was wondering if someone could tell me:

1. This new endpoint: /api/v2/stats/get-pies takes ~1-2 seconds for (my internal) Tdarr instance to respond and with a high amount of libraries, it may high take some time for a metric scrape request to complete. I don't recall this behavior in the old API, is this normal and I should look into some limited concurrency to speeds things up or perhaps indicates an issue?
2. Is there perhaps a different API or param I can use to get the data more quickly/efficiently? Or perhaps request per library stats breakdown in a single API call. Right now I am using:

   
   ## to get library ids and names
   # /api/v2/cruddb
   {
   "data": {
       "collection": "LibrarySettingsJSONDB",
       "mode": "getAll",
       "docID": "",
       "obj": {}
   }
   }

to get per library stats ( 1 per library - maybe a way to batch?)

/api/v2/stats/get-pies

{ "data": { "libraryId": "{lib_id}" } }

3. I'm assuming the stats API will continue to change, is there any chance to update the API docs for those as well?

Thank you!

[HaveAGitGat]

Hi yes that's right, it's from the improvement for large libraries in 2.24.01. The previous situation had some issues:

• The previous pie stats were automatically re-calculated and updated every 10secs to 5mins every time files changed. For large libraries this put a lot of extra load on the server. For most people they only need the pie stats when they are looking at the Tdarr UI so it was unnecessary in that case to keep recalculating the stats in the background.

• The previous pie stats structure was also quite confusing with arrays within arrays

There is no way to speed it up as that's how long it takes if you have quite a few files. Previously it took even longer but constantly running in the background when files changed, but now it's a bit quicker and on-demand. It doesn't do all libraries at the same time to speed things up if the user is only looking at a single library.

I suppose what you could do is check if the number of total files or transcodes has increased. If they haven't then no need to get new stats, just use the old ones. If they have changed then can get the new stats. I could add similar logic internally for the next version so that could solve the issue. And yes the next update is a large update to the API and documentation.

[pchang388]

Thanks for the quick reply and getting back to me with details and the suggestion. That makes sense and I understand why the changes were made especially for the format inside the arrays.

I will take a look and implement something to work with the new API!