search

YunoHost / issues

General issue tracker for the YunoHost project
72 stars 8 forks source link

yunohost log list is super slow #2394

Closed selfhoster1312 closed 2 months ago

selfhoster1312 commented 4 months ago

After some discussion in the chatroom, we realized yunohost log list is super slow because it needs to read every YML file for every log file ever.

The log files are structured like this in /var/log/yunohost/operations/:

In most cases, we don't need to read the metadata unless with_details or with_suboperations parameters are passed. However, when the log is related to a parent operation, this is registered in the YAML metadata, so it requires parsing the entire metadata just to know whether to display the operation log or not.

We could maybe store the parenthood relationship as a symlink like DATE-ID-OPERATION.parent where the absence of this symlink would mean there is no parent. This way we only need a stat operation in the best case, instead of a full read/parse.

I could write this code, but i have no idea how to write a yunohost migration. Help welcome :)

selfhoster1312 commented 4 months ago

Tested on a raspberry pi with 1 single log entry:

root@raspberrypi:~# time yunohost log list
operation:
  0:
    description: Postinstall your YunoHost server
    name: 20231105-135436-tools_postinstall-raspberrypi.local
    path: /var/log/yunohost/categories/operation/20231105-135436-tools_postinstall-raspberrypi.local.yml
    started_at: 2023-11-05 15:54:36
  1:
    description: Regenerate system configurations 'list-pending'
    name: 20240516-155112-regen_conf-list-pending
    path: /var/log/yunohost/categories/operation/20240516-155112-regen_conf-list-pending.yml
    started_at: 2024-05-16 17:51:12

real    0m8.860s
user    0m7.978s
sys     0m0.541s

Yes that's several seconds for a single log entry.

alexAubin commented 2 months ago

Fixed via https://github.com/YunoHost/yunohost/pull/1907