page-spy-api 怎么配置回放日志存储时长

qiushaocloud commented 4 months ago

Description

建议提供一份 page-spy-api 支持的配置说明

Alternative

No response

Additional context

No response

Validations

[X] Read the FAQ.
[X] Read the Contributing Guidelines.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] The provided reproduction is a minimal reproducible example of the bug.

qiushaocloud commented 4 months ago

type Config struct { Port string json:"port" Debug bool json:"debug" NotAllowedDeleteLog bool json:"notAllowedDeleteLog" RpcAddress []Address json:"rpcAddress" CorsConfig CorsConfig json:"corsConfig" StorageConfig *StorageConfig json:"storageConfig" MaxRoomNumber int json:"maxRoomNumber" // max log file size, unit is mb MaxLogFileSizeOfMB int64 json:"maxLogFileSizeOfMB" // max log file size, unit is day MaxLogLifeTimeOfHour int64 json:"maxLogLifeTimeOfHour" }

我看了代码，应该是配置 maxLogFileSizeOfMB 和 maxLogLifeTimeOfHour，但是配置了后不好使，回放并没有被删除

wangdashuaihenshuai commented 4 months ago

type Config struct { Port string json:"port" Debug bool json:"debug" NotAllowedDeleteLog bool json:"notAllowedDeleteLog" RpcAddress []Address json:"rpcAddress" CorsConfig CorsConfig json:"corsConfig" StorageConfig *StorageConfig json:"storageConfig" MaxRoomNumber int json:"maxRoomNumber" // max log file size, unit is mb MaxLogFileSizeOfMB int64 json:"maxLogFileSizeOfMB" // max log file size, unit is day MaxLogLifeTimeOfHour int64 json:"maxLogLifeTimeOfHour" }

我看了代码，应该是配置 maxLogFileSizeOfMB 和 maxLogLifeTimeOfHour，但是配置了后不好使，回放并没有被删除

配置以后，会有一个一个小时运行一次的定时任务来删除，可以等等定时任务运行完才会删除。

wangdashuaihenshuai commented 4 months ago

最近文档在重新整理成一个单独文档页面。

qiushaocloud commented 4 months ago

文件并没有清理完全，只清理了一点文件，按理说到根据我的配置，清理时应该只保留最近1小时的1MB的文件，但是只清理了一点点


pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file by size 2509mb > max size 1mb" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.f092e9dd183b969fec60979ad350bdbf name 2024_7_11_12:41:05.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.70265cf79a05f550354481e4f3724afa name 2024_7_11_12:41:15.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.4905cbae0d3ad1641cd087eb23a8bc3c name 2024_7_11_12:41:25.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.c0d8aefc2d1741e5c067d1f5d7b4f7ac name 2024_7_11_12:41:35.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.db3321fdc5c6e3d159e296841d43f4a6 name 2024_7_11_12:41:45.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.7341c904e53edced4422243b157f8e42 name 2024_7_11_12:41:55.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.582540404b87443d079cc6bb97661fa6 name 2024_7_11_12:42:05.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.8cb4fa3dbdb49f543b0f047dca8a1291 name 2024_7_11_12:42:15.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.cc4cd24a93cc2f8617f74907b5a24d33 name 2024_7_11_12:42:25.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.64623d9e323b0b970a5b51096e33f1a9 name 2024_7_11_12:42:35.json by size" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file by time 10 file timeout before 2024-07-12 14:30:40.479109 +0800 CST m=+0.055112370" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.94bb769396c6dd3d976e072f42de98c6 name 2024_7_12_13:56:40.json by time createdAt 2024-07-12 13:56:40.323802 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.031b834ec1158ee2cc9ff98e22e46ccc name 2024_7_12_13:55:26.json by time createdAt 2024-07-12 13:55:26.368008 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.e7a9e7b50f1dd4e09a1516b973839249 name 2024_7_12_12:21:29.json by time createdAt 2024-07-12 12:21:31.43116 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.e1d29afc9bcab66a1dfe56b32d9619f8 name 2024_7_12_12:20:43.json by time createdAt 2024-07-12 12:20:43.642104 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.97c486d404289a06a6ac437b7dde468e name 2024_7_12_12:20:29.json by time createdAt 2024-07-12 12:20:31.082869 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.40365ed1d24e7fa9b0b5370a47414ebb name 2024_7_12_12:19:43.json by time createdAt 2024-07-12 12:19:43.65481 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.e57ff3577775ab427ebbffbff7d98109 name 2024_7_12_12:19:28.json by time createdAt 2024-07-12 12:19:30.161116 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.aaaeec694f943a3494819d9f13ce3f46 name 2024_7_12_12:18:43.json by time createdAt 2024-07-12 12:18:43.647199 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.7c8f86cf76e92e4b413038d03665530c name 2024_7_12_12:18:27.json by time createdAt 2024-07-12 12:18:29.365282 +0800 +0800" module=core

pageSpyApp stderr: time="2024-07-12T15:30:40+08:00" level=info msg="clean file local.536ea993b7ae37264907f7eb858aa1c5 name 2024_7_12_12:17:43.json by time createdAt 2024-07-12 12:17:43.653651 +0800 +0800" module=core

total 3705616
-rw-r--r--  1 qiushao  staff   1.0M  7 11 13:12 local.000646a643c6fd69b554c934a7e00dde
-rw-r--r--  1 qiushao  staff    71K  7 11 19:26 local.0012f689f8edf8495ead0d18b6c0162c
-rw-r--r--  1 qiushao  staff    46M  7 12 12:10 local.0023acdbf7a7401c683788fa6a39138d
-rw-r--r--  1 qiushao  staff   1.6M  7 11 13:35 local.003fbe95da8186f72500af38b4b79f44
-rw-r--r--  1 qiushao  staff   8.9M  7 11 14:03 local.006e9e28544e57d545cc0edc0310f2ed
-rw-r--r--  1 qiushao  staff   622K  7 12 12:09 local.007dc82800e0478101dbb7c86ecc67b1
-rw-r--r--  1 qiushao  staff   3.2M  7 11 13:50 local.00ef7e4fbfaa8fd221233a7b08b53757
-rw-r--r--  1 qiushao  staff    27K  7 12 10:16 local.00f44b6051d802a0084beb92c795ae8f
-rw-r--r--  1 qiushao  staff    84K  7 11 19:09 local.00fd14f7c61fd3a614492c2d253d29ad
-rw-r--r--  1 qiushao  staff   3.9M  7 12 15:32 local.011dacb1eabde2f8c4339603d0c1756c
-rw-r--r--  1 qiushao  staff   286K  7 11 19:13 local.01385200755f2060f1d51a0d83bedc72
-rw-r--r--  1 qiushao  staff   371K  7 11 19:15 local.015f00ca9bd2a8b0778acb46c508f4bf
-rw-r--r--  1 qiushao  staff   1.7M  7 12 14:49 local.0231cc8f5787f9c01b4a8970941a7d97
-rw-r--r--  1 qiushao  staff   1.2M  7 11 13:23 local.0234701bb4712ded69899591d71c48d5
-rw-r--r--  1 qiushao  staff   1.4M  7 11 13:34 local.024745f3b09326e40fbc60d22b3bd8e4
-rw-r--r--  1 qiushao  staff   8.7K  7 11 19:08 local.0321d78eeea755099d8fd40bccbfbaeb
-rw-r--r--  1 qiushao  staff   1.4M  7 11 13:35 local.0370707bd68defcd5ee129c8a675381e
-rw-r--r--  1 qiushao  staff   160K  7 11 19:11 local.03cf6dccb7fc5fdbac9eb07bce3a83ad
-rw-r--r--  1 qiushao  staff   366K  7 11 19:07 local.04503f8953174d5cb28fc70bdd9552f1
-rw-r--r--  1 qiushao  staff   601K  7 12 14:58 local.049b569592f3547e9f5081b48b41d9c6
-rw-r--r--  1 qiushao  staff    46K  7 11 19:02 local.04b9ac9c3638d61350001838a57c1975
-rw-r--r--  1 qiushao  staff   247K  7 11 19:13 local.04e6f7138b86e2eb5ef08ace10c5f695
-rw-r--r--  1 qiushao  staff    27K  7 12 10:44 local.053b80dc0f76e285c58e96c2bbc41dc7
-rw-r--r--  1 qiushao  staff    21K  7 12 10:43 local.055b6e049a82f3fbd371b51c9c277c80
-rw-r--r--  1 qiushao  staff   915K  7 11 12:56 local.05feb56c35958490cb05b8932bad0ad3
-rw-r--r--  1 qiushao  staff    22K  7 12 10:05 local.065f7c6c089864f727c84244cd4eafd9
-rw-r--r--  1 qiushao  staff   649K  7 11 13:38 local.071782767d8f2c81ab8374d4125d6d81
-rw-r--r--  1 qiushao  staff   437K  7 11 19:16 local.072143ac8d024ecddcc48b23768f9135
-rw-r--r--  1 qiushao  staff   183K  7 11 18:48 local.07233fefb674059deae88f45db8664fa
-rw-r--r--  1 qiushao  staff   8.2M  7 11 14:02 local.07623885f403cf2a110a6c95bd589508
-rw-r--r--  1 qiushao  staff    30K  7 12 10:44 local.07e9b938e305334b0e7746baa70502eb
-rw-r--r--  1 qiushao  staff    37K  7 12 09:54 local.0814dc93b4abd0a377bd44e52a6ac71b
-rw-r--r--  1 qiushao  staff   307K  7 11 19:36 local.081f0dc08625a6b8db37dabe8bb45df8
-rw-r--r--  1 qiushao  staff   298K  7 11 19:14 local.08265f4f73bbf9bbddd61c9b23da3136
-rw-r--r--  1 qiushao  staff   1.3M  7 12 14:56 local.08d694bd654ad3d411be595d32273136
-rw-r--r--  1 qiushao  staff   412K  7 11 19:16 local.097a44e9e940087517774352a59fd20a
-rw-r--r--  1 qiushao  staff   185K  7 11 19:11 local.09a74dbe76862bba0cb7dbd6ee33406b
-rw-r--r--  1 qiushao  staff   266K  7 11 19:05 local.09f074e5be472e964b2cc5b01568ca5e
-rw-r--r--  1 qiushao  staff   416K  7 11 19:16 local.0a3c5e36c42e5078a8f331e9546ab4b0
-rw-r--r--  1 qiushao  staff   1.9M  7 11 13:47 local.0ac6a462f2af0de0fac8671f24eab76f
-rw-r--r--  1 qiushao  staff    28M  7 12 12:09 local.0b2a589ce6d6fe0a26bda5b322a73edd
-rw-r--r--  1 qiushao  staff   1.7M  7 12 14:59 local.0b334b5def985c6805f7eb98af674278
-rw-r--r--  1 qiushao  staff   283K  7 11 19:06 local.0bba797a6bb14b2f3b669d06bb5f1c2c
-rw-r--r--  1 qiushao  staff   256K  7 11 19:13 local.0c0abaaf1d8ec8d9a9d47a5a0b9699fb
-rw-r--r--  1 qiushao  staff   1.9M  7 11 13:36 local.0c42bbd9cdeffe11d4514f68a0bf5ff4
-rw-r--r--  1 qiushao  staff    13M  7 11 17:25 local.0cdc012de0985a68ac9356de725066c2
-rw-r--r--  1 qiushao  staff    13K  7 11 19:44 local.0d62ad12b542af6a784d1d41f690ed3d
-rw-r--r--  1 qiushao  staff   288K  7 11 19:36 local.0e82e643c1a2164da7ad319a09dc6e95
-rw-r--r--  1 qiushao  staff    11K  7 12 10:43 local.0eebb5b0b7769d2cee417d659faa3703
-rw-r--r--  1 qiushao  staff    19K  7 11 18:52 local.0f2e2c4a8eff2d9a8cb5e5e229acd427
-rw-r--r--  1 qiushao  staff    23K  7 12 10:30 local.0f592250896e433cc6f26fce6c14df42
-rw-r--r--  1 qiushao  staff   2.1M  7 12 14:47 local.0f66dd760bff6d102ea19d90b0bf4171
-rw-r--r--  1 qiushao  staff    45K  7 11 19:30 local.0fea9b20e6f3f22b4c6677d6948cc0b1
-rw-r--r--  1 qiushao  staff    18K  7 11 19:45 local.10439730e2b07b26b38713a533dca560
-rw-r--r--  1 qiushao  staff   118K  7 11 19:10 local.113ee6916f45aa192e9ea1f44c836d3d
-rw-r--r--  1 qiushao  staff   6.1M  7 11 13:57 local.1147523cdad043bfcd0176b24f785375
-rw-r--r--  1 qiushao  staff   3.5M  7 12 15:30 local.116ea63ba5001344aca87f07df3c5815
-rw-r--r--  1 qiushao  staff    12K  7 11 19:07 local.1190d37a7172b90f55386043742a48af
-rw-r--r--  1 qiushao  staff    30K  7 12 09:57 local.120eee32072d4575932f292bfb205eb0
-rw-r--r--  1 qiushao  staff    13K  7 11 19:29 local.12c7f764dc90b2b045432a6a681a260f
-rw-r--r--  1 qiushao  staff   340K  7 11 19:37 local.13050531b4d4b93be7d273d6836e08b0
-rw-r--r--  1 qiushao  staff   1.6M  7 11 13:46 local.13397da56a757e01e66c769d9cc390dc
-rw-r--r--  1 qiushao  staff    31K  7 11 19:31 local.137fbd58cdc0253668e81f2420e9bbbb
-rw-r--r--  1 qiushao  staff   180K  7 11 19:11 local.13c5a943853efe62a3bd59729c9d50c7
-rw-r--r--  1 qiushao  staff   5.6M  7 12 12:03 local.1413403586f7dacf8ebb9cab8b44ed3f
-rw-r--r--  1 qiushao  staff    79M  7 12 12:13 local.1431e635c8f7048714ddecabd9db4424

wangdashuaihenshuai commented 4 months ago

每次任务删除都只会删除最久的10条，这部分逻辑确实不好应对需要大量删除的场景，之后会增大删除的批量，还有频率。

HuolalaTech / page-spy-web