offspot / metrics

Metrics subsystem for Kiwix Hotspot
GNU General Public License v3.0
0 stars 1 forks source link

Add support for the new file manager #104

Closed benoit74 closed 9 months ago

benoit74 commented 9 months ago

In replacement of Edupi, we now have a new file manager.

Computation of added / removed shared files hence has to be updated to the new system API.

benoit74 commented 9 months ago

This seems to be a complex issue.

While it was easy with EduPi to detect files added and removed simply from HTTP verb and URL, tinyfilemanager uses one single URL for all actions and details about action to perform is inside the request (form data).

For instance to upload a file, one or multiple payload (files are truncated in multiple requests when too big) like the one below are sent to /admin/index.php?p=folder1 (only the target folder is present as a URL query string):

-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dzuuid"

44f17186-a31f-4f3a-8bfa-ae0a26b52c22
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dzchunkindex"

0
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dztotalfilesize"

42718
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dzchunksize"

2000000
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dztotalchunkcount"

1
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="dzchunkbyteoffset"

0
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="p"

folder1
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="fullpath"

65b77dccc056cd91766ff12a.img.torrent
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="token"

3bcdc7b52f3c277245ac497d4c6c571598af0fb37ad09342b95162988e9bd220
-----------------------------14090790777978564891573427461
Content-Disposition: form-data; name="file"; filename="65b77dccc056cd91766ff12a.img.torrent"
Content-Type: application/octet-stream

... continued ...

And it is possible to delete a whole folder and its content, or even multiple selected items (files or folder) in one pass:

POST http://localhost:8881/admin/index.php?p=
Content-Type: application/x-www-form-urlencoded
p=
group=1
token=3bcdc7b52f3c277245ac497d4c6c571598af0fb37ad09342b95162988e9bd220&file%5B%5D=folder1
delete=Delete

I'm not sure how we can leverage this (even if we can slightly modify the source code of tinyfilemanager) to get stats about number of files added and removed in a given period only based on Caddy logs (which do not contains the payloads for sure, only URL & request / response headers)

rgaudin commented 9 months ago

I think we can edit tfm to add headers describing what action completed. Can you list the different actions/infos you are looking for?

benoit74 commented 9 months ago

Metrics, for now, needs to know:

Your idea of using response headers looks great, thank you, I will have a look at it!