Develop a graphical ZIM explorer

openzim / overview

:balloon: Start here for current projects, how to get involved, and joining community calls. A resource for new and veteran members of the offline commmunity

2 stars 1 forks source link

Develop a graphical ZIM explorer #37

Closed benoit74 closed 3 months ago

benoit74 commented 3 months ago

While checking ZIM quality or diagnosing issues, the QA team often needs to:

look at the list of entries of a given ZIM
search for a specific text or patterns in this list (and count entries, get total size in ZIM)
get details (mainly size) of a given entry and observe content (when its text)
get list of duplicated entries

This is especially important for Zimit ZIMs where the ZIM content is way less predicatable than other scrapers.

While as a developer I do this with a combination of zimcheck / zimdump / grep / wc / ..., this is not possible for the rest of the content team and this is not convenient at all for me, consuming too much of my time.

Should we develop a graphical ZIM explorer?

@RavanJAltaie @Popolechien does it make any sense for you? Do you wanna add more?

Popolechien commented 3 months ago

That is a good question. Feels like this would be needed, but I also suspect the tool would still be too complex for us to use as we wouldn't know what it is we're looking for in a zim. Other than that, isn't there a bit of overlap with the CMS (or couldn't some of the features you mention be integrated into it)?

rgaudin commented 3 months ago

I still believe training a couple people to use a few tools would be a better way.
there is indeed overlap with the CMS but CMS is a (to-be) publication & QA tool. I can see use cases that it wont cover in your workflow.
a good part of the needs is available via fuse module. Maybe packaging it properly for windows would help

mgautierfr commented 3 months ago

a good part of the needs is available via fuse module. Maybe packaging it properly for windows would help

I may have miss something. But we have that ?

rgaudin commented 3 months ago

I may have miss something. But we have that ?

I though you had someone working on one…

mgautierfr commented 3 months ago

We have this https://github.com/openzim/zim-tools/pull/400 But I don't know if juzz0 is actually working on it.

kelson42 commented 3 months ago

Definitly a ticket for ZIM tools, I would recommend to make one ticket per question. Current tools should be able to do the job. If people can not use the command line and follow a howto then they are not a fit to diagnose anything for a broken ZIM IMHO. ZIM fuse might be indeed part of the response, but this is Linux only.

rgaudin commented 3 months ago

ZIM fuse might be indeed part of the response, but this is Linux only.

It's not ; via third party tools which is why I mentioned packaging. Not sure setup can be simplified enough though.

benoit74 commented 3 months ago

I had something way simpler than command line tools, and way more straightforward than downloading ZIM, issuing proper command line, pushing result to a text editor because I still suck at using less, forgetting to delete the ZIM and finally having a disk full, ... Even for me it would save me time. Is it worth it? I don't have measures to bring to the table, but I know it would increase significantly my confort.

v1 could be just a web UI running on storage server so that ZIMs are already available locally, and displaying a raw list of files, and once a ZIM is clicked displaying the list of entries.

I just realize this is indeed part of the CMS whishlist "Book Overview with basic search/filter". Let's close this issue then, CMS feature will be enough from my perspective if I understand it correctly.

rgaudin commented 3 months ago

For some reason I read Desktop while it's not written anywhere 😅