search

kiwix / kiwix-js

Fully portable & lightweight ZIM reader in Javascript
https://www.kiwix.org/
GNU General Public License v3.0
309 stars 135 forks source link

ZIM backend : enable full-text search #112

Closed mossroy closed 2 years ago

mossroy commented 9 years ago

That will need to be able to handle the index of ZIM files. This index uses Xapian file format, which does not seem to have a javascript implementation for now. Maybe with emscripten?

Jaifroid commented 5 years ago

@kelson42 I'm still interested in experimenting with the full text index in ZIM files, but there is a lack of information to go on. I understand from https://wiki.openzim.org/wiki/ZIM_file_format#Namespaces that the index is in namespace X, but that's all I've been able to find out. There is a broken link on that page to information about the full text index, with the address ZIM_Index_Format (see screenshot) -- I assume this hasn't been written yet, but on the offchance, is there a draft that might provide some hints about the format (other than that it's based on Xapian)?

image

mossroy commented 2 years ago

This might be superseded by #769 (use libzim for that), but there are still blockers like #513

kelson42 commented 2 years ago

@Jaifroid The ft/suggestion indexes are not part of the openZIM spec. Our Actually, libzim as ref. implementation can be built withput any dependence to Xapian. But, if it is, then the whole conplexity of using Xapian or locating the indexes is abstracted by the libzim. Therefore, you should now about the libzim API only IMO. More info about the libzim API at https://libzim.readthedocs.io/en/latest/.

kelson42 commented 2 years ago

Should we close this ticket as duplicate of #769?

mossroy commented 2 years ago

Yes, I think so : closing

Jaifroid commented 2 years ago

@kelson42 Ha! I think you were replying to a post of mine from August 2019!

kelson42 commented 2 years ago

@Jaifroid Crazy... somehow this notification came in my inbox. Sorry for the slight delay in my answer ;)