doc/README.xapian needs some love

cyrusimap / cyrus-imapd

Cyrus IMAP is an email, contacts and calendar server

http://cyrusimap.org

Other

543 stars 149 forks source link

doc/README.xapian needs some love #3002

Open elliefm opened 4 years ago

elliefm commented 4 years ago

The instructions for "How to install Xapian for Cyrus 2.5" say to download 3.0.0-beta1...

I was about to fix the (now broken) download URL, but it looks like the whole document needs a rethink?

dilyanpalauzov commented 4 years ago

According to my notes from a year ago, I do not understand about Xapian, whether it indexes #addressbooks/ and #calendars/.

rsto commented 4 years ago

@dilyanpalauzov Cyrus indexes some properties of calendar events in Xapian: description, summary, attendees, organizer and location. All these properties are indexed separately here.

Addressbooks and contacts are not indexed in Xapian.

dilyanpalauzov commented 4 years ago

So cyrus does not implement CARDDAV:addressbook-query? This is stated so in http_carddav.c:44-45 since 2013. Or it implements it, but there is no index?

From index.c:extract_icalbuf() I do not see how the index in the ical_uid.sqllite3 table is used to search for SUMMARY. But if you say that iCalendar:SUMMARY is indexed, then I belive you.

That said my reading of the last comment is, that:

For calendars and addressbooks Xapian is not used. How does the squatter process determine, that it shall skip a calendar mailbox on full reindex?
Sometimes the cyrus.index and sometimes the sqlite3 database is used to index fields in vCard/iCalendar. On what procedures it is determined, when to use cyrus.index and when the sqlite3 database? Is the former used just for From(ORGANIZER), Date(DTSTAMP) and Subject(SUMMARY)?

rsto commented 4 years ago

For calendars and addressbooks Xapian is not used. How does the squatter process determine, that it shall skip a calendar mailbox on full reindex?

Both calendar-query and addressbook-query are implemented for CaldDAV and CardDAV. However, they do not use Xapian. The Xapian-indexed calendar events are used in JMAP calendars, where text search requires stemming and fuzzy search. Accordingly, squatter does index calendar events on full reindex.

On what procedures it is determined, when to use cyrus.index and when the sqlite3 database?

@ksmurchison knows this better than me. But in my understanding we do not use the sqlite3 database for calendar-query? The metadata stored in sqlite3 is useful for lookups during GET as well as using some extracted event data e.g. for alarms.