pat
commented
4 years ago Hmm… do you have address set for the appropriate environment in config/thinking_sphinx.yml?
Closed JasonBarnabe closed 4 years ago
pat
commented
4 years ago Hmm… do you have address set for the appropriate environment in config/thinking_sphinx.yml?
JasonBarnabe
commented
4 years ago Connection is via socket. My intention is that the application server is the only thing that talks to Sphinx (which is on the same server). The Sidekiq server has no connection to Sphinx.
For updates everything works out because it goes through Delayed Job (also hosted on the application server), but for deletes it seems not to.
pat
commented
4 years ago Oh, you're using both Sidekiq and Delayed Job, I missed that distinction - and Sphinx on the Delayed Job server. Hmm 🤔
I'm not sure there's a good workaround for this if you want to keep using sockets rather than TCP - TS is trying to mark records as deleted in Sphinx, to ensure they're not returned by searches later. Deltas can't account for this because they're essentially only lists of new/changed records.
There might be a way to interrupt the TS deletion callback, but that'll lead to nil values being returned in search results… and while you could ignore those, any pagination of search results will be unbalanced.
JasonBarnabe
commented
4 years ago The reason for this setup is that:
If I could run ts-sidekiq-delta against a remote Sphinx then I'd prefer that.
Barring that, in this particular case, these records are not indexed anyway (they are soft-deleted). Is there a way I can tell it to not run the ts callbacks?
pat
commented
4 years ago All of that sounds reasonable, and yes, I'm afraid delta processing must be on the same machine as Sphinx. Sphinx has no way to manage daemon states or indexing remotely.
I don't think there's an especially elegant way to stop the callbacks, so the simplest approach may be to override the callback class directly:
class ThinkingSphinx::ActiveRecord::Callbacks::DeleteCallbacks
def after_destroy
# do nothing
end
def after_rollback
# do nothing
end
endI don't especially want to never run the destroy callback, but just on this one specific call in this one specific job. Ideally there would be something like
without_ts_callbacks do
record.destroy
endI suppose I could monkey-patch something for that.
pat
commented
4 years ago If you're happy to skip all TS-related callbacks on this action:
ThinkingSphinx::Callbacks.suspend do
record.destroy
end(I hadn't realised it was just one situation you wanted this done for - suspending callbacks will also stop delta updates, hence I didn't make this suggestion for a presumed global approach)
JasonBarnabe
commented
4 years ago Yeah this is just a "clean up soft-deleted things" job. That works!
If this (not being able to delete remotely) is not something that's fixable, it would be good if the restrictions on running against a remote Sphinx were spelled out all in one place.
pat
commented
4 years ago Glad to know that approach is good enough :)
FWIW, deleting remotely works with TCP connections, just not with socket connections. The same goes for searching as well (i.e. you won't be able to run any searches on your Sidekiq server). I'll look at making that a bit more clear in documenting the ability to use socket connections.
JasonBarnabe
commented
4 years ago To be clear, my Sidekiq server doesn't have a socket connection to Sphinx either. It just got the same thinking_sphinx.yml as the application so that's the error that was reported when it tried to connect.
The confusion on my part is more about deletes not working if there is no connection to Sphinx, but updates being OK.
pat
commented
4 years ago Made a note for remote Sphinx + UNIX sockets a little while in the docs, so I'm going to close this to keep the issues tidy, but further doc edits are very welcome! https://github.com/pat/thinking-sphinx/commit/d1fb696ff5a884876d1bb4d5c6ae3f69ceb50a57
App server:
If I connect from another server (one running Sidekiq) and perform an update, things are cool. Delayed jobs are added and are processed properly.
However if I delete from another server,
It's trying to contact Sphinx, which is not present or accessible from that remote server.
Relevant backtrace: