Closed kalsan closed 1 year ago
Thanks for logging this issue @kalsan :) It has also been covered by #1213 - but then I reverted that change because it no longer worked with Sphinx, and I was hoping that it might get fixed in Manticore. That said, I never logged an issue with them!
I'll try to find some time to verify what should be the right behaviour for Manticore, and then we can decide whether the TS code needs changes or if a new release of Manticore will fix it.
Granted, it'd be nice if TS just worked properly with both engines… further PRs are definitely welcome if anyone has time before I get to it (this month is already looking pretty hectic).
Sounds great! Thank you for your support and looking forward to hearing from you :-)
As a temp workaround I'm injecting total_pages into the mask class instance.
p.send(:mask_stack).first.instance_variable_set '@total_pages', (p.total_entries / p.per_page.to_f).ceil
Hi there! The problem is much worse than thought. Manticore 5.0.2 randomly forgets objects until ts:index
is run again.
Here's an example:
Mymodel.count # 14
Mymodel.search.count # 3 (wrong!)
# Is it TS or Manticore's fault? Let's try to get closer to Manticore...
ThinkingSphinx::Connection.take{|c| c.execute 'show tables;'}.to_a
# [..., {"Index"=>"mymodel", "Type"=>"distributed"},
# {"Index"=>"mymodel_core", "Type"=>"rt"}, ]
ThinkingSphinx::Connection.take{|c| c.execute 'select id from mymodel;'}.to_a # returns 3 IDs, wrong!
ThinkingSphinx::Connection.take{|c| c.execute 'select id from mymodel_core;'}.to_a # returns 3 IDs, wrong!
This leads to the suspicion that Manticore 5.0.2 is broken on itself. Would you agree @pat ? I will likely downgrade Manticore and test with an older version. Which exact version number would you recommend me to try?
Note that this bug only occurs after several days in a fairly busy production environment holding hundreds of thousands of data records. Some indices are affected, others are not. Restarting Manticore does not help, re-indexing helps. The affected indices typically lack between 1 and 20 records, so the amount of lost or missing objects is not proportional to the size of the index (I'm missing 9 of 14 objects in the example above, another index is missing only 4 of over 100'000 objects.
Best, Kalsan
You could try the freshly released Manticore 5.0.3? But otherwise, maybe jump back to the latest 4.x release?
But also: are these real-time indices, or SQL-backed indices, and in either case, how are you tracking changes (e.g. do you have callbacks in place)? Are you doing anything custom here - in particular, anything that may prompt a deletion of data from Manticore while it remains in your database?
Hi Pat
Just wanted to let you know that I've been working on downgrading the version and pushed a 4.x manticore to the server in question recently. Now observing and crossing fingers.
Best, Kalsan
Hi Pat
After running it for 2 weeks, the problem occured again with Manticore 4.2.0_211223.15e927b28-1
. I don't think there is anything you can do as this is most likely an upstream bug. Would you like to close this issue?
Best, Kalsan
@kalsan it's frustrating to hear that the problem with Manticore continues - but yes, sounds like the problem is on their side. Given it's intermittent, it's a hard thing to create a reproducible bug report for their team, but may be good to report nonetheless?
I figured out the cause of the problem with total_pages
- since v5.0.0, Manticore has changed the default behaviour of query stats to just consider the current page of results, not how many results there may be overall. I've changed the default behaviour in Thinking Sphinx to ensure total_pages is now set correctly (#1239), and released v5.5.1 of Thinking Sphinx with this fix. Which I guess is a bonus justification for closing this issue 😅 - sorry it took so long to get to the bottom of the cause!
But also, looking back at my previous suggestions: it seems I was mistaken about there being a proper v5.0.3 release of Manticore (at least, thus far). They have a tag in their repo, but no built binaries.
Hey Pat!
Great to hear you managed to solve the page issue, thanks a lot.
Yeah, I opened https://github.com/manticoresoftware/manticoresearch/issues/912 but the bug is currently not manifesting. It's a slippery fish. I agree with your assumption that the problem is likely on their side.
Have a great day!
Hi Pat
It appears that
total_pages
is incorrect withManticore 5.0.2 348514c86@220530 dev
as backend. I know it's Manticore because the problem did not occur with Sphinx.Here's how to reproduce it:
Here's another example, this time not using a model class but ThinkingSphinx directly:
Maybe this is of interest to you. Luckily the total page number can easily be calculated from the other numbers, so there is an obvious workaround while the problem exists.
Best, Kalsan