search

hunt-framework / hunt

A flexible, lightweight search platform
59 stars 10 forks source link

Inifinite loop while inserting Quickstart documents #74

Closed alexbiehl closed 10 years ago

alexbiehl commented 10 years ago

(I am on GHC 7.8.2, 32 Bit)

When running the Quickstart examples I noticed when executing the document inserts hunt runs forever with: 

[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
[2014-06-11 16:59:19 UTC : Hunt.Interpreter : DEBUG] Exec: InsertList [..]
...

All but the first three (equals example doc count) InsertList commands interpreted by execCmd had an empty list in them.

The glitch seems to be in toBasicCommand. More precisely in line 265: 

[splitBatch 30000000000 $ foldl (\(Cmd.InsertList us) (Insert u)
                -> Cmd.InsertList (u:us)) (Cmd.InsertList []) cs']

Reducing the number to something in the range of integers like 2147483647 (or anything below MAX_INT) worked fine. It's probably because I'am on a 32bit machine.

Edit: splitEvery cannot handle negative numbers. (30000000000 becomes -2147483647)

chrisreu commented 10 years ago

You are right. Thanks for taking a look at the project.

I set it to maxBound for now. This should fix the problem for now. I'm not sure if the need of splitting batches is still necessary at all.

Also note: Hunt is intended to be run on a 64bit machine. I am not sure how stable it might run on a 32Bit machine.

alexbiehl commented 10 years ago

Ah yes, thought so too, since the index is stored in-memory 32bit machines could only handle max 4gbs anyway.