timshannon / badgerhold

BadgerHold is an embeddable NoSQL store for querying Go types built on Badger
MIT License
515 stars 52 forks source link

Question on String Sort #55

Closed richip closed 3 years ago

richip commented 3 years ago

When using Find and Query, what is badgerhold's definition of string sorting? The reason I ask is that I'm getting a weird order for a string field that I've asked Badgerhold DB to return. The field is a string and I get a result set similar to:

Sorting is apparently doing something because when I do a Reverse, it returns the list to me in reverse order but in this weird, non-alphabetic order.

I'm using github.com/timshannon/badgerhold v0.0.0-20200316131017-7bcffb989f0d

timshannon commented 3 years ago

Are you doing an actual sort with .SortBy("Field") in a query? If you aren't, then the order could either be by the index order, or by the key order, both of which will be byte order of of the encoded values (by default GOB).

richip commented 3 years ago

Yes, using .SortBy("FieldName"). And for FieldName, I'm using the Go struct attribute/field name, first letter uppercase since its required to be exposed. Introspecting the query variable shows the fieldname as the only string in the query.sort []string slice.

timshannon commented 3 years ago

Could you write up a self contained test func so I can replicate this? I have several tests on sorting by strings here: https://github.com/timshannon/badgerhold/blob/master/sort_test.go

richip commented 3 years ago

Hi. So I did some troubleshooting and found the cause. For my data structure, I'm using *string to store strings and in the code, query.compare is comparing them as pointers. Since we need to differentiate between the empty string (string zero value "") and nil, we have to keep using string pointers. Is there a way to make this work with sorting? I suppose I could write code that dereferences pointers in query.go, but is there a better way?

richip commented 3 years ago

How would one use a Comparer?

richip commented 3 years ago

I'll try MatchFunc(). Thanks!

richip commented 3 years ago

Sorry. MatchFunc won't work (I'm not using it to filter but to sort) so I thought I'd try Comparer. This is more a Go question, but I'm hoping this is a common enough badgerhold scenario that people here can help.

I'm trying type MyString string and just defining a func (value *MyString) Comparer(other interface{}) (int, error) but working with this custom type is tripping me up. I can't assign literal strings to it (e.g. var a MyString and a = "string"). I can't make a string pointer and assign it to a variable of type *MyString. Any help would be appreciated.

Instead of comparing fields in a record, can I compare the record itself and just use a custom Comparer to target my field? What would the syntax for composing a Query be?

richip commented 3 years ago

Alright. In the end, I was able to make my custom type MyString string work. I opted to go with implementing Comparer, but I did toy around with the idea of just implementing String() string and let compare() go to the default type and have them Sprintf'd into string. But with Comparer, I could control the ordering of nil values.

Thanks again!

richip commented 3 years ago

There's this little bit of code at https://github.com/timshannon/badgerhold/blob/eb78b03f80970f983759e77db7d666ef86e67f5a/compare.go#L54 that might not be a bad idea to pull into compare().