GROQ: sort by random

[Realetive]

It would be handy to have the ability to sort by random entities in GROQ. The syntax would be similar to sort(): [_type=="doc"][0..9] | rand() will return 10 random docs.

[deviant32]

I agree, this feature would be really nice, it would also cut down on the number of assets requested. Currently we have to query for all of the documents, shuffle the array and slice.

[judofyr]

Bikeshedding: shuffle() is probably a better name. It's at least what it's called in Lodash, Ruby, and Python.

Also, the query should be ([_type=="doc"] | shuffle())[0..9], no?

[judofyr]

Some more thoughts: A query with shuffle() is essentially uncacheable, which is unfortunate. Potential solution: The API could support a seed-parameter. The same seed will always give the same random response, and the combination query+seed can be cached. It's then up to the client to decide how much randomness they want. If they do seed = Math.round((Math.random()*n)) they have n different random results. A low n gives higher chance of hitting a cached response, but less variability, while a high n gives "more" randomness, but less cachability.

[the94air]

Been a while since this issue was open. Will this be implemented? 🤔

[Kirbyasdf]

Random Option for sort order would be great, use some of that USD 50 million raised and ≈ 100 employees to implement the <10 lines into your library.

[kmelve]

@Kirbyasdf I guess you're joking, but often in these cases the thing that takes time isn't necessarily the implementation but having enough time to take a decision for an API that many people use that you'll have to live with and support for a long time. And as @judofyr is pointing out, we also have to consider how this impacts caching, which is a really real thing when things scale!

That being said, we do see how this is a super useful feature, and it's something that we're interested in providing support for!

[Kirbyasdf]

@kmelve no cap quoting your head of developer relations,

"That being said, we do see how this is a super useful feature, and it's something that we're interested in providing support for!"

so lets stop the talk and be about

@judofyr literally did the logic for you

[judofyr]

@Kirbyasdf: Thanks, we've noted that you're interested in this feature and we will update this issue when there's more progress.

[schellenbergk]

I need it too :)

[judofyr]

I need it too :)

I would love to hear more about the use case. Or, more specifically: is this a case where you have a thousands of documents and you want to randomly pick X, or is more for shuffling a smaller array (and not having to do it on the client-side)?

[Kirbyasdf]

@Kirbyasdf: Thanks, we've noted that you're interested in this feature and we will update this issue when there's more progress.

"we will update this issue when there's more progress."

"more progress"

3 years....it has been 3+ years

let me be clear here, you have raised 50mm+, and have ≈100 employees

lets do some math here, conservatively..... because look at your partners and I very much doubt the founders gave away 50% already at Series B

lets give you a valuation of double, and credit lines reflecting this; so we are at 100mm, lets say each employee's avg salary is USD 150k, thats 15mm + 32% SS tax(4.8mm) so 19.8mm for employees....lets throw another 10mm for services and 20mm for tangible items/space....this brings us to 49.8mm.

Now you are telling us you don't have the ability to generate a solution for this feature from 3 years ago? This is what you are telling us.... you are typing on your keyboard and telling us that you can't find the resources with 50.2mm left over to put in a a simple feature cracking the code challenge level II feature........that you already pseudo-coded 3 years ago.

"progress" you say

While I'm at it, fix your docs....you guys write it like you don't care if someone uses your lib to its full potential....libs like Tachyons are written extremely well because it's literally all done by 1 person; they care because they own it....take accountability for your product.

[the94air]

TBH all the time since I have been following this feature request, I didn't get the impression that there is a serious work going on. The time we all waited shows an issue on the way you schedule your tasks and backlog (it's not just about personnel) additional to the way you look into the input of the developers who implement features to your customers. But at the end of the day, this is none of my business and you owe me nothing. I am just trying to show how does it feels like, from my point of view as a developer. Try to improve from there, whatever was the reason that was holding to make this possible. This is an unnecessary message that I usually don't bother to make, but I love your product and I would like to see it improve.

[kmelve]

Hi folks,

First of all. We're all just people here. We truly appreciate it when you feel passionate about what we make, and we share your frustration when we can't prioritize a feature that you'd like to see happen. But let us clarify some things here.

This is not about being able to make a feature. It's about if we have prioritized it or not. Sure, having more people and funds help, but the equation above doesn't take into account that we have prioritized roadmaps that follow other rationales and product life cycles, or that there is a whole lot more that goes into building a software company. It also takes time to build product development teams, hire developers and product managers etc..

This feature specifically isn't a “just do it” thing, as few are. Yes, the implementation over might work in theory. But it also needs to work through all of GROQs grammar, we need to add it to the specification, and to all our libraries/test suites. We need to make sure it scales appropriately (think if you have 1mill documents), that it has proper docs (we care deeply about that too), and that we have great education about how to think about caching these queries. You see where this is going.

We haven't been able to prioritize new GROQ features for a while, because our Content Lake team has been busy on other API surfaces and scaling our service as we get more customers and users. We're working on some other GROQ features that we plan to ship soon.

We don't claim that we're doing everything perfectly or can't improve. We appreciate all feedback, especially when things aren't awesome.

Sometimes it takes time to get to a feature request, and sometimes we don't get to it at all, even though some people would like to see it happen. If you want to advocate for a feature, it's super helpful for us if you explain your use case and/or how you're blocked. If you want to append to this issue, then we ask you to help us understand what you hope to solve with this feature.

[Kirbyasdf]

"you are typing on your keyboard and telling us that you can't find the resources with 50.2mm left over to put in a simple feature cracking the code challenge level II feature........that you already pseudo-coded 3 years ago."

Re your comment on "product development teams, hire developers and product managers etc."...... everyone you are hiring is for marketing/sales inb4 the one relations based engineer, come on now.

[kmelve]

OK, @Kirbyasdf. We get that you're frustrated and we have tried to give you more context while saying that we appreciate the input and asked for concrete use cases. You're drawing unreasonable conclusions based on a few data points, and you're not helping us understand why you need this.

I don't think we're getting anywhere constructive. Rest assured that there are actual humans behind Sanity that do want to ship awesome features, but spending time on this line of arguing won't help you or us to get to that place.

I'm locking this conversation now. If you have use cases that require sort by random functionality in GROQ, feel free to email me and I'll relay them to the right people.

[atombender]

Just to put it out there, another possibility here is to offer a math::random() function, and let you sort using it:

* | order(math::random())