search

raycast / extensions

Everything you need to extend Raycast.
https://developers.raycast.com
MIT License
5.27k stars 2.97k forks source link

[Gitlab] Do client-side filtering of search results #14648

Open meesvandongen opened 5 days ago

meesvandongen commented 5 days ago

Extension

https://www.raycast.com/tonka3000/gitlab

Description

Doing client-side filtering would probably speed up searching significantly, for example for projects; though I am not sure how feasible it is.

Who will benefit from this feature?

Anyone using the gitlab extension

Anything else?

No response

raycastbot commented 5 days ago

Thank you for opening this issue!

đź”” @tonka3000 @Loskir @immint023 @duboiss @metakirby5 @macbookandrew @shaoye @charl3j @emlez @HBunnick @jfkisafk @RomanSchmid @jessedobbelaere you might want to have a look.

đź’ˇ Author and Contributors commands The author and contributors of `tonka3000/gitlab` can trigger bot actions by commenting: - `@raycastbot close this issue` Closes the issue. - `@raycastbot close as not planned` Closes the issue as not planned. - `@raycastbot rename this issue to "Awesome new title"` Renames the issue. - `@raycastbot reopen this issue` Reopens the issue. - `@raycastbot assign me` Assigns yourself to the issue. - `@raycastbot good first issue` Adds the "Good first issue" label to the issue. - `@raycastbot keep this issue open` Make sure the issue won't go stale and will be kept open by the bot.
macbookandrew commented 5 days ago

I suspect it would improve performance for users with access to tens of resources. For users with access to hundreds or thousands of resources, the extension would have to load everything up front and then filter client-side, resulting in significantly slower overall behavior. 🤷🏻‍♂️

jessedobbelaere commented 5 days ago

I was using Alfred with the Gitlab extension before using Raycast. It was very snappy to search for a project. In that extension they chose to build a local cache and refresh it periodically. Searching a project was then always done on the client-side. I see that the Raycast gitlab extension has a cache.ts too.

In #14316 I added a project-membership dropdown to only search in projects you are assigned to (instead of all public gitlab projects), so for that use-case it could make sense to keep a local cache and search inside the cache instead of invoking an http call to the Projects API on each search query change... We have 500+ projects; it was way more performant with client-side filtering (alfred) than waiting for http calls to resolve, so I would be happy to see some experiments regarding this đź‘Ś

macbookandrew commented 5 days ago

Caching would be great!

jfkisafk commented 5 days ago

Does loading large number of projects not open this extension to JS heap memory issues for large workspaces? We can perform client-side filtering but keeping all those resources in memory/cache will come with risks (that are currently not present with server-side filtering).

PS: Not shooting down the idea, just indicating that if we release this feature, we might need to do it behind feature flag to not risk existing large customers.

macbookandrew commented 4 days ago

I recalled some previous discussion about server-side vs. client-side caching: https://github.com/raycast/extensions/pull/10372#issuecomment-1915597316

I believe the “My Groups” and “My Projects” commands do pull all of the groups/projects, cache them, and use client-side filtering; the “Search Groups” and “Search Projects” commands include the search term in the API request, so use server-side filtering.

Unsure about the other “My” vs. “Search“ commands without doing some digging, but @meesvandongen you might get better performance using the “My” commands instead of “Search”?

meesvandongen commented 3 days ago

I did some testing and I can see the problem now; getting a single page (100 results) from gitlab.com takes around 5-10 seconds. And I need quite some pages. Building this cache after opening the "My Projects" page takes too long to be useable (in fact I do not get any results at all in the UI). I compared it with the graphql API and it seems this one takes around 2-3 seconds per page, so this could be an option.

If you combine the graphql API to get the total project count and feed this into the rest api with the offset-based pagination, you can probably get as much as the rate limit allows; in a test I was able to get 1100 projects in 9 seconds, which seems like a fine UX if it is fast after this.