Feature Request: Allow fetching data for Options across pages via network on Select/Multiselect

[fmenezes]

Hi there 👋

thanks for the great work at the survey library, we use it at MongoDB in the AtlasCLI. 🎉

I just wanted to check if this feature makes sense? Unfortunately, for some of our use cases, we have to find ways to list more than 500 items, which our APIs would not allow.

Feature Request: Ability to fetch options via network

Today we are allowed to provide options for Select/Multiselect as a []string and we can even set page size, unfortunately, if our dataset is big we have no option to go to the network and fetch more data after the user navigated to the bottom of the page.

If instead, we could set a function to provide options that would allow us to fetch data on demand.

before

type Select struct {
    Options       []string
        ...
}

after

type Select struct {
    Options       func(page int) ([]string, error)
        ...
}

Note: although my description seems to be defining a breaking change I think the best would be to implement it in a non-breaking change manner, the description was intended to be informative only.

[fmenezes]

FYI I can work on the PR, just wanted to check if this is something that would be acceptable?

[mislav]

As someone who has implemented large Select/MultiSelect with data fetched from the server, I feel you, but I'm not yet 100% sold on the idea yet. How would we determine when you have navigated to the bottom of the page? What kind of UI loading indicator would there be for the user while the next page of data is fetching? How would filtering functionality work when data is being obtained from the server?

Is it an option that you define your own Select component in your app and implement the functionality there? A custom survey component only needs to implement this interface: https://github.com/AlecAivazis/survey/blob/c2be27f4a7faa58a920ea402d0984647a5b8deb2/survey.go#L123-L129

You could copy over the existing Select implementation to get you started.

[fmenezes]

@mislav I think that is a cool idea, to implement a custom Select copy/pasting the code.

I'll work on that and share the results at a later stage.

[fmenezes]

Hi there @mislav as agreed I'm reporting back on this.

We've indeed implemented the custom version of Select that is able to fetch via network in here. And decided not to move forward. The context on why not to move forward is in the PR.

My final solution was to have a two-step process

1. fetch
2. (if too many results) ask for the filter (survey.Input) and go to step 1

Until we finally have an acceptable number of options.

To answer your original questions:

How would we determine when you have navigated to the bottom of the page?

In my proof of concept I've had the count available (our API uses offset pagination) so I could guess, with cursor pagination it would probably require fetching a few pages to cache (like previous, current, and next).

What kind of UI loading indicator would there be for the user while the next page of data is fetching?

I had to use spinner, for an API we could possibly have a few presets (like we currently have the icon presets).

How would filtering functionality work when data is being obtained from the server?

In my code Options was moved from a []string to func (filter string, page int) (options []string, count int, err error) so while navigating across pages I've included the current filter. Everytime a filter was changed I would move back to first page.

There was also a behavior change, I had to wait for users to press <ENTER> to to filter after they've finished typing to avoid calling APIs indefinitely. (Most likely there would be a better algorithm to figure out when to call the API while typing)

On another note, I've propagated network errors up the component, in other words, things like retrial and error encapsulation would have to be done on by the developer implementing the functionality to fetch data.

I still think it would make sense to try having an interface for this but I see the difficulties. That being said anyone facing a similar issue in the future my approach was to combine multiple questions to have the desired outcome.

[mislav]

Thank you for sharing! 🙏