API access for demo equipment (e.g., cobot, microscope) - moving from streamlit to gradio

[sgbaird]

For equipment like MyCobot or the OpenFlexure microscope, we'd like people to be able to run full workflows with their own custom code outside of the app after requesting temporary access. This requires exposing a remote access API.

While the temporary access key approach is working well with HuggingFace + streamlit + HiveMQ Starter plan + MongoDB, streamlit doesn't support exposing a remotely accessible API directly as of now. We also don't want to make the broker address public. While gradio has a remote access API, gradio isn't so much of a dashboard framework like streamlit. This is why all the hardware control examples are written with streamlit. I have some examples written with gradio that I wrote while trying to decide between the two on the training lab repo.

Aside: Accelerated Discovery forum has a topic about temporary access credentials.

Maybe worth considering flask, https://chatgpt.com/share/670a6b28-abc8-8006-ab20-b7b283db7590, though I'd prefer streamlit. FastAPI maybe also an option.

EDIT: looks like gradio can do plot output now: https://huggingface.co/docs/hub/en/spaces-sdks-gradio

Also, supposedly gradio could support a two step process, requesting an access key, and then doing the control piece.

https://chatgpt.com/share/670a6ec1-d810-8006-88d3-4d688b567834

[sgbaird]

It would probably then make sense to link to a prefilled colab notebook that accesses the HF API and shows a simple workflow example.

Maybe there's a more straightforward overall solution though. Mainly, we want people to be able to interact with the device as if they were running the commands/functions directly themselves, just with the proper safeguards imposed, and without exposing private information.

[sgbaird]

Worth thinking about how difficult it would be to convert from streamlit apps to gradio apps.

[sgbaird]

Looks like YouTube video embedding may not be an issue if we switch to gradio: https://github.com/gradio-app/gradio/issues/2894#issuecomment-1366277045.

Aside: in a Colab notebook, we'll also want to show how to get a hugging face token to use with the gradio API so that people don't run into the API rate limits for non-authenticated access (something we experienced for crabnet-hyperparameter HF space).

[sgbaird]

Documentation for token: https://huggingface.co/docs/hub/en/security-tokens (maybe good enough to just share the link with a brief description of why)

[sgbaird]

Thanks @SissiFeng for trying out Gradio with the microscope: https://huggingface.co/spaces/SissiFeng/microscope.

I really like the way multiple endpoints are exposed in the "use with API" at the bottom, and how it has an "API recorder".

(more endpoints below)

It looks like most everything can be accomplished in gradio, same as streamlit. How was the conversion experience moving from one to the other?

(Note for others: the Gradio app linked above is not connected to the microscope)

[SissiFeng]

I summarized some key points to consider when converting from streamlit to gradio:

1. Streamlit uses a script-like sequential structure; Gradio uses a functional API, requiring defined input/output interfaces.

2. Streamlit uses session_state for managing state; Gradio primarily passes state through function parameters.

3. Gradio requires explicit definition of events and callback functions.

4. Consider using Gradio's caching mechanisms.

[sgbaird]

Thoughts on using Interface vs. Blocks? @gursi26 @SissiFeng @Jonathan-Woo

• https://huggingface.co/learn/nlp-course/en/chapter9/7
• https://chatgpt.com/share/6724facf-f178-8006-b87f-806fef809d17

States (i.e., whether to share something between all users, within a user session, etc.). Probably best to have one MQTT client instantiated (and reused) within a single user session.

• https://www.gradio.app/guides/interface-state
• https://www.gradio.app/guides/state-in-blocks

[SissiFeng]

I prefer “Blocks” for its flexibility, fine-grained UI design and complex workflow control, similar to Flask, with coordinated components and data flow across modules. State persists throughout each session, maybe the accuracy in callback updates is essential to avoid unintended overwrites or inconsistencies. Also, "Blocks" may lead to increased concurrent connections of multiple calls from one user.

[sgbaird]

The more I'm looking, I'm leaning towards Blocks a bit too, but it's also more verbose and I've been noticing issues with the number of threads (not sure if Interface would have a similar issue).

[SissiFeng]

That's right. With a multi-module, multi-step setup like Blocks, where each session maintains its own state all the time, the accumulated number of concurrent sessions is what will cause the thread count to increase (this should be properly optimized) Interface won't be as bad unless it's handling multiple user requests at the same time.

[SissiFeng]

I made a quick change to my previous gradio deployment (not published through HF yet)

[SissiFeng]

Made a few tweaks of "ping-pong" —details are on: https://github.com/SissiFeng/ping-pong