gradio-app / gradio

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
http://www.gradio.app
Apache License 2.0
33.39k stars 2.52k forks source link

Convert HTTP Calls to Async and make Gradio faster #2052

Closed omerXfaruq closed 1 year ago

omerXfaruq commented 2 years ago

Is your feature request related to a problem? Please describe.
There several http calls in Gradio that could be more performant than sync calls. Though converting them to async requires some redesign and planning.

  1. Calls inside launch. If we run these calls in the background we won't need to wait for these calls, and make launch faster.
  2. load related calls. If we migrate these calls to background, they will be able to work in concurrent manner with other load calls and launch.

Problem: Since we don't want to user-API functions to be async, this limits our options to use async functions in launch.

Describe the solution you'd like

  1. We can move these calls to routes.startup-events.
  2. We can add startup-events to Blocks class, and run these calls in routes.startup-events. Also put an custom error or warning to these components that will output 'The model/space is not finished downloading yet, %xx'.

Additional context
Converting any requests usage in gradio to Request custom class would be great overall.

omerXfaruq commented 2 years ago

cc: @aliabid94 @abidlabs @freddyaboulton

hibestil commented 2 years ago

@FarukOzderim In addition to your findings, I wanted to search requests usage in the code base. I saw lots of usages, and listed all functions(and their belonging files) below (except the tests). Since we've been planning to use the Request class, we have to convert those functions (and the functions which use them) to async ones. That'll push us to make a lot of effort :S

To simplify this work, I think we can start with networking.py functions because they are easier to convert at first look. Then we can extend our scope to convert all requests usages to Requestones. While we doing this step by step if we notice a missing feature, we can improve the Request class by implementing missing functionalities.

WDYT?

And I wonder will the current design support async functions if we convert existing sync functions to async?

- gradio/utils.py
    - version_check
    - get_local_ip_address
    - initiated_analytics
    - launch_analytics
    - integration_analytics
    - error_analytics
    - readme_to_html

- external.py
    - get_models_interface
    - encode_to_base64
    - query_huggingface_api
    - get_spaces
    - get_spaces_interface
- networking.py
    - setup_tunnel
    - url_ok
- processing_units.py
    - encode_url_or_file_to_base64
- queueing.py
    - queue_thread
- strings.py
    - in file direct usage
- index/__init__.py
    - build