How to make API calls with shinylive?

[JamesHWade]

I'm working on a toy application to show how a shinylive app could make API calls. It's turning out to be harder than I thought. It seems like networking in general does not work very well with pyodide (and thus shinylive). I'm still learning here, but the issue looks to be accessing sockets.

A few posts (e.g., here) suggest that you should be able to do this with javascript. I see the tags reference in support, but I'm a bit lost in how to implement this in practice.

My toy app is calling a published API that was built on the palmerpenguins data set. Here is the code that works as a standalone shiny app but not with shinylive:

from shiny import *
import urllib.request
import json

app_ui = ui.page_fluid(
    ui.layout_sidebar(
    ui.panel_sidebar(
    ui.input_select("species", "Penguin Species",
                    {"Gentoo":"Gentoo", "Chinstrap":"Chinstrap", "Adelie":"Adelie"}),
    ui.input_action_button("go", "Predict", width="100%")),
    ui.panel_main(ui.h2(ui.output_text("txt"))))
)

def server(input, output, session):
    @output
    @render.text
    @reactive.event(input.go)
    def txt():
       url = "http://penguin.eastus.azurecontainer.io:8000/predict"
       payload = [{"species":input. Species(),
                   "bill_length_mm":40.5,
                   "bill_depth_mm":18.9,
                   "flipper_length_mm":180,
                   "body_mass_g":3950}]
       headers = {"Content-Type": "application/json"}
       data = json.dumps(payload).encode("utf-8")
       headers = {k: v.encode("utf-8") for k, v in headers. Items()}
       request = urllib.request.Request(url, data, headers)
       response = urllib.request.urlopen(request)
       data = response. Read()
       response_data = json.loads(data. Decode("utf-8"))
       prediction = response_data[0]
       class_pred = prediction[".pred_class"]
       return f"The {input.species()} 🐧 is predicted to be {class_pred}." 

app = App(app_ui, server)

Is there a known workaround with the python route that I'm unable to find? If not, any suggestions on how to insert a javascript component that can take the dynamic inputs from the UI to be used for the prediction?

[Fedorov-Artem]

urllib.request cannot be used inside the shinylive app, you need to use pyodide.http.pyfetch instead.

They have updated documentation, so there is an answer to your question: https://shinylive.io/py/examples/#fetch-data-from-a-web-api

[palmajoao]

Very newbie to shiny and shinylive and I am testing it to access other output as an http request. In my example, I am trying to access this output: http://www.isa.ulisboa.pt/proj/clipick/climaterequest_fast.php?em=12&sy=2000&tspan=d&fmt=csv&lon=-8.335343513&dts=METO-HC_HadRM3Q0_A1B_HadCM3Q0_DM_25km&ar=4&ey=2003&sm=1&ed=31&lat=39.2816836&mod=hisafe&sd=1

which is a climate (format can be changed in the fmt variable to json or htmltable if needed further testing)

But by replacing line 64 in the example https://shinylive.io/py/examples/#fetch-data-from-a-web-api with this line: return f"http://www.isa.ulisboa.pt/proj/clipick/climaterequest_fast.php?em=12&sy=2000&tspan=d&fmt=json&lon=-8.335343513&dts=METO-HC_HadRM3Q0_A1B_HadCM3Q0_DM_25km&ar=4&ey=2003&sm=1&ed=31&lat=39.2816836&mod=hisafe&sd=1"

I would expect some retrieval. but it throws "Failed to fetch". And the debugger isn't very helpful either.

Any idea? Is there a limit on time/volume of data acquisition?

[wch]

@palmajoao If you open the browser's JavaScript console (right-click, select Inspect, and then click on "Console", you will find some useful information there.

When I try your version of the app, I see this error in the console:

Mixed Content: The page at 'https://shinylive.io/py/shinylive/pyodide-worker.js' was loaded over HTTPS, but requested an insecure resource 'http://www.isa.ulisboa.pt/proj/clipick/climaterequest_fast.php?em=12&sy=2000&tspan=d&fmt=json&lon=-8.335343513&dts=METO-HC_HadRM3Q0_A1B_HadCM3Q0_DM_25km&ar=4&ey=2003&sm=1&ed=31&lat=39.2816836&mod=hisafe&sd=1'. This request has been blocked; the content must be served over HTTPS.

This is because shinylive.io is served over https, but the URL you are fetching is http, and browser security restrictions require a https page to only allow outgoing https requests. (This is mentioned in the comments at the top of app.py in the example.)

After changing the URL to https, I get the following error:

Access to fetch at 'https://www.isa.ulisboa.pt/proj/clipick/climaterequest_fast.php?em=12&sy=2000&tspan=d&fmt=json&lon=-8.335343513&dts=METO-HC_HadRM3Q0_A1B_HadCM3Q0_DM_25km&ar=4&ey=2003&sm=1&ed=31&lat=39.2816836&mod=hisafe&sd=1' from origin 'https://shinylive.io' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

This is also due to browser security restrictions. If a web page uses fetch() to request a resource from a different origin, the browser will normally only provide an opaque response to the JS code on the page. This means that you can't look at the content of the response. Opaque responses aren't useful for most cases, but see https://stackoverflow.com/a/36303436 for what they can be used for.

In order for the page at https://shinylive.io to be able to download data from your URL, your server must provide a HTTP response with the Access-Control-Allow-Origin header which allows shinylive.io to make a cross-origin request to it. See this for more information: https://stackoverflow.com/questions/10636611/how-does-the-access-control-allow-origin-header-work

I think the example app should be updated to provide more information about cross-origin requests.

[palmajoao]

@wch Thanks! I added the header to the script (console is now receiving the header). However, unfortunately, the need for serving the result as httpS might be a limitation from the current servers at the university... So I'll have wait until there is some solution from their end. But thanks anyway (And I guess it will be useful to others trying to serve APIs to shinylive - btw great project!)