Use waitress to display a progress bar from a download

[fBedecarrats]

Hello, This is more a request for a new feature or some guidance in your package documentation. For use cases where shiny apps download heavy files from the internet (e.g. open data), it would be useful to have a simple way to connect to a download function. httr::GET seems a good candidate, as it has offers a complementary httr::progress function to display progress on the console. However, I have been trying to link the output to httr::progess to waiter::waitress, but I could not figure out how to do it. I think that it would be of other users' interest if there would be some binders to your package, or some hints in your documentation on how to achieve that. Kind regards.

[JohnCoene]

Good idea, let me look into this.

[JohnCoene]

I just pushed httr_progress, you can see an example here. Would appreciate it if you could test it.

[fBedecarrats]

Many thanks. I prepared a test app here ; https://github.com/fBedecarrats/test_httr_progress/blob/main/test_httr_progress_shinyapp.R (content below), but it fails with the following error message: _Warning: Error in httr_progress: could not find function "httrprogress" I tried to install the package with both methods commented at the beginning, but I get the same error. httr_progress() is listed on the package help index, but I fail to call it from my code.

## First installation method
# remotes::install_github(repo =  "https://github.com/JohnCoene/waiter")
## Second installation method, after downloading from https://codeload.github.com/JohnCoene/waiter/zip/refs/heads/master
## and unzipping
# devtools::install("waiter-master")
library(shiny)
library(waiter)
library(httr)

datasource <- "https://data.nantesmetropole.fr/explore/dataset/244400404_nombre-convives-jour-cantine-nantes-2011/download/?format=csv&use_labels_for_header=true"

# A UI where we want to see a progress bar on the navbar
ui <- navbarPage("A navbar to visualize progress",
                 tabPanel("Trigger download",
                          useWaitress(),
                          actionButton("dwld", "Download !")
    )
)

# Testing the function
server <- function(input, output) {

    waitress <- Waitress$new("nav", theme = "overlay", min = 0, max = 10)

    observeEvent(input$dwld, {
        my_data <- GET(datasource, httr_progress(waitress))
    })

}

shinyApp(ui = ui, server = server)

[JohnCoene]

Oh goodness, so sorry: I forgot to export this function. It's good now.

[fBedecarrats]

Hello there. Now the function loads correctly but I'm unable to make it work on my shiny app. On the following code, the file downloads correctly, but progress bar appears. I tried changing the theme, moving the max to 1 or 100, and removing min and max, but it doesn't work. When I change the theme for "overlay-percent", the percent label goes above tens of thousand percents.

## Installing from the repo
# remotes::install_github(repo =  "https://github.com/JohnCoene/waiter")
library(shiny)
library(waiter)
library(httr)

datasource <- "https://data.nantesmetropole.fr/explore/dataset/244400404_nombre-convives-jour-cantine-nantes-2011/download/?format=csv&use_labels_for_header=true"

# A UI where we want to see a progress bar on the navbar
ui <- navbarPage("A navbar to visualize progress",
                 tabPanel("Trigger download",
                          useWaitress(color = "#7F7FFF"),
                          actionButton("dwld", "Download !")
    )
)

# Testing the function
server <- function(input, output) {

    waitress <- Waitress$new("nav", theme = "overlay", min = 0, max = 10)

    observeEvent(input$dwld, {
        # A 5.3Mo file
        my_data <- GET(datasource, httr_progress(waitress), 
                       write_disk("my_data.csv", overwrite = TRUE))
    })

}

# Run the application 
shinyApp(ui = ui, server = server)

[fBedecarrats]

I tried to understand what goes wrong, but it remains very mysterious to me. Do different API have different behaviour and some do not send the required information to display a progress bar? Between the two intents below, the only thing that changes is the url: the first one downloads the file AND displays the waitress progress bar, while the second one downloads BUT doesn't display the progress bar... This one works (I added https and removed cap_speed to enhance similarities):

library(httr)
library(shiny)
library(waiter)

ui <- navbarPage("Blobloblo",
  tabPanel("Blablabla",
  useWaitress(),
  actionButton(
    "dl",
    "Download"
  ),
  plotOutput("plot"))
)

server <- function(input, output){
  w <- Waitress$new("nav")

  dataset <- eventReactive(input$dl, {
    x <- GET(
      "http://httpbin.org/bytes/102400", 
      httr_progress(w), write_disk("todelete.txt", overwrite = TRUE) 
    )

    runif(100)
  })

  output$plot <- renderPlot(plot(dataset()))
}

shinyApp(ui, server)

And this one doesn't work:

library(httr)
library(shiny)
library(waiter)

ui <- navbarPage("Blobloblo",
  tabPanel("Blablabla",
  useWaitress(),
  actionButton(
    "dl",
    "Download"
  ),
  plotOutput("plot"))
)

server <- function(input, output){
  w <- Waitress$new("nav")

  dataset <- eventReactive(input$dl, {
    x <- GET(
      "https://data.nantesmetropole.fr/api/v2/catalog/datasets/244400404_nombre-convives-jour-cantine-nantes-2011/exports/csv", 
      httr_progress(w), write_disk("todelete.txt", overwrite = TRUE) 
    )

    runif(100)
  })

  output$plot <- renderPlot(plot(dataset()))
}

shinyApp(ui, server)

[JohnCoene]

Right, thanks for reporting this.

This is because the total amount of data to be downloaded is not known.

https://github.com/r-lib/httr/blob/21ff69f219ad11298854a63b8f753389088cf382/R/progress.R#L40

I'm gonna have to think of a way to sort this out, not sure how I can handle that just yet.