Timeout for downloading a file from Shiny using downloadHandler

[daattali]

This originally came up on the Shiny google forum

It seems that when the downloadHandler takes a bit of time serving the wanted downloadable file, we get an error. It'd be nice to be able to change that timeout ourselves (if that's indeed what's happening).

I've seen this error come up on StackOverflow as well and it's affected me too

[keithmcnulty]

I am experiencing this problem. Its really restrictive bacause it prevents you offering any sort of substantial r markdown batch report generation. I get a network timeout if the process of generating the reports takes longer than 60 secs. Really need a fix or a way to config this.

[jturchetta]

I'm getting the same error on CentOs 6+ platform. I believe @jcheng5 add a directive called http_keepalive_timeout which allows you to specify a longer time frame than 45 seconds. My problem is that this directive seems to have been added to version 1.5.2 which I'm not sure if the pre-built shiny-server download is available yet. Just checking to see if this is still the case or there was a way to get 1.5.2 on a Centos 6+ platform yet?

[keithmcnulty]

I ended up solving this issue by separating the rmarkdown process from the download. I initiated the rmarkdown file creation within shiny and told the system to only execute the downloadHandler command when shiny was no longer busy. Happy to share the code if it helps. Risky if you have multiple simultaneous app users.

[jturchetta]

I'd love to see you're code if it's not too much! I'm actually using the ReporteRs package to initiate the download of a powerpoint and word document but I think I see what you're implying. Creating the doc in the server than just writing it in the downloadHandler?

[heejongkim]

I'm experiencing very similar issue. When I tried the downloadHandler in the local machine, it just works but nothing gets downloaded in shiny server. I would be more than happy if I can get a chance to see your code. Thanks!! @keithmcnulty

[jturchetta]

Here is some code I figured out below. The basic premise is that you pass the document created using ReporteRs or rmarkdown to an object than use the download handler to download that document. It does require and additional button to build that content which is annoying but at least it works:

D <- reactiveValues(documents = NULL)
  observeEvent(input$go, {
    # Create a Progress object
    progress <- shiny::Progress$new()
    # Make sure it closes when we exit this reactive, even if there's an error
    on.exit(progress$close())
    progress$set(message = "Building Content...Please Wait", value = 0)

    doc <- pptx(template = '/insertfolderlocation/ PPT Template.pptx' )
    docx <- docx(template = '/insertfolderlocation/Word Template.docx')
##Insert code to build document####

###Create list of reactive object
  D$documents <- list(doc, docx)
  })

 ####################ReporteRs Download Handler Powerpoint#############################################################
  output$downloadPPT <- downloadHandler(
    filename = "Network Analysis.pptx",
    content = function(file) {
      # temporarily switch to the temp dir, in case you do not have write
      # permission to the current working directory
      owd <- setwd(tempdir())
      on.exit(setwd(owd))
      doc <- D$documents[[1]]
      validate(
        need(!is.null(doc), "You have to press Build Network Content First and let it run, usually takes 1-2 minutes.")
      )
      writeDoc(doc,file)
    }
  )
  ###################ReporteRs Download Handler Word####################################################################
  output$downloadWord <-  downloadHandler(
    filename = "Network Analysis.docx",
    content = function(file) {
      # temporarily switch to the temp dir, in case you do not have write
      # permission to the current working directory
      owd <- setwd(tempdir())
      on.exit(setwd(owd))
      doc <- D$documents[[2]]
      validate(
        need(!is.null(doc), "You have to press Build Network Content First and let it run, usually takes 1-2 minutes.")
      )
      writeDoc(doc,file)
    })

@heejongkim

[heejongkim]

Awesome, @jturchetta. From your code, I got enough inspiration to apply it onto my shiny app. From your code, I used withProgress and outputOption to simplify the code and show the download button after the report is built.

[jturchetta]

That's amazing @heejongkim ! Can you share a little of you're code for how that outputOption and withProgress work towards that effect? This would really make my code more streamlined for users!

[heejongkim]

@jturchetta Absolutely. I don't know how I missed this. Here's the gist: in ui part, conditionalPanel(condition = "output.reportbuilt", br(), downloadButton("excel_report_download", "Download"))

in server part, output$reportbuilt <-reactive({ return(!is.null(exce_report_holder$filepath)) }) outputOptions(output, 'reportbuilt', suspendWhenHidden=FALSE)

So, as the report gets ready, exce_report_holder$filepath will get the value. You might not need outputOptions but I haven't tested without it. I'm sure there might be a better way to handle this more elegantly but, at least, I hope this is helpful for you to get there.

[cosmin-novac]

Adding http_keepalive_timeout 600; to the top level in the shiny-server.conf file solved the timeout issue for me.

[gtumuluri]

Adding http_keepalive_timeout 600; to the top level in the shiny-server.conf file solved the timeout issue for me.

This setting did not solve my problem. Is there something more than adding the line and restarting the shiny server? Any other options?

[jcheng5]

@gtumuluri If you have a proxy server or load balancer sitting between the client and shiny-server, make sure that it doesn't impose its own HTTP timeout.

[gtumuluri]

@gtumuluri If you have a proxy server or load balancer sitting between the client and shiny-server, make sure that it doesn't impose its own HTTP timeout.

I bet that is the exact problem in my case. I was just literally thinking that the AWS load balancer we use must be the problem. Sadly, I have no control over it. Need to do some other 'shiny-centered' way.

[judytlewis]

@gtumuluri I had the very same issue but was able to fix it by going to the AWS EC2 console, selecting Load Balancer from the menu on the side, scrolling down and editing the Attributes to increase the Idle timeout value.

[tylerlittlefield]

I can't thank @jcheng5 enough for the suggestion on HTTP timeouts being imposed by something other than shiny-server. I am running shiny server on NGINX and added the following to my nginx.conf file:

proxy_read_timeout 20d;
proxy_buffering off;

It seems that this may have resolved the issue for me. I originally updated http_keepalive_timeout in shiny-server's configuration file but was still having an issue. I just made the change and tested once, so time will tell if it officially solved things but so far it's looking good.

[Drwhit]

Thanks @tyluRp for those parameters, I have been playing with send_timeout, etc, but hasn't fixed the problem for me. The parameters you mentions aren't doing anything either. In fact, no parameters seem to have any effect. Are you using https or just http? My site is https, would I need to put the params somewhere other than the nginx http block?

We are also running shinyproxy from inside docker, does anyone know if that could influence why the nginx parameters don't seem to be fixing the problem for us??

[gexijin]

@jturchetta's solution worked for me. Thanks! None of the other tricks did. I am using Nginx load balancer and using https. The http_keepalive_timeout makes the app totally unavailable.

One thing I improved (?) upon @jturchetta's code is to still use one button on the main UI, but a pop up shows up when the file is ready, which includes a download button at the same time. This is the code for rendering a R Markdown file that takes a long time. This code is used by https://RTutor.ai.

  # Code on the UI ------------------------ 
   actionButton(
       inputId = "render_report",
       label = "Render Report"
   )

  # Code for the server side ---------------
  eda_file <- reactiveVal(NULL)

  observeEvent(input$render_report, {

    withProgress(message = "Generating Report (5 minutes)", {
      incProgress(0.2)
      tempReport <- file.path(tempdir(), "EDA.Rmd")
      tempReport <- gsub("\\", "/", tempReport, fixed = TRUE)
      output_file <- gsub("Rmd$", "html", tempReport)

      file.copy(from = "app/www/eda.Rmd", to = tempReport, overwrite = TRUE)

     # passing data to R Markdown
      params <- list(
        df = df,
        target = eda_target_variable
      )
      req(params)

      tryCatch({
        rmarkdown::render(
          input = tempReport, # markdown_location,
          output_file = output_file,
          params = params,
          envir = new.env(parent = globalenv())
        )
      }, 
        error = function(e) {
          showNotification(
            ui = paste("Error when generating the report. Please try again."),
            id = "eda_report_error",
            duration = 5,
            type = "error"
          )
      },
        finally = {
          eda_file(output_file) # update file name.
          showModal(modalDialog(
            title = "Successfully rendered the report!",
            downloadButton(
              outputId = "download_report",
              label = "Download"
            ),
            easyClose = TRUE
          ))
        }
      )
    })
  })

  output$download_report <- downloadHandler(
    filename = "EDA.html",
    content = function(file) {
      validate( need(!is.null(eda_file()), "File not found."))
      file.copy(from = eda_file(), to = file, overwrite = TRUE)
    }
  )