Deployment and speed issues caused by database connection

[willking98]

Bug description

Two issues here.

1. I have been unable to deploy surveys while the database is connected. It is fine when ignore = TRUE but when I am connected to supabase and try to deploy the survey to shinyapps.io I get this error page.

   

2. When I connect to the database and run the survey locally the speed of the show_if conditions decrease significantly. Roughly 5 seconds to render a conditional question when connected to the database. This is fine when I am not connected to the database and conditional questions are rendered immediately. NB: I have tried this on a simple survey without much additional code and this was still slow when db connected.

Steps to reproduce

This repo should be able to reproduce this behaviour.

Expected behavior

Deployment success and quick rendering of conditional questions.

Actual behavior

Deployment failure and 5 second rendering of conditional questions.

Your environment

No response

[jhelvy]

I can't try this at the moment, but I just looked at your repo and noticed that for some reason the yaml header in your survey.qmd file is inside a call-out block? It should just be a standard quarto yaml header between the --- marks. Can you check to make sure you have that fixed and then update whether either or both of these issues persist?

[willking98]

I've just removed the callout block (rookie mistake!).

I can confirm that both issues still persist after the removal of the callout.

[jhelvy]

Okay I just tested with the much smaller show_if demo, and I was able to deploy my app connected to a supabase database to shinyapps.io. So if you are having a deployment issue, I am guessing there is something in your survey causing an error. One thing to be careful of in deployment is to make sure you restart your RStudio (and sometimes your computer) as there can sometimes be cached elements that cause issues. I had to restart my mac just now and then it deployed.

As for the speed on show_if, I think the reason it's only happening when you have the database connected is because the size of your survey is relatively large. With so many questions and values being stored, the write operation to the database is taking a bit longer, and that operation has to complete before the show_if logic gets triggered. We could probably address this by simply changing the logic to first process the show_if conditions before sending a write operation, so that is what I'll look to do next. Hopefully that just works. If not, then we can look for more opportunities for performance engineering to speed up the data writing process.

[willking98]

I have done a full restart and tried this on multiple PCs. Weirdly I am not experiencing any errors when I deploy this survey with the database not connected ignore = TRUE. If you get the chance please could you try to deploy this app with the database connected and see whether you are successful? I will have a go at reducing the number of show_if conditionals and see if this affects the deployment.

The speed issue is probably of secondary priority as currently I am still unable to get the survey online and collect data.

[jhelvy]

Just merged some edits and bumped to v0.3.1. Can you reinstall and see if this makes things run faster?

[willking98]

I've just updated to the new version and there is no noticable change on my end. The speed is probably not a big issue (at least in my case) as the longer you spend on the page (reading the options) the faster the rendering of the required conditional question will be when the correct answer is selected. I have completed the survey reading through the options slowly and the speed is acceptable.

I am still encountering errors when deploying to shinyapps.io when the database is connected. No issues are presented in the R Console during the deployment process but when I load up the survey url I get the same exit status 1 message.

I also periodically get this poolCheckout() error when I close the app locally. Not sure if this is related.

[jhelvy]

Darn, I saw some pretty snappy speeds on my end when I used your demo. It was noticeably faster after the changes I made compared to before. I think once we get cookies working we'll be able to make this far more efficient. Right now when you refresh your browser, the application re-launches from the start. We're going to implement a way to store a user's responses in the cookies so that the users state can be restored on refreshing. This will give us a chance to also speed things up as we can use cookies for temporary storage and then do batch writes to the database (probably on each page turn instead of on each question, which is how we have it now).

As for the shinyapps.io issue, could you perhaps open a separate issue for that and provide a link to a demo repo where I can reproduce it? It is either an error in your survey (unlikely since you're able to run it locally) or perhaps just a setting on your machine. I would also recommend trying to clone and deploy one of the demos because we know there are no bugs in them. If you get an error with one of them, then it's a setting on your machine.

[willking98]

Sorry for the delay in getting back to you here. I have managed to fix my deployment issues by going back to the templates and more or less starting again with lots of restarting R and my computer. Not quite sure exactly what the problem was - but I have a suspicion that it was related to use_html = TRUE not being enabled and trying to render the long complicated .qmd each time.

Speed wise there has definitely been a marked improvement since the last update. Cheers for that!

[jhelvy]

Nice! Yes we've gone back and forth about whether the default setting for use_html should be TRUE or FALSE. For now we're still going with FALSE because we want to make sure the changes anyone makes in their survey.qmd file are used when the app runs. You have to be more intentional to manually set it to TRUE. Glad things seem to be moving a little faster now.

[jhelvy]

I'm going to go ahead and close this as the speed seems pretty good now in the latest version.