API Keys start returning 500 if the user hasn't opened the project in 72 hours

[dwhend]

REDCap version 10.3.3 (saw it in 10.1.1 as well)

I'm a Database Administrator for the University of Kentucky, and I'm at my wits end. I can't get any traction on other channels so I apologize if this question is outside of the scope of this wiki, but I'm hoping someone out there knows something about this issue.

As the title suggests, we are using the API to import/export records in several automation workflows. These are projects that are intentionally left alone for periods of time while data is collected, so user interaction is minimal. We noticed the ETL's began failing and upon inspecting the logs, we captured the HTTP 500 responses from the server. We ended up fiddling with the user rights menu to confirm nothing had happened to permissions, and the API calls started returning 200 again. Several days later the keys started failing again. Over the course of a month we have established the 72 hour pattern of errors, and we have established that the user that owns the api token merely needs to open the project home page for the key to start working again. We've also been able to predict when the keys will start failing by pulling the max(ts) out of redcap.redcap_log_view for the project_id and user_id where the session_id is not null (a non-API type interaction with the project).

We cannot find anywhere in the documentation that confirms that API keys require user activity within the past 3 days for calls to work, nor can we find any configuration option in the menus to relax this security feature if it does indeed exist. When we initially developed the projects relying on these ETL's we were on REDCap 9.8.7, and these keys worked without fail for months without issue or interaction.

Does anyone have any information?

[nutterb]

This does not seem normal to me. I don't know that there is much to be done from the R side. You may need to raise an issue with the REDCap Consortium. Unfortunately, I no longer have a position with reliable REDCap access, so I can't facilitate this with you.

@wibeasley, is this something you might have some insight with?

Does the 72 hour lock out require a UI connection? I'm wondering if scheduling a job to use the API to grab, say, the Users table would be recognized as opening the project to keep it open. If so, a daily job to download a small table would be a low resource way to keep the project open for your automated processes until a permanent solution is identified.

[wibeasley]

dunno, that's weird. I haven't heard of it, nor do I see any hits on the community site/forums.

I suggest talking to your site's REDCap admin to post the question in the forums.

@nutterb, and I, and others wrote this document to help these scenarios. When you guys post the question, I'd remove R from the equation so more people can get involved. https://ouhscbbmc.github.io/REDCapR/articles/TroubleshootingApiCalls.html#api-1

[dwhend]

Thank you both for your quick replies. I appreciate your time.

We're currently pulling the metadata for the 6 projects every 10 minutes as a "ping" and if the response returns 500, it sends me an email to let me know the API keys have started failing. This is how we nailed down the 72 hour pattern. To be honest, we're not even using R. We're issuing the calls with a few lines Powershell to either extract data to csv, or import new records into the target projects from csv. My issue doesn't fit very well on your github, but it's very hard to find anyone outside the community site discussing REDCap in any technical depth. I will try to get our issue posted to the main forums.

Thanks again.

[dwhend]

I want to add one final comment to this issue just in case someone else out there stumbles across it and is having the same problem. We seem to have solved this by added an entry in the HOSTS file on our webserver pointing to the IP address that the website was bound to. In most setups this is probably not required. We have our site bound to a second IP address on the host machine sitting in our DMZ, not the primary IP address for the machine. We realized that other features like the API playground, and other AJAX calls in REDCap were failing, so we did some basic network testing and realized that traceroutes from the web host to its address were not resolving as a self-hop, but instead taking 20+ hops outside to the public gateways, and then timing out. This was a big red flag. It was a 2 second fix once the line was added to the HOSTS file. All AJAX calls began working, the API playground was functional, and API keys across projects were working without issue.