Unable to find destination data template

[mcgrievm]

Not sure if you're still actively maintaining this project, but I'm getting the below error when trying to transfer a project. Note that I previously posted a bug report as "senorrojo" a few years ago (which you solved - thank you) and I've just started using this package again because we're migrating from one REDCap server to another. Thank you so much for creating this project - it truly saved me the last time I needed it and I had many, many successful transfers of large projects.

I get the same error for transferring a single record or all records:

[matt@localhost redcap-api-transfer]# sh transfer_single_record.sh Test 25 /app/library/export_data.rb:50:in throw': uncaught throw "Unable to find a destination data template file for the project /app/export_data/dest_template-1708116087.tmp." (UncaughtThrowError) from /app/library/export_data.rb:50:infetch_dest_data_template' from /app/library/export_data.rb:43:in dest_data_cols' from /app/library/export_data.rb:15:ininitialize' from /app/library/transfer_records.rb:50:in new' from /app/library/transfer_records.rb:50:ininitialize' from transfer_single_record.rb:7:in new' from transfer_single_record.rb:7:in\<main>' Loading Configuration ... Downloading SOURCE Project Template ... Downloading DESTINATION Project Template ... Downloading Data Dictionary ... Loading Export Data ... [matt@localhost redcap-api-transfer]#

[matt@localhost redcap-api-transfer]# ls app/export_data/ data.csv

data.csv has a CSV export of the project data from the SOURCE. I also tried putting the data dictionary in that export_data directory to no avail. Below is my config file. I'm happy to provide verbose logging or any other information that might be helpful.

settings:
  processes: 8 # Default number of processes the script will fallback to if none is specified in specific projects below
  verbose: false # When true, outputs additional information to logs about the HTTP requests being made

projects:
  Test:
    source:
      url: https://redcap.healthsciencessc.org/api/
      token: XXXX

    destination:
      url: https://redcap.research.sc.edu/api/
      token: YYYY

    transfer_new_records_only: true # Must have API Export Rights on Destination REDCap Server!
    processes: 8 # Takes precedence over the processes listed in settings
    verbose: false  # Takes precedence over the verbose flag listed in settings

Again, thank you!

[mcgrievm]

I messed around with this a bunch more, I think changing transfer_new_records_only in config.yml from "true" to "false" has solved it. Not sure why that is, but at this point, I'm happy to have it working.

Also, once I was able to get it to work, I got an SSL error from curl: E, [2024-02-18T16:19:22.570735 #1] ERROR -- : Failure for 25 on destination. Possible reason: E, [2024-02-18T16:19:22.570774 #1] ERROR -- : Error: [Curl::Err::SSLPeerCertificateError, "SSL peer certificate or SSH remote key was not OK"]

To fix that, I downloaded the latest cacert.pem from curl.se (https://curl.se/docs/caextract.html), added to the root redcap-api-transfer directory, then switched this line in app/Dockerfile from

RUN /bin/bash -c "curl-7.71.1/./configure --without-nss --with-openssl" to RUN /bin/bash -c "curl-7.71.1/./configure --without-nss --with-openssl --with-ca-bundle=/cacert.pem"

Transferring by single records or all records now work.

[mcgrievm]

I'm sorry to keep commenting on my own issue, but I realized today that the SSL error was the result of a misconfigured intermediate certificate on our REDCap server. So if anyone runs into the SSL issue, and you don't have control to change things on the REDCap server, the Dockerfile fix should work.

Also, as an update, I transferred a few dozen large/complex projects (some with thousands of fields and records) using the script. Other than not being able to use the transfer_new_records_only config option, everything else worked beautifully for single and all record transfers. Once again, a total lifesaver!