Upon deploying 2.0.6, email attachments via REDCap variable stop working

[jasonlones]

Observed, after deployment of 2.0.6, the module stops attaching PDFs designated via REDCap variable to emails. This was for existing alerts put in place during a previous version (likely 2.0.1). Observed in both longitudinal and non-longitudinal projects. No work-around discovered causing us to roll-back to 2.0.1.

RC Version 8.5.11

[knil-maloon]

@jasonlones I just made a test with the current version 2.0.6 on a simple non-longitudinal and on a repeatable instrument project and, on both cases, it sent me the file I added into the record. Can you give more details about the issue? It seems to be working for me.

[jasonlones]

@knil-maloon We had a production project using Email Alerts 2.0.1 successfully attaching PDFs from a REDCap field. I enabled 2.0.6 and the same alerts no longer attached the PDFs. I rolled back to 2.0.1 and they started working again.

The project uses the 'Save Survey to PDF' module to make a PDF of a consent form and put it to a File Upload field on a separate form. The Email Alerts module then attaches that PDF to the email. Hope this helps.

[knil-maloon]

@jasonlones we will check it out. It's weird because that functionality hasn't been modified since it was created. It feels more as if the delay functionality that ensures any other modules run before the email is not working properly. Do those automated pdfs have a lot of data in it?

[jasonlones]

@knil-maloon, the PDFs are rather small - around 40kb.

[jasonlones]

Issue remains with 2.0.7.

[knil-maloon]

Hi @jasonlones We've been testing different versions with PDFs heavier than 40kb but the email always sends them. We've tested with the newest 2.0.7 and 2.0.5 and both work. We are unable to replicate the issue.

[alewis64]

Jason Can you send us the data dictionary and PDF you are trying to attach. Eva and I are both having a hard time replicating this issue. Other then the PDF and Email modules is there any other code running on these projects (module, hooks, plugins)

[jasonlones]

Sure thing. There are no hooks or plugins running specific to this project. Here are the modules running on this project:

Admin Dashboard v.2.4.1 Codebook Concertina v1.0.1 Email Alerts v2.0.7 Instance Marker v1.7 Save Survey PDF to a Field v1.1.2 Shazam v1.1.9

ExModsEmailAlerts_DataDictionary_2018-09-25.zip

[jasonlones]

For clarification, we are using 'Save Survey PDF' to package up the form which triggers the email alert to be attached to the email. Those are being stored on a separate form. This workflow works with 2.0.1. Thanks for your help!

[knil-maloon]

Hi @jasonlones We looked into it and it seems that PDF module runs on the survey_complete while the email module does it on save. Save is called before survey_complete so the PDF never gets generated before the email sends. The reason why that older version works is because in older versions the email was using survey_complete function as well but due to some bugs we took it out. We're going to look into it and try to add a fix into the new release so that modules that use the survey_complete are compatible with the email.

[jasonlones]

@knil-maloon Thank you!

[knil-maloon]

Hi @jasonlones Monday we will make available a new version that should fix this. As soon as Rob approves it it should be visible.

[jasonlones]

@knil-maloon, thank you! I'll be looking for it.

[jasonlones]

@knil-maloon, @alewis64, unfortunately, 2.0.8 still has the same issue (not attaching PDFs on survey complete).

[knil-maloon]

Hi @jasonlones It's hard to recreate the issue as it is working for us. We are running REDCap version 8.8.1, Email module 2.0.8, and Save Survey PDF to a field v1.1.2. Can you try to get this to work using these specifications and no other modules? It could be that one of the other modules you have in the project is affecting things.

[jasonlones]

I can't get to 8.8.1 as we are on LTS, but I'll strip all of the other modules and focus on 2.0.8 for EA and 1.1.2 for Save Survey.

[jasonlones]

@knil-maloon, I removed all of the other modules and still have the issue. Is there anyway to delay the email on my side?

[knil-maloon]

@jasonlones the email alert already has a delay in its code to ensure it goes last so I don't think there's anything else we can add. It's weird that it still does not send the file as now if you call it from a survey it should go to survey complete and be sent after the pdf.

[jasonlones]

@knil-maloon, it is so strange. Even my log has the reverse order.

[alewis64]

@jasonlones @knil-maloon That indeed is weird here is what my log looks like.

[jasonlones]

@alewis64 @knil-maloon , just to confirm the log order, I totally replicated my scenario in a new project and it is still the reverse order from you guys. Just brainstormin here, is it possible the module version which made it to the repo is still the old version? Seems unlikely, but I thought I'd ask.

[knil-maloon]

@jasonlones I double checked but the version I submitted is correct. It's the same one we are using.

[jasonlones]

@knil-maloon Thanks, Eva. I figured. I'm just grasping at straws now. This is a big need for many of our projects and the pressure mounting. Hopefully I can find a solution. Thanks for all your work!

[alewis64]

@jasonlones are you able to download another version of REDCap onto a local environment and test the functionality on a different version? Or are you able to revert the versions of the modules back to when they did work?

[jasonlones]

@alewis64, we are currently using 2.0.1 in our PROD environment, which is working for the attachments. However, that version has issue regarding the 'Surverylink' piece which have been fixed in the newer versions of the module. When we rolled to 2.0.6, that is when I first had problems with the attachments, so we rolled back to 2.0.1. Just this morning I found a work-around using Smart Variables for the 'Surveylink' piece. I think the current plan is to stay on 2.0.1 until I can find time to test in a different RC version. Unfortunately, we will be on 8.5.x for awhile as Duke stays on the LTS branch and our validation cycles are often long and slow.

[jasonlones]

@alewis64, do you think it would be worthwhile to ask Rob what may be causing the difference in processing these modules in the two RC versions? And maybe what version where the change could have occurred?

[alewis64]

@jasonlones I don't think it would hurt. I am a little surprised they are not the same. I know he is on vacation this week so it might be a few days before you hear back from him.

[jasonlones]

@alewis64, do you know where the delayModuleExecution function is in the EM framework?

[alewis64]

Hey @jasonlones I am not 100% sure as to its exact location. @mmcev106 should be able to give you a better explanation then I.

[jasonlones]

@alewis64 Thanks, Adam!

[mmcev106]

@jasonlones, it is just another method on the AbstractExternalModule class. It's in the docs along with all the other module methods.

[jasonlones]

Thanks @mmcev106. I appreciate your quick reply! I guess I'm trying to understand how the method works. It looks like to my very novice eye that the function simply calls itself. Is there something besides lines 707-717 in the AbstractExternalModule class which creates the method? Please forgive my ignorance.

[mmcev106]

@jasonlones, the AbstractExternalModule::delayModuleExecution() method calls the ExternalModules::delayModuleExecution() function (on a different class).

[jasonlones]

@mmcev106, that is what I thought. Thanks. I'll look for the function in the other class. I think we are experiencing a PHP version specific issue, but I'm hoping to find out. For some reason, the delay is not working for us (RC 8.5.11 / PHP 7.1.8) in the Email Alerts module (Save Survey to PDF fires after the Email Alert already goes out).