Exception when using mail.build() for personalization that contains JSONArray of JSONObjects

olNYXlo commented 3 years ago

Issue Summary

I have designed a middleware which uses the SendGrid Java package to execute transactional email communication. https://github.com/sendgrid/sendgrid-java Currently I am facing an issue when trying to trigger an API call for a template that has dynamic variables. I have a JSONArray of JSONObject, but am getting an exception when calling the mail.build() method. When calling the mail v3 send API directly on Postman, I am able to send the email. However, when using the Java package, I am getting an exception that there is no serializer for JSONObject.

For the personalization array, I have added the values as shown below. This is the value of the array that causes the issue : { "facilityTypeEN":[{ "facilityName" : "Corporate Vendor Financing (1% - 1.25% IR p.m.)", "percentage" : "100%" }, { "facilityName" : "Invoice Financing AR Disclosed (1.3% - 1.4% IR p.m.)", "percentage" : "100%" }, { "facilityName" : "Working Capital Financing (1.2% - 1.3% IR p.m.)", "percentage" : "100%" }] } I have attached the personalization using the code : personalization.addDynamicTemplateData("facilityTypeEN", facilityTypeEN);

Steps to Reproduce

Create JSONArray of JSONObjects
Attach it to email personalization
Try to build the mail & trigger transactional email sending.

Code Snippet

public Response sendEmail(JSONObject data, List<MultipartFile> files, String params) throws UnirestException, IOException {

// templateVars is a JSONObject that contains all the dynamic keywords & values to be added as personalization

JSONObject content = (JSONObject) data.get("templateVars");

// This will extract every JSONKey from the JSONObject & attach it with the corresponding value
// The generic content.get() method is used to cater to all data types so that the value will be attached as an Object.
for(String key : content.keySet()) {
personalization.addDynamicTemplateData(key, content.get(key));
}

mail.addPersonalization(personalization);

// All other mail creation logic has been omitted as is not relevant to this issue (Sender, Recipient, API key, etc)
Response response = new Response();
request.setMethod(Method.POST);
request.setEndpoint("mail/send");
// This is the root cause of the exception.
request.setBody(mail.build());
response = sg.api(request);

}

Exception/Log

`No serializer found for class org.json.JSONObject and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: com.sendgrid.helpers.mail.Mail["personalizations"]->java.util.ArrayList[0]->com.sendgrid.helpers.mail.objects.Personalization["dynamic_template_data"]->java.util.TreeMap["facilityTypeEN"])`

Technical details:

sendgrid-java version: sendgrid-java-4.7.2.jar
java version: 8

thinkingserious commented 3 years ago

Hello @olNYXlo,

While I try to reproduce the issue, could you please take a look at this example? Thank you!

With best regards,

Elmer

olNYXlo commented 3 years ago

Thanks for the info. I was able to trigger email sending for my other templates which had string dynamic data. But for this particular template where I require a JSONArray of JSONObjects to be displayed on the email, the mail is not able to be built with the helper class. The Mail helper class is able to attach dynamic data of JSONArray of String. I have tested once for this as well and the email got sent. "facilityTypeEN":[ "Corporate Vendor Financing (1% - 1.25% IR p.m.) (30% Max exposure)", "Corporate Vendor Financing (1% - 1.25% IR p.m.) (50% Max exposure)"], As in my email template I am using Handlebars for Iteration, I would need a JSONArray of JSONObjects to properly display the content for each row / list item.

If I directly call the API on postman, I am able to send the email. https://api.sendgrid.com/v3/mail/send

JSON Request Body { "template_id": "<template-id>", "from": { "email": "<sender>" }, "personalizations": [ { "to": [ { "email": "<recipient>" } ], "dynamic_template_data": { "facilityTypeEN": [ { "facilityName": "Corporate Vendor Financing (1% - 1.25% IR p.m.)", "percentage": "100%" }, { "facilityName": "Invoice Financing AR Disclosed (1.3% - 1.4% IR p.m.)", "percentage": "100%" }, { "facilityName": "Working Capital Financing (1.2% - 1.3% IR p.m.)", "percentage": "100%" } ] } } ] }

thinkingserious commented 3 years ago

Hello @olNYXlo,

Have you tried converting the JSONObjects to string (with .toString();?

olNYXlo commented 3 years ago

Nope I did not try that. So are you suggesting I convert the JSONArray of JSONObjects into a JSONArray of String?

Meaning I convert the individual JSONObjects inside the of facilityTypeEN only? Or do you mean I convert the entire dynamic_template_data to String?

Let me try both and get back to you again. Not sure how converting it to string would work though. As the handlebar iteration requires a List of elements for it to work properly. If i have a JSONArray of String not sure if it will work properly.

thinkingserious commented 3 years ago

Hi @olNYXlo,

Please let us know how your tests go, thanks! Also, I found this possible solution.

olNYXlo commented 3 years ago

I have tried JSONObject.toString() for each of the JSONObject in the array. "facilityTypeVI" : ["{\"percentage\":\"100%\",\"facilityName\":\"Tài trợ theo hoá đơn (lãi suất từ 1.3% - 1.4% hàng tháng)\"}","{\"percentage\":\"100%\",\"facilityName\":\"Tài trợ theo hoá đơn, bên mua là đối tác của Validus (lãi suất hàng tháng từ 1% - 1.25%)\"}","{\"percentage\":\"100%\",\"facilityName\":\"Tài trợ vốn lưu động (lãi suất từ 1.2% - 1.3% hàng tháng)\"}"]

My handlebar declared in the template is `

{{this.facilityName}} ( {{this.percentage}} )

              {{/each}}
            </ul>`

When I triggered the email sending, this portion of the email was coming as empty brackets ( ). Hence, I am assuming that SendGrid end received the array of string & did not identify it as a JSONObject.

The issue I got was coming from the mail.build() method in the package. This method was unable to build the Request body properly for an Array of JSONObjects. public String build() throws IOException { try { //ObjectMapper mapper = new ObjectMapper(); return SORTED_MAPPER.writeValueAsString(this); } catch (IOException ex) { throw ex; } }

I am using the SendGrid helper Request & SendGrid helper Response to call the API only. Hence, I used mail.build() instead of manually creating a request body.

olNYXlo commented 3 years ago

hi @thinkingserious , Is there any plans to fix the JAVA helper package? Or should I resort to manually executing a HTTP call out to sendgrid instead of using the helper package

shwetha-manvinkurke commented 3 years ago

Hi @olNYXlo little confused. You say that you have done personalization.addDynamicTemplateData("facilityTypeEN", facilityTypeEN); to attach the json object but the code says otherwise.

Here's my sample code if it helps:

Mail mail = new Mail();

mail.setFrom(new Email("me@gmail.com"));
mail.setTemplateId("d-34a5155225274c9cac6c7ce6bfcc16ee");

final Map<String, String> jsonObject = new HashMap<>();
jsonObject.put("facilityName", "Corporate Vendor Financing (1% - 1.25% IR p.m.)");
jsonObject.put("percentage", "100");

final List<Map<String, String>> jsonObjectList = new LinkedList<>();
jsonObjectList.add(jsonObject);

Personalization personalization = new Personalization();
personalization.addDynamicTemplateData("facilityTypeEN", jsonObjectList);
personalization.addTo(new Email("you@gmail.com"));

mail.addPersonalization(personalization);
SendGrid sg = new SendGrid(System.getenv("SENDGRID_API_KEY"));
Request request = new Request();
request.setMethod(Method.POST);
request.setEndpoint("mail/send");
request.setBody(mail.build());

System.out.println(request.getBody());

Response response = sg.api(request);

olNYXlo commented 3 years ago

Hi @shwetha-manvinkurke ,

I see, in my code I am using `// templateVars is a JSONObject that contains all the dynamic keywords & values to be added as personalization

JSONObject content = (JSONObject) data.get("templateVars");

// This will extract every JSONKey from the JSONObject & attach it with the corresponding value // The generic content.get() method is used to cater to all data types so that the value will be attached as an Object.

for(String key : content.keySet()) { personalization.addDynamicTemplateData(key, content.get(key)); } `

In which case one of the key values will be a JSONArray of JSONObjects. Could the reason of my failure be because I am using JSONArray instead of the LinkedList in your code snippet? As this method is run based on an API call, when the code uses content.get(key), it will return a JSONArray of JSONObject instead of manually creating the LinkedList. Let me try this fix.

thinkingserious commented 3 years ago

Hello @olNYXlo,

Just checking in to see if you had a chance to try out the fix. Thanks!

With best regards,

Elmer

olNYXlo commented 3 years ago

Hi @thinkingserious ,

Sorry have been preoccupied with other tasks. Have not been able to try out the fix, but I had done a simple hotfix to create the request body manually instead of relying on the helper package. So my code will create the request body manually but use the helper package to trigger the API. So no issues currently for this issue. Thanks for the prompt response & help everyone! :)

sendgrid / sendgrid-java