The error message is unclear/misleading when sending transactional email with basic authentication

[farlee2121]

First off, I'd like to say thank you for this wonderful library.

I'm running into an issue with basic authentication though. The credentials I was using with the official sendgrid library (https://github.com/sendgrid/sendgrid-csharp) are throwing the exception

System.AggregateException: '  Content-Length: 0ext/json, application/xml, text/xml, applicatio)'

Inner Exception: SendGridException: Permission denied, wrong credentials

This exception was originally thrown at this call stack:
    StrongGrid.Internal.CheckForSendGridErrors(Pathoschild.Http.Client.IResponse)
    StrongGrid.Utilities.SendGridErrorHandler.OnResponse(Pathoschild.Http.Client.IResponse, bool)
    Pathoschild.Http.Client.Internal.Request.Execute()
    System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
    System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
    StrongGrid.Resources.Mail.SendAsync(System.Collections.Generic.IEnumerable<StrongGrid.Models.MailPersonalization>, string, System.Collections.Generic.IEnumerable<StrongGrid.Models.MailContent>, StrongGrid.Models.MailAddress, StrongGrid.Models.MailAddress, System.Collections.Generic.IEnumerable<StrongGrid.Models.Attachment>, string, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>, System.Collections.Generic.IEnumerable<string>, System.Collections.Generic.IEnumerable<System.Collections.Generic.KeyValuePair<string, string>>, System.DateTime?, string, StrongGrid.Models.UnsubscribeOptions, string, StrongGrid.Models.MailSettings, StrongGrid.Models.TrackingSettings, StrongGrid.Models.MailPriority, System.Threading.CancellationToken)

I've also tried switching to the username = "apikey" and password = "[api key goes here]" format that sendgrid recommends for smtp integration. It produces the same error

I'm using Strong Grid version 0.66.0

Any ideas?

[Jericho]

Glad this library is useful to you.

I have never heard of hardcoding "apikey" as the username and using your apikey as the password. Maybe this is a "feature" I am not aware of?!?!?!?!?!

Anyway, the StrongGrid library offers you two ways to specify credentials: either you provide an API key or you provide a username and password. Something like this:

// Specify your apiKey
var apiKey = "SG.Qe-Pf-xxxxxxxxxxxxxxxxx.yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy";
var strongGridClient = new Client(apiKey);

// or specify username and password
var username = "myuser";
var password = "mypassword";
var strongGridClient = new Client(username, password);

[farlee2121]

This is what sendgrid shows under the integration info section

Right, I was able to authenticate with just api key.

However, when I tried basic auth, I was not able to login. I know the credentials are correct, because I'm migrating from the sendgrid official library and the credentials worked there. To be clear, the credentials are not "apikey"/"actual api key string". That's just another format I tried because sendgrid appears to recommend it.

[Jericho]

The instructions are for integrating via SMTP, no? So not applicable to StrongGrid (as far as I can tell).

Anyway, I don’t really know what to tell you. You provide a valid username/password combination and SendGrid keeps rejecting it?!? That’s strange. Have you tried with an API key?

[farlee2121]

They are the SMTP integration instructions. I wasn't sure how StrongGrid connected under the hood, so I tried it out.

I did try StrongGrid with apikey, and that works. I just thought I should report that the same username and password I used to login with the official Sendgrid library, do not work with StrongGrid.

Maybe I'm missing something silly. If you're tests are all working, then it's probably something weird on my end.

[Jericho]

I just tested with my username and password, worked just fine.

StrongGrid doesn't really do anything special when you specify a username and a password. These two values are simply Base64 encoded and added to the "Authorization" HTTP header of all subsequent API requests. In fact if you use a tool such as Fiddler to intercept all HTTP calls, you will be able to see this encoded value. It should look something like this:

GET https://api.sendgrid.com/v3/scopes HTTP/1.1
Host: api.sendgrid.com
Accept: application/json, text/json, application/xml, text/xml, application/x-www-form-urlencoded
StrongGrid-Diagnostic-Id: 7d01c991fc4d4342a99b113849506274
User-Agent: StrongGrid/DEBUG (+https://github.com/Jericho/StrongGrid)
Authorization: Basic <...removed for security reasons...>

you can copy the encoded authorization value and decode it with the following C# code:

var encodedBasicAuth = "<paste the value that you captured in Fiddler or whatever other tool you use to intercept HTTP calls>";
var decodedBasicAuth = Convert.FromBase64String(encodedBasicAuth);
var basicAuthAsString = Encoding.ASCII.GetString(decodedBasicAuth);
var parts = basicAuthAsString.Split(":");
var username = parts[0];
var password = parts[1];

[farlee2121]

I'll take a look. Based on the malformed json in the error message. It might be an issue the characters in the username/password

[Jericho]

I'm just curious if you figured out the issue?

[farlee2121]

Sorry, I was out of office.

The result is not what I expected. I set up a simple unit test for making sure I was sane and sendgrid was working but strong grid wasn't, minus the credentials, here it is

// email details
MailAddress to = new MailAddress("farlee@niceshoulders.com", null);
MailAddress from = new MailAddress("meow@niceshoulders.com", null);
string subject = "Will it send";
string htmlBody = "<html><body><h1>test test</h1></body></html>";
string textBody = "test test";

//send with old sendgrid api
var credentials = new System.Net.NetworkCredential(username, password);
var transportWeb = new SendGrid.Web(credentials);
var myMessage = new SendGrid.SendGridMessage(new System.Net.Mail.MailAddress(from.Email),
    new[] { new System.Net.Mail.MailAddress(to.Email) }, subject, htmlBody, textBody);

await transportWeb.DeliverAsync(myMessage); // works

//send with strong grid
var emailClient = new StrongGrid.Client(username, password);
await emailClient.Mail.SendToSingleRecipientAsync(to, from, subject: subject,
 htmlContent: htmlBody, textContent: textBody);
 // throws StrongGrid.Utilities.SendGridException : Permission denied, wrong credentials

When snooping with fiddler. Neither of the requests shows an Authorization header. I'll have to dig a bit more to figure out what's going on.

Also, to check if special characters were the issue, I tried

var bytes = Encoding.ASCII.GetBytes(password);
string base64 = Convert.ToBase64String(bytes);
byte[] fromBase64 = Convert.FromBase64String(base64);
string rehydrated = Encoding.ASCII.GetString(fromBase64);
Assert.Equal(password, rehydrated);
// did the same with username

Encoding this way did not reveal any issues.

[Jericho]

I'm really surprised that you don't see the authorization header. A typical request should look like this:

[farlee2121]

Classic. I needed to restart fiddler after enabling HTTPS decryption.

Here is what I get for headers

POST /v3/mail/send HTTP/1.1
Host: api.sendgrid.com
Accept: application/json, text/json, application/xml, text/xml, application/x-www-form-urlencoded
StrongGrid-Diagnostic-Id: 94d05d668b884605a04e2abbd14056d5
User-Agent: StrongGrid/0.66.0 (+https://github.com/Jericho/StrongGrid)
Authorization: Basic [redacted]
Content-Type: application/json; charset=utf-8
Content-Length: 396

When decoded, the authorization token is in the expected form username:password and the username and password are as expected. It is calling to api.sendgrig.com/v3/mail/send

The working sendgrid call, however is calling to api.sendgrid.com/api/mail.send.xml and the credentials are part of the form data, not the header.

[Jericho]

api/mail.send.xml is the end point for SendGrid's v2 API. StrongGrid was designed to work against SendGrid's v3 API. So that explains the difference. Maybe you are using an old version of their nuget package? I'm pretty sure that later version of their package was updated to use the v3 API.

[farlee2121]

You are correct. My next experiment will be to see if the credentials can work with the v3 api.

[Jericho]

I just realized that when testing username+password, I was invoking various endpoints of SendGrid's API (such as create a contact, create a list and a segment, delete a contact, etc) but not specifically mail/send so I just tried that and here's the result:

HTTP/1.1 401 Unauthorized
Server: nginx
Date: Mon, 06 Apr 2020 15:40:54 GMT
Content-Type: application/json
Content-Length: 88
Connection: keep-alive
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html

{"errors":[{"message":"Permission denied, wrong credentials","field":null,"help":null}]}

Could it be that SendGrid is requiring an API token and disallowing username+password when sending an email?

[farlee2121]

It looks like they expect the basic auth in a different format, but it's not super clear https://sendgrid.com/docs/API_Reference/Web_API_v3/How_To_Use_The_Web_API_v3/authentication.html I've tried

• <encoded-username><encoded-password>
• encoded-usernameencoded-password (didn't expect this one to work, but tried it)
• encoded-username:encoded-password
• and what you showed earlier, where the string username:password is all encoded together

The short of this is that api keys should probably always be used for sending email, but it would still be nice to have an answer to what sendgrid is expecting

[farlee2121]

Maybe it needs some combination of both url encoded and base64 encoded?

[Jericho]

What's confusing is that encoding the username and password like I showed you previously works perfectly fine for all other SendGrid API calls. For some reasons mail/send seems to expect something different (or they simply don't allow username+password when sending email with the v3 API).

To take StrongGrid out of the picture, I wrote the following using nothing but the standard .net HttpClient:

var username = "myUsername";
var password = "myPassword";
var encodedAuhtorization = Convert.ToBase64String(Encoding.ASCII.GetBytes(string.Concat(username, ":", password)));

var httpClient = new HttpClient();
httpClient.BaseAddress = new Uri("https://api.sendgrid.com/v3/");
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", encodedAuhtorization);

// First request: retrieve the current user's profile. This works fine
var userProfileRequest = new HttpRequestMessage(HttpMethod.Get, "user/profile");
var userProfileResponse = await httpClient.SendAsync(userProfileRequest).ConfigureAwait(false);

// Second request: send an email. This fails with the following error: "Permission denied, wrong credentials"
var mailSendRequest = new HttpRequestMessage(HttpMethod.Post, "mail/send")
{
    Content = new StringContent("{\"personalizations\":[{\"to\":[{\"email\":\"john.doe@example.com\",\"name\":\"John Doe\"}],\"dynamic_template_data\":{\"verb\":\"\",\"adjective\":\"\",\"noun\":\"\",\"currentDayofWeek\":\"\"},\"subject\":\"Hello, World!\"}],\"from\":{\"email\":\"noreply@johndoe.com\",\"name\":\"John Doe\"},\"reply_to\":{\"email\":\"noreply@johndoe.com\",\"name\":\"John Doe\"},\"template_id\":\"<<YOUR_TEMPLATE_ID>>\"}")
};
var mailSendResponse = await httpClient.SendAsync(mailSendRequest).ConfigureAwait(false);

The first request works fine but the second one fails. Important to note that both calls user the same username and password which demonstrate that the issue is not that the username and/or password is wrong or incorrectly encoded.

[farlee2121]

Yeah. It's possible they change up the format to keep people from using basic auth. Two things

• I opened a support ticket with sendgrid for clarification
• I found this release note that says they reject basic auth calls if two-factor auth is enabled. However, I do not have two-factor auth setup for my account

[Jericho]

Cool. Let me know what their support says. I'm genuinely curious and I'll be happy to make any modification to StrongGrid if necessary.

[farlee2121]

No word yet. I'll be sure to let you know. Thanks for your expedient support!

[farlee2121]

I heard back from SendGrid, and you were right. They simply don't allow basic authentication for sending mail on the v3 api.

I've suggested a clarification in their documentation

[Jericho]

... and also the error message they return should be clearer!

[Jericho]

Here's an idea: why don't we improve StrongGrid to detect if you are trying to send an email with basic authentication? If so, we could throw an exception with a meaningful message such as SendGrid does not support Basic authentication when sending transactional emails.

[farlee2121]

Agreed! Is that something you'd like me to contribute?

[Jericho]

I'm already on it. I'll have something you'll be able to test shortly.

[farlee2121]

Sweet, thanks!

[Jericho]

Release candidate available on my MyGet feed. Let me know if you have a chance to try it.

[farlee2121]

I don't think I'll be able to tackle this until early next week

[farlee2121]

Looks like it's behaving as expected.

Behavior I see

1. No error instantiating client with username & password
2. SendToSingleRecipientAsync, SendToMultipleRecipientsAsync, SendAsync all throw StrongGrid.Utilities.SendGridException : SendGrid does not support Basic authentication when sending transactional emails.
3. Other api calls like WebhookSettings.GetAllInboundParseWebhookSettings() work with basic auth
4. With an apikey, SendToSingleRecipientAsync, SendToMultipleRecipientsAsync, and SendAsync all work as expected

[Jericho]

Excellent. Thanks for testing. I will publish to nuget momentarilly.

[Jericho]

:tada: This issue has been resolved in version 0.68.0 :tada:

The release is available on:

• GitHub Release
• NuGet Package

Your GitReleaseManager bot :package::rocket: