Closed felipe-iob closed 6 months ago
Hi @felipe-iob,
Thank you for reporting this issue. Using below code sample, I am unable to get the EndpointResolver Exception.
public static async Task Main(string[] args)
{
AmazonSQSClient client;
AmazonSQSConfig config = new AmazonSQSConfig();
Environment.SetEnvironmentVariable("AWS_ACCESS_KEY_ID", "<<redacted>>");
Environment.SetEnvironmentVariable("AWS_SECRET_ACCESS_KEY", "<<redacted>>");
Environment.SetEnvironmentVariable("AWS_SQS_ENDPOINT", "https://sqs.us-east-1.amazonaws.com");
if (Environment.GetEnvironmentVariable("AWS_SQS_ENDPOINT") != null)
{
config.ServiceURL = Environment.GetEnvironmentVariable("AWS_SQS_ENDPOINT");
}
AWSCredentials credentials = null;
if (Environment.GetEnvironmentVariable("AWS_ACCESS_KEY_ID") != null)
{
// take credentials from environment variables
credentials = new EnvironmentVariablesAWSCredentials();
client = new AmazonSQSClient(credentials, config);
}
else
{
// take credentials from instance profile
// client = new AmazonSQSClient(regionEndPoint);
client = new AmazonSQSClient();
};
var queueUrl = "https://sqs.us-east-1.amazonaws.com/<<redacted>>/QueueName";
var response = await client.SendMessageAsync(queueUrl,"body text");
Console.WriteLine("Status Code: "+response.HttpStatusCode);
}
Can you please share the AWS_SQS_ENDPOINT
value you are setting? And also, do you get similar issue on the latest SQS/Core version?
Regards, Chaitanya
Hi Chaitanya. Thank you for the answer The AWS_SQS_ENDPOINT I use goaws url [http://us-east-1.goaws.com:4100/100010001000/CRUD_EVENT] to test local and the our official aws queueUrl. Both configuration I get the error.
About the SQS/Core version. Yes, I get the same error using the latest version. Regards, Felipe
Oh, newer versions of the SDK will use a JSON-based wire protocol when communicating with SQS (more details in this blog post: https://aws.amazon.com/blogs/aws/new-for-amazon-sqs-update-the-aws-sdk-to-reduce-latency/)
I found this issue in the goaws
repository: https://github.com/Admiral-Piett/goaws/issues/279, it doesn't look like it supports the new JSON protocol yet (that also explains why @bhoradc wasn't able to reproduce the problem).
If you'd like to keep testing locally using that library, their issue will need to be resolved before you update the Edit: Please ignore this sentence, this isn't relevant to the issue here.AWSSDK.SQS
package to the latest version
Last SDK's version. And I send the message direct to aws url queue. I got the same error.
Do you know if the Hypercube or another dependency is hooking into the AWS SDK's request pipeline at all via AddHandler
? I see it in the stack trace above:
Hypercube.Data.EntityFramework.Repositories.AddRepositoryBase`1.CrudEventProcess
Version 3.7.100 changed how the SDK's internal request pipeline handles endpoint resolution, which removed the EndpointResolver
step in the SDK pipeline. Any code that was inserting pipeline handlers before or after EndpointResolver
may no longer work.
We discussed this internally this morning, and it also may be possible when mixing AWSSDK.Core and service packages before and after the 3.7.100 update related to endpoint.
I see your package references are specifying 3.7.300+, but could you also upload or check your deps.json
file in \bin
folder to see if a different version of AWSSDK.SQS and/or AWSSDK.Core is being resolved?
My complete code in deps.json. All SDK versions are 3.7.300+
Thanks (FYI I edited your comment so it's easier to read). Did you have a chance to check the other question we asked: https://github.com/aws/aws-sdk-net/issues/3189#issuecomment-1948868849
Do you know if the Hypercube or another dependency is hooking into the AWS SDK's request pipeline at all via AddHandler? Version 3.7.100 changed how the SDK's internal request pipeline handles endpoint resolution, which removed the EndpointResolver step in the SDK pipeline. Any code that was inserting pipeline handlers before or after EndpointResolver may no longer work.
I would also search your codebase for any usage of the RuntimePipelineCustomizerRegistry
. Judging from the original call stack somebody registered a IRuntimePipelineCustomizer
with RuntimePipelineCustomizerRegistry
which is executed as part of the ApplyCustomizations
method we see in the call stack. These are considered internal APIs that we put in a "internal" namespace and we don't put them in our API documentation. We only use them for dependencies we control because they rely on internal structure of the SDK which is subject to change like the EndpointResolver
changed about a year ago.
@dscpinheiro
Do you know if the Hypercube or another dependency is hooking into the AWS SDK's request pipeline at all via AddHandler? None of then are hooking into the AWS SDKs request pipeline at all.
Not even the Hypercube.Data.EntityFramework.Repositories.AddRepositoryBase`1.CrudEventProcess method? As Norm said (https://github.com/aws/aws-sdk-net/issues/3189#issuecomment-1953355037), it looks like it's using an internal API of the SDK.
Hypercube.Data.EntityFramework.Repositories.AddRepositoryBase`1.CrudEventProcess
public class CrudEventProcess {
public static async Task SendEvent<TEntity>(TEntity entity, EntityOperationEnum operation, ILogger logger, ActivitySource activitySource) where TEntity : class
{
using var activity = activitySource.StartActivity($"CrudEventProcess_{nameof(SendEvent)}", ActivityKind.Internal)!;
try
{
/*Gera o evento*/
if (entity is ICrudEventProcess eventEntity)
{
Guid entityId = Guid.Empty;
if (entity is IBaseEntity baseEntity)
{
entityId = baseEntity.Id;
}
Guid tenantId = Guid.Empty;
if (entity is ITenantEntity tenantEntity)
{
tenantId = tenantEntity.TenantId;
}
var entityOperationEvent = EntityOperationEvent.Create
(
entityId,
tenantId,
operation,
entity.GetType().Name,
Newtonsoft.Json.JsonConvert.SerializeObject(entity)
);
#region Publisher
AmazonSQSClient client;
AmazonSQSConfig config = new AmazonSQSConfig();
var regionEndPoint = RegionEndpoint.GetBySystemName(Environment.GetEnvironmentVariable("AWS_REGION") ?? "us-east-1");
if (Environment.GetEnvironmentVariable("AWS_SQS_ENDPOINT") != null)
{
config.ServiceURL = Environment.GetEnvironmentVariable("AWS_SQS_ENDPOINT");
}
AWSCredentials credentials = null;
if (Environment.GetEnvironmentVariable("AWS_ACCESS_KEY_ID") != null)
{
// take credentials from environment variables
client = new AmazonSQSClient(key, secret, config);
}
else
{
// take credentials from instance profile
// client = new AmazonSQSClient(regionEndPoint);
client = new AmazonSQSClient();
}
var queueUrl = await GetQueueUrl(client, EnvironmentVariables.Get("SQS_CRUD_EVENT_QUEUE_NAME") ?? "crud_event_queue");
var response = await SendMessage(client, queueUrl, Newtonsoft.Json.JsonConvert.SerializeObject(entityOperationEvent), new Dictionary<string, MessageAttributeValue>());
#endregion
}
}
catch (System.Exception ex)
{
var errorMessage = $"CrudEventProcess_{nameof(SendEvent)} Couldn't send message: {ex.Message}";
logger.LogError(ex, errorMessage);
activity.SetStatus(ActivityStatusCode.Error, errorMessage);
throw;
}
}
protected static async Task<string> GetQueueUrl(AmazonSQSClient client, string queueName)
{
try
{
var response = await client.GetQueueUrlAsync(queueName);
if (response.HttpStatusCode == System.Net.HttpStatusCode.OK)
{
Console.WriteLine($"The URL for {queueName} is: {response.QueueUrl}");
return response.QueueUrl;
}
else
{
throw new Exception("Error getting queue URL");
}
}
catch (QueueDoesNotExistException ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine($"The queue {queueName} was not found.");
throw;
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
throw;
}
}
/// <summary>
/// Sends a message to an SQS queue.
/// </summary>
/// <param name="client">An SQS client object used to send the message.</param>
/// <param name="queueUrl">The URL of the queue to which to send the
/// message.</param>
/// <param name="messageBody">A string representing the body of the
/// message to be sent to the queue.</param>
/// <param name="messageAttributes">Attributes for the message to be
/// sent to the queue.</param>
/// <returns>A SendMessageResponse object that contains information
/// about the message that was sent.</returns>
protected static async Task<SendMessageResponse> SendMessage(
IAmazonSQS client,
string queueUrl,
string messageBody,
Dictionary<string, MessageAttributeValue> messageAttributes)
{
var sendMessageRequest = new SendMessageRequest
{
DelaySeconds = 10,
MessageAttributes = messageAttributes,
MessageBody = messageBody,
QueueUrl = queueUrl,
};
var response = await client.SendMessageAsync(sendMessageRequest);
Console.WriteLine($"Sent a message with id : {response.MessageId}");
return response;
}
}
Hello guys. After remove this reference and this line of code. The lib worked like a charm.
Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
If you get a chance, what version of OpenTelemetry.Contrib.Instrumentation.AWS
were you using? I believe we fixed this in 1.0.2 via the following PRs, so I'm curious if you were still seeing this issue on that version or later?
We used the version 1.0.1. Now we no longer use this lib.
Describe the bug
When the instantiate the client = new AmazonSQSClient() object with the correct configurations I receive that error "Cannot find a handler of type EndpointResolver".
Expected Behavior
Message sended to queue.
Current Behavior
Exception {System.InvalidOperationException: Cannot find a handler of type EndpointResolver at Amazon.Runtime.Internal.RuntimePipeline.AddHandlerAfter[T](IPipelineHandler handler) at Amazon.Runtime.Internal.RuntimePipelineCustomizerRegistry.ApplyCustomizations(Type type, RuntimePipeline pipeline) at Hypercube.Data.EntityFramework.Repositories.AddRepositoryBase`1.CrudEventProcess(TEntity entity, EntityOperationEnum operation) in ....}
Source AWSSDK.Core
StackTrace at Amazon.Runtime.Internal.RuntimePipeline.AddHandlerAfter[T](IPipelineHandler handler) at Amazon.Runtime.Internal.RuntimePipelineCustomizerRegistry.ApplyCustomizations(Type type, RuntimePipeline pipeline) at Hypercube.Data.EntityFramework.Repositories.AddRepositoryBase`1.CrudEventProcess(TEntity entity, EntityOperationEnum operation) in .....
Reproduction Steps
AmazonSQSClient client; AmazonSQSConfig config = new AmazonSQSConfig();
Possible Solution
No response
Additional Information/Context
No response
AWS .NET SDK and/or Package version used
.net6.0 AWSSDK.SQS last version and version 3.7.0.9.
Targeted .NET Platform
.NET Framework 6.0
Operating System and version
Ubuntu