Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
585 stars 776 forks source link

Queue, topic, subscription, and rule name maximum length should be consistent. #257

Open SeanFeldman opened 5 years ago

SeanFeldman commented 5 years ago

This is a superset of #6

As of today, queues and topics allow 260 characters maximum name length. Subscriptions and rules are limited to 50.

  1. What's the reason for this difference?
  2. Can it be made consistent and have 260 on all entity types. 50 characters is extremely limiting and often pushes customers to create workarounds such as "hashed" entity name just to be able to map a logical name to the entity.
axisc commented 5 years ago

@SeanFeldman thanks for reporting this. Added it to the backlog.

andreujuanc commented 4 years ago

This is a big issue for us. We even had to create a "Mandatory unit test" that scans the assembly, looks for functions subscription names and asserts their length.

johnwatson484 commented 3 years ago

Is there any update on this issue? There doesn't seem to have been any activity for a few years.

Because there is no local development capability as covered in this issue -> https://github.com/Azure/azure-service-bus/issues/223, we have had to find ways of incorporating a shared namespace into our local development and CI approach.

For local development, we have duplicated every, queue, topic and subscription for every developer and tester. Not ideal and difficult to maintain especially when you scale a microservices architecture that predominantly uses asynchronous messaging but that's all related to #223 so won't go too much into that here.

For our CI pipeline, the limitation of 50 characters for subscriptions is a real issue.

For every commit with an open pull request, our CI pipeline runs integration tests against immediate infrastructure. e.g if an application uses a Postgres database or Redis cache, we spin up a Postgres or Redis container and run the application tests across it.

Because we can't emulate Azure Service Bus, we instead dynamically create a copy of each queue, topic and subscription used by the application and run the tests against this dynamic infrastructure. In order to avoid collisions between builds we name these entities with the convention <microservice name>-pr<pull request number>-b<build number>-<queue name>

We also have to add clean up steps to every test to make sure each has a clean test run, but I guess that's another #223 issue.

We also dynamically create an AKS namespace for that pull request and deploy the microservice to in isolation. We spin up all dynamic infrastructure using Azure PaaS offerings so we have as close to a production like deployment for every build prior to code being merged. This allows extra automated tests to run and also can support teams in manually testing or showcasing their changes to get a higher level of confidence prior to merge. Once the PR is merged/closed, this infrastructure is automatically cleaned up.

The naming convention for these queues/topics/subscriptions is <microservice name>-pr<pull request number>-<queue name>

In both these scenarios we need to feed the dynamic name to the application configuration so it can use these queues. For the CI tests, we use Docker Compose, for the dynamic deployment we use Helm. We rely on naming convention in order to match everything up correctly.

With subscriptions, the limit of 50 means we often have scenarios where we breach this limit for dynamic queues.

Our options are:

  1. keep our microservice names short which means they can become less meaningful
  2. keep our subscription names short which means they can become less meaningful
  3. add something to our CI that automatically shortens our queue names, eg remove all vowels but that still risks collisions at scale
  4. have a separate convention for subscriptions for dynamic infrastructure but this brings complexity to making sure the application picks up the correct configuration.

If the limit of 50 characters for subscriptions was increased it would make a big difference.

esbenbach commented 2 years ago

This issue is still open - but there seems to be no activity here. Could someone from the SB team at least consider commenting on it? I.e not gonna happen, maybe when we have tons of time, or "for X reasons its done like this and its hard/expensive to fix!", or whatever.

Its a really weird that we can't get some sort of feedback here.

SeanFeldman commented 2 years ago

@EldertGrootenboer, this is still an issue that's causing problems. Just today saw someone mentioning exactly this problem.

EldertGrootenboer commented 2 years ago

This limit comes from internal restrictions, so we are not sure if this is something we can extend. However, we have opened an investigation task for this in our backlog, and will update this issue when we have more information.

esbenbach commented 2 years ago

Somewhere else i posted a question about WHY the limitations are there. They seems archaic and frankly like a left over from the 1990 or something when we should conserve as many bytes as possible.

If the SB team (i am aware that is not you!), could give us a reasonable explanation as to WHY the limits are there (specific performance issues for instance), then that information could be in the docs, and everyone would be better of (if not exactly happy).

So in case the internal team can't remove the limitations, could you please have them try and document the WHY :)

madmalkav commented 1 year ago

Working on migrating some stuff from IBM MQ to Azure Service Bus, last thing I expected was a recent, cloud-oriented piece of software to have more restricting naming rules that an ancient product...

SeanFeldman commented 1 year ago

Anything to share from the investigation finding, @EldertGrootenboer?

navneet10sep commented 11 months ago

Has there been any progress on revisiting these limitations, or is this issue still in the backlog? I'm curious if there might be a workaround or lower-level API within the Azure Service Bus SDK that could help to navigate these constraints more effectively. @EldertGrootenboer

EldertGrootenboer commented 11 months ago

Thank you for your feedback on this item, it helps us in our efforts to continuously prioritize the different requests we get from all our various feedback channels. We have added this feature in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this feature, as well as explain their scenarios.

EldertGrootenboer commented 4 months ago

This item in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this item.

esbenbach commented 4 months ago

Still no reasoning for the restrictions? :-)

EldertGrootenboer commented 4 months ago

Still no reasoning for the restrictions? :-)

As for reasoning around this, there are many places where there are references to these properties, both internally and externally. Hence, changing this might cause unexpected behavior. As always, we have to make choices between all the items we are tracking, next to day to day operations and other priorities which are constantly coming in. Unfortunately, this item didn't make the cut this time, but we do ask for others to keep upvoting this, so we get good insights into how much impact this has, and can plan it accordingly.

dmadrian commented 2 weeks ago

I'd like to register my interest in seeing this limitation on the length of topic names addressed. It's consumed a lot of time and effort over the years with clients I've worked with trying to adopt messaging.