AzureServiceBus Transport - Allow Configuring of ConnectivityMode

[robinbihun]

I'm in a particular environment that will allow us to use AzureServiceBus using HTTP or HTTPS transports. In our Saga/Handler endpoints we had to have a configuration as follows, which sets the AzureTransport Connectivity Mode to HTTPS:

configuration.UseTransport<AzureServiceBusTransport>()
    .UseTopology<ForwardingTopology>()
    .ConnectivityMode(ConnectivityMode.Https);

Currently there does not appear to be a way to do this when setting up a new service control instance, I do not see any documentation that would indicate a way to set this in the config file either for the instance. For our case, this causes ServiceControl to fail with the following errors in the log.

2016-11-29 12:18:58.3548|63|Warn|NServiceBus.Azure.Transports.WindowsAzureServiceBus.AzureServiceBusQueueSender|Transient MessagingCommunicationException occured when sending to queue Particular.ServiceControl
Microsoft.ServiceBus.Messaging.MessagingCommunicationException: Could not connect to net.tcp://mycustomendpoint.servicebus.windows.net:9354/. The connection attempt lasted for a time span of 00:00:01.0060911. TCP error code 10061: No connection could be made because the target machine actively refused it 40.84.185.67:9354.  ---> System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://mycustomendpoint.servicebus.windows.net:9354/. The connection attempt lasted for a time span of 00:00:01.0060911. TCP error code 10061: No connection could be made because the target machine actively refused it 40.84.185.67:9354.  ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 40.84.185.67:9354
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.ServiceModel.Channels.SocketConnectionInitiator.ConnectAsyncResult.OnConnect(IAsyncResult result)
   --- End of inner exception stack trace ---

[SeanFeldman]

@robbihun ASB transport has a setting called ConnectivityMode on AzureServiceBusQueueConfig configuration section you'd need to add to the SC .config file (SC installation path\ServiceControl.exe.config).

Note: do not provide the connection string via transport configuration section as it's not going to be used. It's set by SC with the value specified in the Management tool.

[johnsimons]

@robbihun here is the doco for it - https://docs.particular.net/nservicebus/azure-service-bus/configuration/azureservicebusqueueconfig

[johnsimons]

And yes ServiceControl does not have a graphical way to set this up, hence the need to hack the config file manually, sorry!

[robinbihun]

@SeanFeldman @johnsimons, I somehow missed it in the documentation the past few hours while looking around for a workaround...

That get's me past the first error but i still cant successfully peek from the azure queues... I now get the following:

Microsoft.ServiceBus.Messaging.MessagingCommunicationException: Channel Open did not complete within the specified timeout of 00:00:43.4388191.

I've seen this in other code that tried to access the queues using an older version of WindowsAzure.ServiceBus - are there plans to update the microsoft dependencies to a more recent version, looks like service control is at 2.6.0 and the current version is 3.4.2

[SeanFeldman]

@robbihun are you setting ConnectivityMode to Http and receive this error? It looks like communication issue. Could you test the environment to verify that this is not firewall related?

[robinbihun]

@SeanFeldman Yes, this is using Http ConnectivityMode, I can make outgoing calls on port 80 to azureservicebus; however, for some reason (and i had this problem with our handlers/sagas as well) using ConnectivityMode=Http causes this error; however, ConnectivityMode=Https does not.

I'm not sure what the cause is in older versions of WindowsAzure.ServiceBus but it is resolved for me at least with the more recent versions.

[SeanFeldman]

@robbihun version 2.x of ASB didn't support Https iirc.

[johnsimons]

@robbihun as pointed out by @SeanFeldman ConnectivityMode.Https is not supported in v2.6 of Microsoft.ServiceBus, so that is the reason it does not work.

So now the big question is, given that ServiceControl users v2.6 of Microsoft.ServiceBus, how can we address this?

@SeanFeldman the only thing I can think of is for us to update Microsoft.ServiceBus to a later version, is this possible ?

[SeanFeldman]

@johnsimons don't think so - v3 had breaking changes. @yvesgoeleven do you recall what those breaking changes are that prevented usage of NSB.ASB v6 with ASB v3?

[SeanFeldman]

@robbihun can I ask you why you're not using the default (TCP)? Is there a specific reason for that? Currently, SC cannot use ASB v3 so you'd be limited to HTTP only.

[yvesgoeleven]

@SeanFeldman @johnsimons The APM interface (Begin/End) that we use in the transport have been internalized in the SDK

[robinbihun]

@SeanFeldman, The default TCP connections are unavailable to us at this time, it is not exactly a technical reason and not within my control. shrug

If I understand correctly, it seems you guys do not have a plan yet to get to Microsoft.ServiceBus v3 within ServiceControl. I just want to verify this is only limited to ServiceControl correct? We are using Microsoft.ServiceBus v3 with our NSB handlers/sagas etc... and it looks like NServiceBus.AzureServiceBus is using Microsoft.ServiceBus v3 as well.

[SeanFeldman]

@robbihun NSB.ASB is on the latest version of ASB and will be. Unfortunately SC is a version behind.

[robinbihun]

Ah, i see now, 'cause the current version of SC depends on NSB.ASB v6 not v7, that was my confusion... I guess we'll need to just figure something out until SC is brought to the current version.

Thanks for your patience with me and your help in understanding where all of this stands.

[SeanFeldman]

You're correct. I'm sorry for the inconvenience. If you need to monitor your error queue, you could look at Azure Functions as a temporary solution.

[robinbihun]

Is there any update on this? We are currently migrating from MSMQ to AzureServiceBus and we're not able to use the monitoring functionality we're used to with ServiceControl/ServicePulse/ServiceInsight.

The azure functions is not an option for us, additionally - it doesn't allow for easily monitoring and notifying based on hearbeats or custom checks.

[SeanFeldman]

@robbihun an option I can think of is to use SC Transport Adapter. This will allow to run endpoints with NSB.ASB v7 configured to use HTTPS and use SC/SP/SI. For the platform tool, you'll be able to run those on MSMQ and won't need to convert those. Is that something that will help you?

[robinbihun]

@SeanFeldman - to make sure i understand the use case of the Transport Adapters, do all of the endpoints have to be configured with this to send the ServiceControl messages to a different endpoint (instead of the queues in ASB)? If that is the case, that does not work for us since the reason we're moving the queues to ASB is because of firewall restrictions around MSMQ/MSDTC.

Or is the TransportAdapter created as a standalone service that we could install alongside service pulse that will retrieve messages from our servicecontrol queue on ASB and forward them to the local MSMQ queue? I see how to configure the transportadapter, but not really now to use it... e.g. I created a blank application that does nothing but the tranportadapter and it runs, but i never see it polling the ASB queues, or populating anything in the MSMQ queues.

[robinbihun]

@SeanFeldman -- Edit, nevermind - I think i may have found it...

I'm still not having much luck though - calling ITransportAdapter.Start() appears to run, but does nothing still. I don't see anything that will allow me to increase logging for me to try to figure out what's going on. This seems promising though, I think i just need some additional guidance on getting it running.

[SeanFeldman]

@robbihun There's also a sample that you can have a look at.

[robinbihun]

@SeanFeldman, Thanks, the sample helps a bit. I'm at least able to get an error now,

System.ApplicationException occurred
  HResult=0x80131600
  Message=Could not build NServiceBus.Transport.AzureServiceBus.MessagePump
  Source=NServiceBus.Azure.Transports.WindowsAzureServiceBus

Have any of you been able to get this setup working? It doesn't seem like this transport adapter has been thoroughly vetted as a recommendation for production use. If i need to put in a support request to get better focused help with this, I can do so. This is escalating beyond a github issue on our end and will become a production issue really fast.

[robinbihun]

@SeanFeldman Okay, i think i got it all working from digging through the code for the different transports, here's what i had to add to get TransportAdapter to work with ASB. (I'm not sure what the best setting for "NServiceBus.SharedQueue" should be). This will be a sufficient workaround for us, until SC is updated to the latest dependencies to where it can work with ASB over HTTPS.

        // Configure our endpoint side (Azure Queues)
        transportAdapterConfig.CustomizeEndpointTransport(transportExtensions =>
        {
            transportExtensions.ConnectionStringName("AzureConnectionString");
            transportExtensions.UseForwardingTopology();
            transportExtensions.ConnectivityMode(ConnectivityMode.Https);
            transportExtensions.Transactions(TransportTransactionMode.ReceiveOnly);
            transportExtensions.GetSettings()
                .SetDefault("NServiceBus.SharedQueue", "audit");
        });

[SeanFeldman]

@robbihun I'm traveling but will get back to you on the config. TransportTransactionMode.ReceiveOnly doesn't look right I'm as ideally it should be atomic sends with receive. will have to verify.

[SzymonPobiega]

@robbihun let me know if you have any questions. We can jump on a call to try resolve any issues you have.

[swettstein]

are there any plans to update ServiceControl to work with NSB.ASB v7?

we're trying to post messages consumed in an Azure Function to the audit and error queues via HTTPS but those messages are encoded as "application/octect-stream" instead of "wcf/byte-array" which NSB.ASB v6 cannot deserialize.

[SeanFeldman]

@swettstein while SC is not on NSB.ASB v7, would going with SC Adapter be an option? There's an ASB transport specific sample you can pull down to see how it works.

[swettstein]

@SeanFeldman not really. if i understand it correctly, that would require another app to get developed, tested, deployed and maintained as well as intermediary audit & error queues. it seems a bit too much overhead.

i figured out a way to use DataContractSerializer to replicate the xml serialization and forcing the transport encoding to "wcf/byte-array". it works for now, but i'd be curious to know if there are plans to support NSB.ASB v7.

[SeanFeldman]

I'll let @Particular/servicecontrol-maintainers chime in.