LinkTests in amqp-dotnet.sln do not all run successfully when run against a real broker

[cYCL157]

I am in the process of trying to figure out if message filtering will help a situation if have. I found the examples and tests in code and saw enough that I thought it would. I wrote my code and kept getting this error:

Amqp.AmqpException: 'The link 'G27:25689057:receiver-ReceiveWithFilter' contains invalid filter type. System only support Jms or Apache selector filter type but we found type 'Microsoft.ServiceBus.Messaging.Amqp.Framing.AmqpSelectorFilter' associated with key 'f1'. TrackingId:2219a319-4e6e-4720-8136-76cdc6793000_B31, SystemTracker:NoSystemTracker, Timestamp:2022-08-17T18:41:40 TrackingId:8d1f4652c8f54078874bba0419497230_G27, SystemTracker:gateway7, Timestamp:2022-08-17T18:41:40'

I was assuming I was doing something wrong and googled things for a while, searched code for the type name that was referenced as a problem, and didn't come up with anything that led to resolution. I then decided to clone the repo and run the tests to confirm. After figuring out how to run the test broker successfully:

dotnet .\TestAmqpBroker.dll amqp://localhost:5672 amqps://localhost:5671 /cert:localhost /creds:guest:guest /trace:frame

(note that I had to remove ws://localhost:80 from the example found in https://github.com/Azure/amqpnetlite/blob/master/docs/articles/test_amqp_broker.md as there was an error about support for ws protocol)

..I was able to get all of the tests to pass. A little dumbfounded, I copied the following test into my project, just to be sure and found that it was a problem there.

` [Fact] public void TestMethod_ReceiveWithFilter() { string testName = "ReceiveWithFilter"; Connection connection = new Connection(address); Session session = new Session(connection);

        Message message = new Message("I can match a filter");
        message.Properties = new Properties() { GroupId = "abcdefg" };
        message.ApplicationProperties = new ApplicationProperties();
        message.ApplicationProperties["sn"] = 100;

        SenderLink sender = new SenderLink(session, "sender-" + testName, queue);
        sender.Send(message, null, null);

        // update the filter descriptor and expression according to the broker
        Map filters = new Map();
        // JMS selector filter: code = 0x0000468C00000004L, symbol="apache.org:selector-filter:string"
        filters.Add(new Symbol("f1"), new DescribedValue(new Symbol("apache.org:selector-filter:string"), "sn = 100"));
        ReceiverLink receiver = new ReceiverLink(session, "receiver-" + testName, new Source() { Address = queue, FilterSet = filters }, null);
        Message message2 = receiver.Receive();
        receiver.Accept(message2);

        sender.Close();
        receiver.Close();
        session.Close();
        connection.Close();
    }

`

After changing the default value in TestTarget class to be that of my own actual azure service bus connection string, found that 16 of the 37 tests fail in the LinkTests class. Not surprisingly, the tests in WebSocketTests failed too.

My first question is.....is this a known/expected thing? I didn't see any disclaimer on support for .net core in terms of limited features but could have just missed it.

Second, if not known, any advice as to where to start? I don't mind spending some time looking and potentially contributing, but not even quite sure where to start. I did turn on frame level traces and can see that it's happening on receive and not at link startup. I assume it's some kind of issue with "serializing" (for lack of a better term) something in the map dictionary entry.

Acknowledgment and any advice for direction would be much appreciated!

Cort

[xinchen10]

Try this,

filters.Add(new Symbol("f1"), new DescribedValue(new Symbol("jms-selector"), "sn = 100"));

Or,

filters.Add(new Symbol("f1"), new DescribedValue(0x0000468C00000004UL, "sn = 100"));

Azure Service Bus may change the symbol name of this filter.

[cYCL157]

@xinchen10 thank you! jms-selector worked for this. I REALLY appreciate the fast response!

Is there a place that I could have found that out on my own??

I am presuming that "sn" could be any key from the messages ApplicationProperties, correct? is there a way to match on anything else other than the session/groupid (that example worked)?

Would string type values be quoted with single quotes? i.e. "sn = 'foo'"?

[cYCL157]

With setting this on the message, where foo is a variable of type string:

message.ApplicationProperties["foo"] = foo; I can then filter like this on receive:

var filter = new Map { { new Symbol("f1"), new DescribedValue(new Symbol("jms-selector"), $"foo = '{foo}'") } };

That answers my question about quoting. Would love to know if there is a way to filter on any other property on the message, i.e. from the Properties object or MessageAnnotations or even body.

[cYCL157]

@xinchen10 i think i came to an erroneous conclusion. While I thought things were working, as in filtering correctly, I have found that the filters are having no effect on the messages received.

I opened back up the amqpnetlite source again and munged the search on the unit test that I was playing with and found the filter there was not appearing to work either, meaning receive got a message that did not match. I was using the TestAmqpBroker, so not sure if that should be expected or not (I could see that being more than the test broker might want to support). So, this test continues to pass with the changed filter expression being foo = 100:

` public void TestMethod_ReceiveWithFilter() { string testName = "ReceiveWithFilter"; Connection connection = new Connection(testTarget.Address); Session session = new Session(connection);

Message message = new Message("I can match a filter");
message.Properties = new Properties() { GroupId = "abcdefg" };
message.ApplicationProperties = new ApplicationProperties();
message.ApplicationProperties["sn"] = 100;

SenderLink sender = new SenderLink(session, "sender-" + testName, testTarget.Path);
sender.Send(message, null, null);

// update the filter descriptor and expression according to the broker
Map filters = new Map();
// JMS selector filter: code = 0x0000468C00000004L, symbol="apache.org:selector-filter:string"
filters.Add(new Symbol("f1"), new DescribedValue(new Symbol("apache.org:selector-filter:string"), "foo = 100"));
ReceiverLink receiver = new ReceiverLink(session, "receiver-" + testName, new Source() { Address = testTarget.Path, FilterSet = filters }, null);

Message message2 = receiver.Receive();
receiver.Accept(message2);

sender.Close();
receiver.Close();
session.Close();
connection.Close();

} `

I am not seeing any negative tests to prove that the filter is indeed filtering out messages -- at least as I understand how it should work.

[cYCL157]

@xinchen10 i found a question online about azure service bus potentially not having filtering enabled for standard level subscription, so I migrated my namespace to a premium subscription and am not seeing any difference. Here is the StackOverflow question that lead me to try that: https://stackoverflow.com/questions/70198518/spring-jms-selector-doesnt-work-with-servicebus

[xinchen10]

The test broker does not support message filtering.

Service Bus supports JMS (over AMQP) and filtering on message properties using SQL like expressions. But this is Service Bus Premium only. if you test against Standard, the filter may be ignored.

Service Bus has its own filter language, one of which being SqlFiler. I think you should be able to pass such filter using the descriptor name/code in their sdk implementation.

var filterValue = new List<object>() { "sn=100", 0 };
filters.Add(new Symbol("com.microsoft:sql-filter:list"), new DescribedValue(new Symbol("com.microsoft:sql-filter:list"), filterValue));

Unlike the JMS filter, the filter key must be the descriptor name in this case.

[cYCL157]

@xinchen10 that seems to be working fine for an ApplicationProperty that is an int underneath but for the life of my I can't get a string to work -- would you happen to have an example of that handy or that you could make? I have tried using single and double quotes around the value, I am just using equal comparison and have changed my key to make sure it has no special characters and needs quoting (something I read somewhere for characters like '-').

I'd also like to understand how to use the correlation-filter as well, this was my best guess but does not seem to be working:

var filterValue = new List<object>() { Constants.EVENT_TYPE_KEY, properties.EventType }; var filter = new Map { { new Symbol("com.microsoft:correlation-filter:list"), new DescribedValue(new Symbol("com.microsoft:correlation-filter:list"), filterValue) } };

and....thank you for the continued help!

[xinchen10]

String type should use single quotes, e.g. "prop1='abc'" If it still doesn't work (but int works), it's beyond my knowledge.

Correlation-filter works on some common standard BrokeredMessage properties. If any of them matches the one of a BrokeredMessage instance, the message is selected (so it is much more efficient than a SqlFilter). You got the descriptor code correct. The value is a list of the following property values (you have to provide a value even if it is null and the order matters).

CorrelationId,
MessageId,
To,
ReplyTo,
Label,
SessionId,
ReplyToSessionId,
ContentType