search

piotr-rojek / devopsifyme-sbemulator

MIT License
66 stars 12 forks source link

Timeout when trying to use #9

Closed JakeGinnivan closed 1 year ago

JakeGinnivan commented 1 year ago

Hey, this is a awesome initiative. I thought I'd give it a spin.

I'm getting a timeout message off the bat:

portal:       "type": "ServiceBusError",
portal:       "message": "The request with message_id \"57f834fd-c7ab-b748-b045-81ca07f5446c\" to \"address\" endpoint timed out. Please try again later.",
portal:       "stack":
portal:           ServiceBusError: The request with message_id "57f834fd-c7ab-b748-b045-81ca07f5446c" to "address" endpoint timed out. Please try again later.

Inside docker have this.

2023-03-25 10:15:41 [02:15:41.747] SEND (ch=0) disposition(role:True,first:0,last:0,settled:True,state:rejected(error:error(condition:amqp:not-found,description:Not response link was found. Ensure the link is attached or reply-to is set on the request.)),batchable:False)
2023-03-25 10:15:41 [02:15:41.747] SEND (ch=0) close(error:error(condition:amqp:internal-error,description:Object reference not set to an instance of an object.))

Am on a M1 mac, happy to dig in with a bit of direction about best way to debug or where to look.

rafak commented 1 year ago

i am experiencing a similar issue (note: i am using nodejs with azure sdk for js)

info: ServiceBusEmulator.ServiceBusEmulatorWorker[0]
      Worker started at: 04/13/2023 21:56:30 +00:00
info: Microsoft.Hosting.Lifetime[14]
      Now listening on: http://[::]:80
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Docker
info: Microsoft.Hosting.Lifetime[0]
      Content root path: /app
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://localhost/health - -
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 GET http://localhost/health - - - 200 - text/plain 192.2564ms
warn: ServiceBusEmulator.ServiceBusEmulatorHost[0]
      AMQP SSL errors RemoteCertificateNotAvailable.
[09:56:39.850] SEND AMQP 3 1 0 0
[09:56:39.856] SEND sasl-mechanisms(sasl-server-mechanisms:[MSSBCBS,EXTERNAL,ANONYMOUS])
[09:56:39.871] RECV sasl-init(mechanism:ANONYMOUS,initial-response:...,hostname:sbemulator)
[09:56:39.874] SEND sasl-outcome(code:Ok)
[09:56:39.881] SEND AMQP 0 1.0.0
[09:56:39.883] SEND (ch=0) open(container-id:ContainerHost-867a9581d1124f6f9c088071de7ab000,host-name:localhost,max-frame-size:262144,channel-max:255,idle-time-out:2147483647)
[09:56:39.887] RECV AMQP 0 1 0 0
[09:56:39.890] RECV (ch=0) open(container-id:487d68d9-d7e5-b44d-bbed-09fba0a1aec0,host-name:sbemulator,max-frame-size:0,channel-max:0,idle-time-out:60000,properties:[product:MSJSClient,version:7.9.0,user-agent:azsdk-js-azureservicebus/7.9.0 Node/v16.19.1 OS/(x64-Linux-4.19.0-23-amd64),platform:(x64-Linux-4.19.0-23-amd64),framework:Node/v16.19.1])
[09:56:39.895] RECV (ch=0) begin(remote-channel:0,next-outgoing-id:0,incoming-window:2048,outgoing-window:4294967295,handle-max:0)
[09:56:39.899] SEND (ch=0) begin(remote-channel:0,next-outgoing-id:0,incoming-window:2048,outgoing-window:2048,handle-max:63)
[09:56:39.915] RECV (ch=0) attach(name:e7778c9c-3aef-a046-a462-3bd4229ebc99,handle:0,role:False,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(durable:0,timeout:0,dynamic:False),target:target(address:$cbs,durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:0)
[09:56:39.931] SEND (ch=0) attach(name:e7778c9c-3aef-a046-a462-3bd4229ebc99,handle:0,role:True,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(durable:0,timeout:0,dynamic:False),target:target(address:$cbs,durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:2147483647)
[09:56:39.937] SEND (ch=0) flow(next-in-id:0,in-window:2048,next-out-id:0,out-window:2048,handle:0,delivery-count:0,link-credit:100,available:0,drain:False,echo:False)
[09:56:39.941] RECV (ch=0) attach(name:cbs-de597895-e2fb-f444-a760-3f71f67a4563,handle:1,role:True,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(address:$cbs,durable:0,timeout:0,dynamic:False),target:target(durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:0)
warn: ServiceBusEmulator.ServiceBusEmulatorHost[0]
      AMQP SSL errors RemoteCertificateNotAvailable.
[09:56:39.953] SEND AMQP 3 1 0 0
[09:56:39.955] SEND sasl-mechanisms(sasl-server-mechanisms:[MSSBCBS,EXTERNAL,ANONYMOUS])
[09:56:39.956] RECV sasl-init(mechanism:ANONYMOUS,initial-response:...,hostname:sbemulator)
[09:56:39.958] SEND sasl-outcome(code:Ok)
[09:56:39.959] SEND AMQP 0 1.0.0
[09:56:39.960] SEND (ch=0) open(container-id:ContainerHost-867a9581d1124f6f9c088071de7ab000,host-name:localhost,max-frame-size:262144,channel-max:255,idle-time-out:2147483647)
[09:56:39.962] RECV AMQP 0 1 0 0
[09:56:39.963] RECV (ch=0) open(container-id:487d68d9-d7e5-b44d-bbed-09fba0a1aec0,host-name:sbemulator,max-frame-size:0,channel-max:0,idle-time-out:60000,properties:[product:MSJSClient,version:7.9.0,user-agent:azsdk-js-azureservicebus/7.9.0 Node/v16.19.1 OS/(x64-Linux-4.19.0-23-amd64),platform:(x64-Linux-4.19.0-23-amd64),framework:Node/v16.19.1])
[09:56:39.964] RECV (ch=0) begin(remote-channel:0,next-outgoing-id:0,incoming-window:2048,outgoing-window:4294967295,handle-max:0)
[09:56:39.964] SEND (ch=0) begin(remote-channel:0,next-outgoing-id:0,incoming-window:2048,outgoing-window:2048,handle-max:63)
[09:56:39.967] RECV (ch=0) attach(name:33404695-cc7e-4d42-9466-7d049479bd7a,handle:0,role:False,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(durable:0,timeout:0,dynamic:False),target:target(address:$cbs,durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:0)
[09:56:39.970] SEND (ch=0) attach(name:33404695-cc7e-4d42-9466-7d049479bd7a,handle:0,role:True,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(durable:0,timeout:0,dynamic:False),target:target(address:$cbs,durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:2147483647)
[09:56:39.972] SEND (ch=0) flow(next-in-id:0,in-window:2048,next-out-id:0,out-window:2048,handle:0,delivery-count:0,link-credit:100,available:0,drain:False,echo:False)
[09:56:39.973] SEND (ch=0) attach(name:cbs-de597895-e2fb-f444-a760-3f71f67a4563,handle:1,role:False,snd-settle-mode:Unsettled,rcv-settle-mode:First,incomplete-unsettled:False,initial-delivery-count:0,max-message-size:2147483647)
[09:56:39.977] RECV (ch=0) attach(name:cbs-5829df45-3780-f549-99bb-4d5ee235aec2,handle:1,role:True,snd-settle-mode:Unsettled,rcv-settle-mode:First,source:source(address:$cbs,durable:0,timeout:0,dynamic:False),target:target(durable:0,timeout:0,dynamic:False),incomplete-unsettled:False,initial-delivery-count:0,max-message-size:0)
[09:56:39.983] SEND (ch=0) attach(name:cbs-5829df45-3780-f549-99bb-4d5ee235aec2,handle:1,role:False,snd-settle-mode:Unsettled,rcv-settle-mode:First,incomplete-unsettled:False,initial-delivery-count:0,max-message-size:2147483647)
[09:56:39.976] SEND (ch=0) detach(handle:1,closed:True,error:error(condition:amqp:internal-error,description:Value cannot be null. (Parameter 'key')))
[09:56:39.984] SEND (ch=0) detach(handle:1,closed:True,error:error(condition:amqp:internal-error,description:Value cannot be null. (Parameter 'key')))
[09:56:39.994] RECV (ch=0) flow(next-in-id:0,in-window:2048,next-out-id:0,out-window:4294967295,handle:1,delivery-count:0,link-credit:1000,available:0,drain:False,echo:False)
[09:56:40.001] RECV (ch=0) flow(next-in-id:0,in-window:2048,next-out-id:0,out-window:4294967295,handle:1,delivery-count:0,link-credit:1000,available:0,drain:False,echo:False)
[09:56:40.022] RECV (ch=0) transfer(handle:0,delivery-id:0,delivery-tag:30,message-format:0,settled:False,more:False,rcv-settle-mode:First,resume:False,aborted:False,batchable:False) payload 450
[09:56:40.033] SEND (ch=0) disposition(role:True,first:0,last:0,settled:True,state:rejected(error:error(condition:amqp:not-found,description:Not response link was found. Ensure the link is attached or reply-to is set on the request.)),batchable:False)
[09:56:40.041] RECV (ch=0) transfer(handle:0,delivery-id:0,delivery-tag:30,message-format:0,settled:False,more:False,rcv-settle-mode:First,resume:False,aborted:False,batchable:False) payload 380
[09:56:40.043] SEND (ch=0) disposition(role:True,first:0,last:0,settled:True,state:rejected(error:error(condition:amqp:not-found,description:Not response link was found. Ensure the link is attached or reply-to is set on the request.)),batchable:False)
[09:56:40.114] SEND (ch=0) close(error:error(condition:amqp:internal-error,description:Object reference not set to an instance of an object.))
[09:56:40.117] SEND (ch=0) close(error:error(condition:amqp:internal-error,description:Object reference not set to an instance of an object.))

the Not response link... is coming from a dependency lib amqpnetlite here: https://github.com/Azure/amqpnetlite/blob/c7596e6f039d8dee706e7d126c30e83cbfa0205e/src/Listener/ContainerHost.cs#L637

the other error: Value cannot be null. (Parameter 'key') - i have no idea what the origin is

the message being sent is very simple:

const { ServiceBusClient } = require('@azure/service-bus')

const connectionString = 'Endpoint=sb://sbemulator/;SharedAccessKeyName=all;SharedAccessKey=CLwo3FQ3S39Z4pFOQDefaiUd1dSsli4XOAj3Y9Uh1E=;EnableAmqpLinkRedirect=false'

const test = async () => {
  const sbClient = new ServiceBusClient(connectionString)
  const sender = await sbClient.createSender('some-topic', 'some-subscription')
  const msg = {
      body: {test: 1, something: 'interesting'},
      subject: 'test subject',
      contentType: 'application/json',
  }

  await sender.sendMessages([msg])
}

test()

my setup is like this:

contents of the docker-compose dependencies file (.ado/docker-compose.common.yml):

version: '3.9'
services:
  rabbit:
    container_name: "rabbit"
    image: rabbitmq:3.11.6-management
    ports:
      - "15672:15672"
      - "5672:5672"
    healthcheck:
      test: rabbitmq-diagnostics check_port_connectivity
      interval: 5s
      timeout: 60s
      retries: 20

is there any special requirements that i am missing? maybe i messed up the setup? how likely is the problem related to the cacert.cer?

other than that i just wanted to say that this service-bus mocking opportunity is something that i am really excited about, hopefuly we can somehow pinpoint and resolve the issues so that it works out of the box. congrats on the idea.

cheers rafal.

piotr-rojek commented 1 year ago

Hey guys, I am not a node.js wizard myself :( Would one of you be so kind and upload an archive with a ready-to-go project that I could use for troubleshooting? :)

Since it looks like a classic "Object reference not set to an instance of an object." exception, I would probably like to just run the whole thing in Visual Studio on my end, then start the sample node.js app, wait for the breakpoint and go from there ;)

rafak commented 1 year ago

@piotr-rojek i uploaded a "ready to run" package (link below). intructions:

NOTE: you can inspect the package.json file - the scripts section has the docker build and run scripts if you run into any trouble.

the following env variables are expected or available:

NOTE: you can change the variable values passed to the container in the .env file OR in the docker-compose.node.yml file OR by setting them via command line CONNECTION_STRING=.... TOPIC=... SUBSCRIPTION=... docker-compose up

i only tested running the scripts (using npm run ... ) and locally (running npm install and npm start)

after running the package you will have an api endpoint exposed /publish that should work with the connection string and when you POST data to it i will try to publish the message to the topic/subscription that you point it to via mentioned env variables.

example (here i am running this on a VM via docker-compose and exposing the api via port 3000 by default):

curl --request POST \
  --url http://192.168.88.32:3000/publish \
  --header 'Content-Type: application/json' \
  --data '{
  "test": "123"
}'

NOTE: you might need to add your sbemulator service to the docker compose file or have it run in the same docker network, but let's not get ahead of ourselves: try to run this and please let me know if you run into any trouble so that i can help.

NOTE: i did not add any ENV or ARG to the Dockerfile, but maybe the mentioned env vars should be there for readability?

and now the link:

sbemu-node (click download)

cheers, hopefuly this makes sense rafal.

piotr-rojek commented 1 year ago

@rafak thanks! I am getting "File not found or corrupt" when trying to download though ;(

WayneDT commented 1 year ago

As per our QnA discussion I thought I would post my simple set up here for you. (sample comes from https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/servicebus/service-bus/samples/v7/javascript) I followed your setup instructions to run the docker images.

  1. make sure node is installed (I am using npm and windows)
  2. create a client folder for the nodejs project
  3. cd to the folder and run npm init accept all the defaults or paste the following into a package.json 
    {
"name": "sbe-testclient",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"start": "node index.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"@azure/service-bus": "^7.9.0",
"dotenv": "^16.0.3"
}
}
  4. set the certificate so that node knows where it is, on the command prompt set NODE_EXTRA_CA_CERTS=C:\\localpath\\devopsifyme-sbemulator\\docker\\rabbitmq-amqp1\\testca\\cacert.pem
  5. run npm install
  6. create a index.js file and paste the following in 
    
const { ServiceBusClient } = require("@azure/service-bus");

// Define connection string and related Service Bus entity names here const connectionString = "Endpoint=sb://localhost/;SharedAccessKeyName=all;SharedAccessKey=CLwo3FQ3S39Z4pFOQDefaiUd1dSsli4XOAj3Y9Uh1E=;EnableAmqpLinkRedirect=false"; const queueName = "testQ";

async function main() { const sbClient = new ServiceBusClient(connectionString);

// createSender() can also be used to create a sender for a topic. const sender = sbClient.createSender(queueName);

try { // Send a single message console.log(Sending one scientists); const message = { contentType: "application/json", subject: "Scientist", body: { firstName: "Albert", lastName: "Einstein" }, timeToLive: 2 60 1000, // message expires in 2 minutes }; await sender.sendMessages(message);

// Close the sender
console.log(`Done sending, closing...`);
await sender.close();

} finally { await sbClient.close(); } }

main().catch((err) => { console.log("sendMessages Sample: Error occurred: ", err); process.exit(1); });

module.exports = { main };



7. execute the script `npm run start`
rafak commented 1 year ago

@piotr-rojek sorry, trying out new stuff, seems like it disappeared,

try this one (we transfer)

piotr-rojek commented 1 year ago

Hey guys, could you please verify the just released v0.1.9? I tested it with the provided code above, so at least sending and receiving should work fine. Turned out it was missing some fields in the message that the SDK expected. Also, one expected field was not sent by node.js to the emulator on AMQP level.

Reach out if there are more issues to address! Thanks for the code snippets, they helped me a lot! Much appreciated ;) It would probably be beneficial to include integration tests for nodejs - let me know if someone is keen on helping with that :)

/Piotr

rafak commented 1 year ago

will test, we can work on some tests too.

rafak commented 1 year ago

@piotr-rojek looks much better, the errors are not showing, i am getting something different now but it must be the cacert setup

{
    "err": {
        "code": "UNABLE_TO_VERIFY_LEAF_SIGNATURE"
    }
}
sbemu-node    |     {"msg":"service bus error","data":{"error":{"code":"UNABLE_TO_VERIFY_LEAF_SIGNATURE"},"errorSource":"receive","entityPath":"test-topic/Subscriptions/test-subscription","fullyQualifiedNamespace":"sbemulator","identifier":"test-topic/Subscriptions/test-subscription-1c505ceb-8c26-6e49-8e8b-94818b6f3540"}}

i will try to work on the certs, i guess i need to add the cacert to the sbemu-node image, will let you know.

rafak commented 1 year ago

@piotr-rojek i got it to work !!! yay!

setting NODE_TLS_REJECT_UNAUTHORIZED: 0 got rid of the error message

how would you like the integration tests to be delivered/designed?

piotr-rojek commented 1 year ago

Great news ;)

I would prefer to handle "smoke test" level scenarios for integration tests. We could add a container running tests to the docker-compose.integration.yml file. Existing c# tests simply send a message and assert that it can be received.