search

Azure / azure-event-hubs-emulator-installer

This repository acts as a central hub for handling issues, collecting feedback, and facilitating discussions related to the Azure Event Hubs emulator.
https://learn.microsoft.com/azure/event-hubs/overview-emulator
MIT License
26 stars 9 forks source link

How to connect local function `func start`? What's the value of SharedAccessKey=SAS_KEY_VALUE #16

Open ranma2913 opened 3 months ago

ranma2913 commented 3 months ago

I'm developing a Javascript v4 function. It's working fine in cloud, but on local I'm not able to connect to my docker-compose started Event Hub Emulator.

I have Questions:

I'm using the connection string provided in the documentation Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true; but get a connection refused: 

Functions:

        MyEventHubTrigger: eventHubTrigger

For detailed output, run func with --verbose flag.
[2024-06-06T16:52:49.253Z] Host lock lease acquired by instance ID '000000000000000000000000B4975DB8'.
[2024-06-06T16:52:56.304Z] EventProcessorHost error (Action='Retrieving list of partition identifiers from a Consumer Client.', HostName='52656bb6-5546-409b-a571-14550d79f670', PartitionId='').
[2024-06-06T16:52:56.304Z] System.Private.CoreLib: Connection refused.
[2024-06-06T16:52:56.313Z] The listener for function 'Functions.MyEventHubTrigger' was unable to start.
[2024-06-06T16:52:56.313Z] The listener for function 'Functions.MyEventHubTrigger' was unable to start. System.Private.CoreLib: One or more errors occurred. (Connection refused). System.Private.CoreLib: Connection refused.

Tagged in Redit Question here

Saglodha commented 3 months ago

@ranma2913 - You don't need to change anything for SharedAccessKey=SAS_KEY_VALUE. Other questions in your issue seems related to #15, can you review it to get more details on local.settings.json?

Additionally, I have similar questions on your setup- Where are the local function and emulator running and how does the networking topology look like?

ranma2913 commented 3 months ago

I'm starting the azure emulator with docker-compose like so: image

docker-compose.yaml

name: microsoft-azure-emulators
services:
  emulator:
    container_name: "eventhubs-emulator"
    image: "mcr.microsoft.com/azure-messaging/eventhubs-emulator:latest"
    volumes:
      - "./eventhubs_emulator_config.json:/Eventhubs_Emulator/ConfigFiles/config.json"
    ports:
      - "5672:5672"
    environment:
      BLOB_SERVER: azurite
      METADATA_SERVER: azurite
      CONFIG_PATH: '/Eventhubs_Emulator/ConfigFiles/config.json'
      ACCEPT_EULA: 'Y'
    depends_on:
      - azurite
    networks:
      eh-emulator:
        aliases:
          - "eventhubs-emulator"
  azurite:
    container_name: "azurite"
    image: "mcr.microsoft.com/azure-storage/azurite:latest"
    ports:
      - "10000:10000"
      - "10001:10001"
      - "10002:10002"
    networks:
      eh-emulator:
        aliases:
          - "azurite"
networks:
  eh-emulator:

eventhubs_emulator_config.json

{
  "UserConfig" : {
    "NamespaceConfig" : [
      {
        "Type" : "EventHub",
        "Name" : "emulatorNs1",
        "Entities" : [
          {
            "Name" : "call-summary",
            "PartitionCount" : "2",
            "ConsumerGroups" : [
              {
                "Name" : "acet"
              }
            ]
          }
        ]
      }
    ],
    "LoggingConfig" : {
      "Type" : "Console,File"
    }
  }
}

And then I start the local function:

cd CallSummaryFunctionApp/
func start --verbose

image

BTW the local.settings.json =

{
  "IsEncrypted" : false,
  "Values" : {
    "AzureWebJobsStorage" : "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME" : "node",
    "SUMMARY_EVENT_HUB_NAME" : "call-summary",
    "SUMMARY_EVENT_HUB_CONNECTION" : "Endpoint=sb://localhost/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
  }
}
bitIO commented 3 months ago

Hi, just my 2 cents.

From what I saw in my PoC using node is that coreamqp library is connecting to 5671 port (TLS port for amqp if I'm not wrong) which is not exposed by the docker compose. Adding it to the docker compose turns into a connection reset error.

Using the customEndpointAddress: "http://localhost:5672", in my EventHubProducerClient also results into a Failed to connecterror.

ranma2913 commented 3 months ago

I've added PR- #20 with my trial project. I've also tried explicitly installing the package Microsoft.Azure.WebJobs.Extensions.EventHubs as described in #15

phs18 commented 3 months ago

@ranma2913 did your issue resolve and were you able to connect

ranma2913 commented 3 months ago

@phs18, sadly no I've not make any progress. Sample-Code-Snippets/JavaScript/EventHubTrigger-FunctionApp/src/functions/MyEventHubTrigger.js still sits without anyone from Azure getting a chance to look into this.

kyrre commented 3 months ago

I'm getting the exact same problem with a Python function. This is on M2.

eventhubs-emulator  | [11:38:11 WRN] <Trc Id="60006" Ch="Debug" Lvl="Warning" Kw="1000000000011110" UTC="2024-07-02T11:38:11.748Z" Msg="An exception was handled at YF. Exception message ExceptionId: e18c23ce-977e-486f-a4c7-e15654774ed6-System.OperationCanceledException: The operation was canceled.&#xA;   at E.EW.C[TAsyncResult](IAsyncResult)&#xA;   at g.gn.A.A(IAsyncResult)&#xA;   at g.gn.A(IAsyncResult)&#xA;   at Y.YF.A.a(IAsyncResult)&#xA;   at u.uL.A.A.a(A, IAsyncResult)&#xA;   at C.CJ`1.a(IAsyncResult)." />
eventhubs-emulator  | [11:38:11 WRN] <Trc Id="60006" Ch="Debug" Lvl="Warning" Kw="1000000000011110" UTC="2024-07-02T11:38:11.760Z" Msg="An exception was handled at Yg. Exception message ExceptionId: a21c0fca-17c6-4950-82ed-19ba6c2cde02-System.OperationCanceledException: The operation was canceled.&#xA;   at E.EW.C[TAsyncResult](IAsyncResult)&#xA;   at g.gn.A.A(IAsyncResult)&#xA;   at g.gn.A(IAsyncResult)&#xA;   at Y.Yg.A.a(IAsyncResult)&#xA;   at u.uL.A.A.a(A, IAsyncResult)&#xA;   at C.CJ`1.a(IAsyncResult)." />
eventhubs-emulator  | [11:38:14 WRN] <Trc Id="33029" Ch="Debug" Lvl="Warning" Kw="1000000000000008" UTC="2024-07-02T11:38:14.099Z" />
azurite             | 172.25.0.3 - - [02/Jul/2024:11:38:14 +0000] "PUT /devstoreaccount1/emulatorblobcontainer-e5bea00f9d744f21bed9928094d37424/INFO.ext?timeout=118 HTTP/1.1" 201 -
eventhubs-emulator  | [11:38:14 WRN] <Trc Id="60006" Ch="Debug" Lvl="Warning" Kw="1000000000011110" UTC="2024-07-02T11:38:14.368Z" Msg="An exception was handled at YF. Exception message ExceptionId: d63bfc07-8dd3-4a01-a48a-a792c237335f-System.OperationCanceledException: The operation was canceled.&#xA;   at E.EW.C[TAsyncResult](IAsyncResult)&#xA;   at g.gn.A.A(IAsyncResult)&#xA;   at g.gn.A(IAsyncResult)&#xA;   at Y.YF.A.a(IAsyncResult)&#xA;   at u.uL.A.A.a(A, IAsyncResult)&#xA;   at C.CJ`1.a(IAsyncResult)." />
eventhubs-emulator  | [11:38:14 WRN] <Trc Id="60006" Ch="Debug" Lvl="Warning" Kw="1000000000011110" UTC="2024-07-02T11:38:14.368Z" Msg="An exception was handled at YF. Exception message ExceptionId: a2f7e0df-73a8-4d30-8fc7-5f6f335857c1-System.OperationCanceledException: The operation was canceled.&#xA;   at E.EW.C[TAsyncResult](IAsyncResult)&#xA;   at g.gn.A.A(IAsyncResult)&#xA;   at g.gn.A(IAsyncResult)&#xA;   at Y.YF.A.a(IAsyncResult)&#xA;   at u.uL.A.A.a(A, IAsyncResult)&#xA;   at C.CJ`1.a(IAsyncResult)." />
azurite             | 172.25.0.3 - - [02/Jul/2024:11:38:18 +0000] "PUT /devstoreaccount1/emulatorblobcontainer-45582a30ca474b4aa79c4a9e5c603ee5/00000000.ext?comp=page&timeout=2 HTTP/1.1" 201 -
eventhubs-emulator  | [11:38:18 WRN] <Trc Id="60006" Ch="Debug" Lvl="Warning" Kw="1000000000011110" UTC="2024-07-02T11:38:18.903Z" Msg="An exception was handled at YF. Exception message ExceptionId: 7601dede-b77b-437a-91b8-945ed4f5714a-System.OperationCanceledException: The operation was canceled.&#xA;   at E.EW.C[TAsyncResult](IAsyncResult)&#xA;   at g.gn.A.A(IAsyncResult)&#xA;   at g.gn.A(IAsyncResult)&#xA;   at Y.YF.A.a(IAsyncResult)&#xA;   at u.uL.A.A.a(A, IAsyncResult)&#xA;   at C.CJ`1.a(IAsyncResult)." />

edit: this solved it: https://github.com/Azure/azure-event-hubs-emulator-installer/issues/15#issuecomment-2187748574

Saglodha commented 3 months ago

@phs18, sadly no I've not make any progress. Sample-Code-Snippets/JavaScript/EventHubTrigger-FunctionApp/src/functions/MyEventHubTrigger.js still sits without anyone from Azure getting a chance to look into this.

@ranma2913 - I posted a comment on the sample that you provided. Could you please review? Similar to @kyrre, I was also able to run it successfully on my local machine

Saglodha commented 2 months ago

@ranma2913 - could you please confirm if you had a chance to look at above feedback? Are you still facing the issue?

mirkoiv commented 1 month ago

The issue is the same as one I encountered with a Java Spring Azure application. As previously mentioned, an Azure Function attempts to connect to the Event Hubs Emulator on port 5671.

I'm using the emulator to develop both Java Spring Azure application and Azure Functions in TypeScript, but with a reverse proxy in front of the emulator.

For more details about this workaround check this link: https://github.com/mirkoiv/azure-event-hubs-emulator-installer/blob/main/Sample-Code-Snippets/Java/spring-cloud-azure-samples/README.md.

To make this workaround work with Azure Functions, ensure that the generated certificate for the Nginx reverse proxy is added to the computer's trusted root certificate store.