Azure / azure-functions-durable-extension

Durable Task Framework extension for Azure Functions
MIT License
715 stars 271 forks source link

COnfiguration guide for the local emulator #1934

Closed zippy1981 closed 3 years ago

zippy1981 commented 3 years ago

I would like to use the local emulator for local development, but docs on this repo are lacking.

This doc should how to use hosts.json to configure the netherite and mssql extensions. I tried guessing, but nothing is working. Is there any documentation for using it?

cgillum commented 3 years ago

Moving this issue to the correct repo (Durable Functions).

cgillum commented 3 years ago

We don't have support for the DTFx local emulator in Durable Functions today. However, Netherite does support in-memory emulation. That might be a preferred option since it's the same core engine that you could use in production. Docs here: https://microsoft.github.io/durabletask-netherite/#/emulation

FYI @sebastianburckhardt

sebastianburckhardt commented 3 years ago

The docs could perhaps be more to the point, I will update them. Here is the short version.

  1. Add the NuGet package Microsoft.Azure.DurableTask.Netherite.AzureFunctions to your function app project.
  2. Set the AzureWebJobsStorage configuration parameter to use local emulation (e.g. set AzureWebJobsStorage=UseDevelopmentStorage=true;).
  3. Set the EventHubsConnection configuration parameter to use local emulation (e.g. set EventHubsConnection=Memory).
  4. Modify the host.json to choose Netherite as the storage provide:
    {
    "version": "2.0",
    "extensions": {
    "durableTask": {
      "storageProvider": {
        "type": "Netherite"
      }
    }
    }
    }

    Let me know if you run into trouble with these steps or if something is unclear.

ghost commented 3 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

jdearing-neudesic commented 3 years ago

I used the "long" development connection string because I'm using azurite and don't want to start the emulator and that basically works. However, its not really solving my problem. I have some functions that use regular queue triggers that are still need azurite or storage emulator or real azure storage.

I was hoping for something completely in process to eliminate the need for even azurite for local development. If I converted all my queue triggers to durable orchestrations could I set UseDevelopmentStorage=false and not need any external azure storage, storage emulation or azurite?

Yes this is for development only.

zippy1981 commented 3 years ago

The above was my work account, consider that my author feedback, thanks.

ConnorMcMahon commented 3 years ago

@zippy1981

Just out of curiosity, what exactly is the reason that you want to remove dependency on Azure Storage emulator and/or Azurite?

We could theoretically add support for the DTFx.Emulator backend, but that would be a decent amount of work to document that use case + any dev work to light it up, just to avoid having to start up the storage emulator.

In general, the different backends may have slightly different quirks and performance characteristics, so I don't know if I would recommend testing on another backend then you are planning to run in prod.

ghost commented 3 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

jdearing-neudesic commented 3 years ago

@ConnorMcMahon Honestly, at one particular client situation I have to use a very locked down laptop and run azurite on a different port in order for things to just work. However, since I can't run azurite automatically on startup like Storage emulator, or install docker to run azurite in the background, its a manual process to run a console app.

So it would be nice to have one less thing to configure, and one less console window to open in addition to pressing F5 in visual studio.

But if there was a csproj option to run azurite on a non-standard port and not have it run in a console window (maybe spewing out to an output window in visual studio) and it didn't require docker, then that would actually make my life easier in this very locked down situation.

ghost commented 3 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

sebastianburckhardt commented 3 years ago

For what it is worth, I believe Azurite does indeed support configuring the port.

I haven't tried it myself but there is documentation here: https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio#blob-listening-port-configuration

ghost commented 3 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.