Question: Create Functions dynamically

[ModernRonin]

Hi,

I'm not certain whether this is the right place to ask, if not, please point me to in the right direction.

I find myself in a situation where I create a lot of functions that look like

 public class GetTenants
    {
        readonly TenantHandler _handler;

        public GetTenants(TenantHandler handler) => _handler = handler;

        [FunctionName(nameof(GetTenants))]
        public Task<IActionResult> RunAsync(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = "tenants")]
            HttpRequest request) =>
            _handler.GetAsync(request);
    }

or

public class OnQueueMessage : AnOnQueueMessageFunction
    {
        public OnQueueMessage(IMessageReceiver receiver) : base(receiver, MetaDataConstants.QueueName) {}

        [FunctionName(nameof(OnQueueMessage))]
        public Task RunAsync([ServiceBusTrigger(MetaDataConstants.QueueName)]
            Message message,
            CancellationToken cancellationToken) =>
            HandleAsync(message, cancellationToken);
    }

As you can see I have already eliminated boilerplate as much as possible. However, because of the need to use attributes to register and bind the functions, I can go no further.

On the other hand, somehow the Functions runtime must detect these and register them for use internally. So one should think there must be an API that would allow me to say something like (pseudo-code):

runtime.Register(typeof(GetTenants), nameof(GetTenants))
            .Method(f => f.RunAsync)
            .UseHttpTrigger(AuthorizationLevel.Function, "get", Route="tenants");

runtime.Register(typeof(OnQueueMessage), nameof(OnQueueMessage))
            .Method(f => f.RunAsync)
            .UseServiceBusTrigger(MetaDataConstants.QueueName);

With some API like this, for the OnQueueMesssage Function which I use in several Function app projects (per microservice), I could just do the register line and even that I could refactor into a common module. And for stuff like GetTenants, I could probably generate them at runtime from other types I already go, again eliminating boilerplate.

Is there anything like that? Or if not, are you planning on exposing something of the kind?

Thanks a lot, MR

[brettsam]

This isn't something that we currently support. You're correct that there are some types internally that we use to discover and register functions, but I'm not sure that we've ever had plans to allow for dynamically registering them like you're proposing. I'll move this to the correct repo and tag it as a feature request.

[ModernRonin]

Even if you could just make your internally used register functions public (and give a hint which they are), this would already be great - then, if needed, we could wrap that API into whatever more comfortable form desired.

[danspark]

Maybe you should look into Function Monkey. I haven't used it myself, but I've done some reading about it, and it allows you to register your functions on startup with a "MediatR"-style programming model.

[Afischbacher]

Hey, I know this question is a bit old, but I have done something similar to what you are looking for and it can be done, this was my approach.

I was able to complete this using T4 templates , Generics, Reflection and a dependency injection library. This enabled me to dynamically template all the functions I needed when building my solution. From there you can then fill in the bodies of your functions any way you would like to with some additions to your templates. Here is a snippet of code as an example.

<#
foreach(var registration in registeredFunctions)
{#>

        public static class <#=registration.Type.Name#>Function
        {

            [FunctionName("<#=registration.Type.Name#>Function")]
            public static async Task Run
            (
                [HttpTrigger(AuthorizationLevel.Function, "<#=registration.httpMethod#>", Route ="<#=registration.httpMethodRoute#>")] HttpRequest request)
            )
            {
                try
                {
                    // Template code here
                }
                catch (Exception exception)
                {
                      // Handle exception here
                }
            }
        }
<#}#>

[mattc-eostar]

Even something like this would very helpful and wouldn't require templating or a new api https://danielk.tech/home/net-core-api-server-how-to-create-a-base-crud-controller

[BrianVallelunga]

Our need is essentially the same. On startup, we want to create functions that listen to a set of queues based on the tenants we have. Being able to dynamically register our own functions would be perfect for this. We can't do this with a T4 template, because this isn't known at compile time.

[BrunoJuchli]

@BrianVallelunga And what would you do when you get a new tenant? Restart?

(I'm contemplating a similar problem and it seems to me it would rather require something more of a "meta" kind of functionality, where not just functions are being added but queues (or what ever tenant specific resources are needed), too.)

[BrianVallelunga]

New tenants are only a few times a month, so yes, restarting would be totally fine. I could even schedule a restart or have our provisioning system perform that job. And yes, we'd have the provisioning system create the queues, databases, etc as well.

[mazzaferri]

I have a similar requirement to process the Cosmos DB change feed for a set of per-tenant containers. The handler logic is the same for all containers but the Cosmos DB trigger is container-specific so we have to create a new Function for each tenant simply to define their trigger.

In this case the issue could be more or less solved if a database-wide trigger was offered (independent of container) provided the handler can determine the container for each trigger invocation, but it would be great to be able to process our config on startup and dynamically register a handler Function for each tenant's container.

[PequeChianca]

Adding more requesters for this feature, I also have same needs, an API where we could add new functions at runtime. I would like to share a common library for many functions that would reuse the implementation for communicate with the ServiceBus and other Http Requests that these functions are responsible to do.

[andsj073]

+1 We have a multi-tenant solution and would like the availability to Functions dynamically.

One special case is to be able to add Timer triggers to one function, but of different cron configurations, dynamically as determined by a user in front of a user interface.

[andsj073]

There is in fact a work around that does this, i.e. adding a trigger binding dynamically. It requires that you upload you binaries with FTP to the wwwroot folder of the site (no package deploy), and then create individual specific function.json file for each new trigger you want, all pointing to a specific method as entry point for execution.

Like below. It is for adding dynamic Timer triggers, but can easily be adapted to add for example Service Bus triggers on new queue names dynamically.

To be generated in a new uniquely named folder under wwwroot (folder name will be the name of the Function): function.json

  {
    "scriptFile": "../bin/FunctionTriggersMethods.dll",
    "entryPoint": "Company.DynamicTimerTrigger.Run",
    "bindings": [
      {
        "type": "timerTrigger",
        "direction": "in",
        "name": "timer",
        "schedule": "0 */5 * * * *"
      },
      {
        "type": "table",
        "direction": "in",
        "name": "tableConfInput",
        "tableName": "%TABLENAME%",
        "partitionKey": "TRIGGERCONFIG",
        "rowKey": "{sys.methodname}",
        "connection": "MGD_IDENTITYBASED_CONNECTION"
      },
      {
        "type": "blob",
        "direction": "out",
        "name": "blobOutput",
        "path": "%BLOB_CONTAINERNAME%/{rand-guid}",
        "connection": "MGD_IDENTITYBASED_CONNECTION"
      },
    ],
    "disabled": false
  }

The code of FunctionTriggersMethods.dll: Upload, with everything else in you bin/Debug/bin folder after build, to wwwroot/bin of your Function App using FTP. DynamicTimerTrigger.cs

using System;
using System.IO;
using System.Text;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Azure.Messaging.ServiceBus;
using Azure.Storage.Blobs.Specialized;

namespace Company.Run
{

    public static class DynamicTimerTrigger
    {

        //[FunctionName(nameof(TimerTrigger))]
        public static void Run(
            //[TimerTrigger("0 */5 * * * *")] 
            TimerInfo timer,
            //[Table("%TABLENAME%", "TRIGGERCONFIG", "{context.FunctionName}", Connection = Trigger.CONNECTION_NAME_PREFIX)] 
            TriggerConfig tableConfInput,
            //[Blob("%BLOB_CONTAINERNAME%/{rand-guid}", FileAccess.Write, Connection = Trigger.CONNECTION_NAME_PREFIX)] 
            BlockBlobClient blobOutput,
            ExecutionContext context,
            ILogger logger)
        {
            string triggerEventId = blobOutput.Name;

            Trigger.HandleTriggerEventAsync(
                triggerEventId,
                tableConfInput,
                blobOutput,
                logger,
                new MemoryStream(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(timer))),
                System.Net.Mime.MediaTypeNames.Application.Json).GetAwaiter().GetResult();

        }

    }
}

You might need to trigger a sync of trigger as well. You can on /sync-function-triggers on how to do that.

Issues with this solution is that since you're not using Package Deploy, your binaries will be locked when the Function App i started, and hence upgrades will lead to longer and more cumbersome down-time than otherwise.

I have thought about whether the process above might work with web deploy, instead of FTP, as well, but have not tried.

@ModernRonin @PequeChianca @mazzaferri @BrianVallelunga would be nice to know if this works for you and if you have any enhancements or new ideas. interested in improving this concept for my own needs

[lgostling]

+1 I have a service which needs to run essentially the same queue triggered function from a subset of a large set of possible queue names. Currently I have to manually define a function binding for all possible queue names, but I would much rather read a subset of the queue names from a configuration file and then dynamically generate bindings for those queues.

[AMeyerInkSoft]

When my team moved from on-premise to Azure, I chose the in-process function app model as a replacement for an old, obscure Java-based cron job scheduler. What I liked about the Azure Functions in-process model apps by comparison to something like Hangfire or Quartz is that it came with a standard UI in the Azure portal for adjusting function params and inspecting logs in cases where we didn't want to manage a particular function instance or function property in git, or we just wanted to make a temporary ad-hoc change without having to PR it. With the in-process model, I was able to define functions like the examples below and then use the Azure portal Code+Test UI for some types of changes. What I like most about this was:

• The Code+Test UI for a single function.json is a lot more organized than the one dimensional app settings slosh, and it (sort of) allowed for comments.
• We could elect to leave some function definitions or temporary changes out of git, and we could make quick or emergency ad hoc changes without a PR and publish process.

If someone could figure out a way to achieve similar behavior in (.Net8 Linux) isolated mode, that would be great.

For people just skimming this thread they found via Bing, sorry to disappoint, but...

This is how I did it in in-process mode. Not a recipe for isolated mode:

./SomeFunctionName/function.json (not managed in git)

{
  // Arbitrary comment for team members about what this does.
  "Url": "https://SomeInternalServer.com/SomeHttpTrigger",
  "AribitraryCustomProperty": 1234,
  "Timeout": "00:10:00",
  "TeamsRecipientOnFailure": "SomeChannelName",
  // Arbitrary comment for team members about scheduling, etc.
  "bindings": [
    {
      "schedule": "0 0 7 * * *",
      "type": "timerTrigger",
      "direction": "in",
      "useMonitor": true,
      "runOnStartup": false,
      "name": "timerInfo"
    }
  ],
  "scriptFile": "../ScheduledHttpRequest.csx"
}

./ScheduledHttpRequest.csx

#r "bin\Jobs.dll"
// It's possible to run the ScheduledHttpRequest.RunAsync method directly from the function.json file, but the runtime complains about configurationSource consistency, so creating this one liner .csx to get rid of the warning.
public static async Task Run(TimerInfo timerInfo, ILogger logger, ExecutionContext executionContext, System.Threading.CancellationToken cancellationToken) => await MyCompanyNamespace.Jobs.ScheduledHttpRequest.RunAsync(null, logger, executionContext, cancellationToken);

Jobs.dll ScheduledHttpRequest.cs

using Microsoft.Extensions.Logging;

namespace MyCompanyNamespace.Jobs;

/// <summary>
/// Called by misc. one-liner run.csx functions to query a given URL at a specified interval.
/// </summary>
public static class ScheduledHttpRequest
{
    /// <summary>
    /// Corresponds to a function's functions.json file.
    /// </summary>
    private class FunctionJson
    {
        /// <summary>
        /// The web address to query with a given schedule.
        /// </summary>
        public string Url {get; set;} = null!;

        /// <summary>
        /// The max amount of time we are willing to wait for our call to <see cref="Url"/> to complete.
        /// </summary>
        public TimeSpan Timeout {get; set;}
    }

    /// <summary>
    /// Ideally, the HTTP client could/should be accessed as a singleton setup via hostBuilder.Services.AddHttpClient() and then passed via DI, but we have some DI limitations when we instantiate many copies of this function.
    /// </summary>
    public static HttpClient HttpClient{get; set;} = null!;

    /// <summary>
    /// Queries the Url given either as a param or via a function's function.json file at the given time interval also set in the functions.json.
    /// </summary>
    public static async Task RunAsync(string? url, ILogger logger, Microsoft.Azure.WebJobs.ExecutionContext executionContext, CancellationToken cancellationToken)
    {
        var functionJson = (await System.Text.Json.JsonSerializer.DeserializeAsync<FunctionJson>(
            File.OpenRead(Path.Combine(executionContext.FunctionDirectory, "function.json")),
            new System.Text.Json.JsonSerializerOptions{ReadCommentHandling = System.Text.Json.JsonCommentHandling.Skip},
            cancellationToken
        ))!;
        url ??= functionJson.Url;
        logger.LogInformation("Calling {Url}", url);
        var getTask = HttpClient.GetAsync(url, cancellationToken);

        if (functionJson.Timeout > TimeSpan.Zero && functionJson.Timeout < HttpClient.Timeout)
            getTask = getTask.WaitAsync(functionJson.Timeout, cancellationToken);

        try
        {
            var response = await getTask;
            logger.LogInformation("Completed {Url}, Status: {StatusCode}, Content: {Content}", url, response.StatusCode, await response.Content.ReadAsStringAsync(cancellationToken));
        }
        catch (TaskCanceledException ex)
        {
            logger.LogError(ex, "Call to {Url} was aborted, possibly because of app restart.", url);
            throw;
        }
        catch (TimeoutException ex)
        {
            logger.LogError(ex, "Call to {Url} timed out due to configured timeout of {functionJsonTimeout}.", url, functionJson.Timeout);
            throw;
        }
    }
}

As another side note, I tried augmenting the app config on startup and using %expressions% in the trigger as well, but I ran into the problem discussed at https://github.com/Azure/azure-functions-dotnet-worker/issues/1238, which is also still open via another linked issue.