scoping fault tolerance strategies

[Ladicek]

Currently, all fault tolerance strategies are global (singleton). This includes stateful strategies such as circuit breaker or bulkhead (or hypothetical rate limit, #437).

But sometimes, you want to apply certain strategies separately for each user (or IP address, or something like that).

For example, consider this article on rate limiting: https://stripe.com/blog/rate-limiters It describes "request rate limiters" and "concurrent requests limiters" (and some other strategies I'm going to ignore here). The request rate limiters are essentially #437 scoped per user, and concurrent request limiters are bulkheads, again, scoped per user.

Adding a generic mechanism for scoping all strategies that apply to given method would enable creating these strategies rather easily. (It would also enable scoped circuit breakers, which I'm not sure are useful, but I'm also not sure if they are useless.)

Usage would be something like @ScopedFaultTolerance(PerUser.class), where PerUser would be a type of a bean that implements a strategy interface. Something like:

public interface FaultToleranceScopeLookup {
    String lookup();
}

Open questions:

1. Should the FT scope be represented by a String? It needs to be an object with equals and hashCode.
2. Should the lookup method be asynchronous? (And should it return CompletionStage or Uni?)
3. Isn't it too limiting to define a single scope for all strategies on one method? In other words, would it make sense for a circuit breaker on one method to require a different scope than a rate limiter on the same method?

[oaklandcorp-jkaiser]

I stumbled upon another Quarkus extension with rate limiting functionality. It provides per user rules by overriding a single interface. The extension doesn't seem to provide as many rules as your, but it should add some weight to the validity of the approach you suggested. https://docs.quarkiverse.io/quarkus-bucket4j/dev/index.html#_population_segmentation

[Ladicek]

Yeah, String is probably the first choice anyone would make. I'm not sure if it's the best choice, but it has all the nice qualities like: instantly familiar, immutable, usable as a key to all the common data structures...

I think it's either String, or something like this:

public final class Key {
    private final byte[] value;

    public static Key of(String value) {
        return new Key(value == null ? null : value.getBytes(StandardCharsets.UTF_8));
    }

    public static Key of(byte[] value) {
        return new Key(value);
    }

    private Key(byte[] value) {
        this.value = value;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (!(o instanceof Key)) return false;
        Key key = (Key) o;
        return Arrays.equals(value, key.value);
    }

    @Override
    public int hashCode() {
        return Arrays.hashCode(value);
    }
}

I don't think I can figure out anything better.

[oaklandcorp-jkaiser]

Just my two cents, but I'd vote to just keep it simple. IP address and JWT subject are the main use-cases in my world today and both naturally fit strings, are request-scoped, and are obtainable just through synchronous methods. If I was the author I'd only stretch for the more complicated implementation if I had real world use-cases that require it or maybe if peer library projects (maybe in the smallrye or Quarkus community?) that a common approach/pattern.