Prefix Generation Strategy

[mohammed-ehsan]

I am currently implementing a solution that automates the process of saving models in Redis through event-sourcing.

The challenge is to make setting the prefixes dynamic. I have created a special implementation that generates ids, and does not inherit from RedisOM IdGenerationStrategy.

This special generator is also responsible for creating the prefixes, however, there is no way for me to set the prefixes dynamically. I was able to workaround the limitations in many cases, however, these limitations are breaking our ability to build indexes with the CreateIndexAsync method.

My suggestion is to refrain from using attributes and rather use abstract class that holds the prefix. Thus, giving the developers the ability to manage prefixes dynamically.

I would love to hear back and collaborate with you to implement such behavior.

[slorello89]

Hi @mohammed-ehsan,

So the reason the prefixes are an important part of the document decoration is because it's the prefixes that define how the index is built (if you don't specify a prefix it will just try to index everything in Redis.

The idea here is that there is generally a 1:1 relationship between models and indexes and you want to establish all the meta-data for the index in the DocumentAttribute of the model class.

Index/Model generation using abstract classes isn't something you generally see in .NET, as this type of work is typically done declartively (through attributes/decorations on classes & properties).

I'm curious. Have you tried decorating your child-classes with a Document attribute and giving them the desired prefix?

For example, you might have a vehicle Model:

[Document(StorageType = StorageType.Json, Prefixes = new string[]{"Bike", "Car", "Boat"})]
public class Vehicle
{
    [RedisIdField] public string Id { get; set; }

    [Indexed] public string Name { get; set; }

}

and you can have a car model

[Document(StorageType = StorageType.Json, Prefixes = new []{"Car"}, IndexName = "vehicle-idx")]
public class Car : Vehicle
{    
}

that has the appropriate prefix and leverages the same index,

then you can just query the car model as needed:

var cars = provider.RedisCollection<Car>();

foreach (var car in cars.Where(x => x.Name == "tesla"))
{
    Console.WriteLine(car.Id);
}

[mohammed-ehsan]

Hi @slorello89, Really appreciate your response! Let me better demonstrate our use case with more concrete example. Your work has saved us a great hustle in many cases! So thank you for that!

Now, we are building a product that utilizes Redis as the read side from CQRS POV. The write side uses Postgres with Event Sourcing. The framework that we build listens to domain events and builds Json Documents in Redis automatically.

The concept that we had in mind when we started implementing this project is to provide a base class that developers can inherit from which will eliminate the need of developers to add the attribute or even know that there is Redis from behind the scenes. For instance, this is the IdGenerationStrategy that we developed:

public class DefaultLinkedIdConstructionStrategy<TRedisLinkedModel, TModel, TId> : ILinkedIdConstructionStrategy<TRedisLinkedModel, TModel, TId>
        where TModel : EntityBase<TId>
        where TId : IComparable, IComparable<TId>, IEquatable<TId>
        where TRedisLinkedModel : IRedisModel
    {
        private readonly IConfiguration _configuration;

        /// <inheritdoc/>
        public string Prefix { get; } = null;

        public DefaultLinkedIdConstructionStrategy(IConfiguration configuration)
        {
            _configuration = configuration;
            var redisConfig = _configuration.GetRedisConfig();
            Prefix = redisConfig.KeyPrefix;
        }

        /// <inheritdoc/>
        public string Construct(TId sourceId)
        {
            var builder = new StringBuilder();
            if (!string.IsNullOrEmpty(Prefix))
            {
                builder.Append(Prefix.Trim());
            }
            builder.Append(typeof(TModel).Name.ToLowerInvariant());
            builder.Append(':');
            builder.Append(sourceId.ToString().ToLowerInvariant());
            return builder.ToString();
        }

        /// <summary>
        /// Deconstruct Redis key into <typeparamref name="TId"/>, currently supported
        /// <typeparamref name="TId"/> types are: <see cref="Guid"/>, <see cref="long"/>,
        /// and <see cref="int"/>
        /// </summary>
        /// <param name="key">Redis key.</param>
        /// <returns></returns>
        public TId Deconstruct(string key)
        {
            if (typeof(TId).IsAssignableFrom(typeof(Guid)))
            {
                return (TId)Convert.ChangeType(Guid.Parse(key.Split(':').Last()), typeof(TId));
            }
            if (typeof(TId).IsAssignableFrom(typeof(long)))
            {
                return (TId)Convert.ChangeType(long.Parse(key.Split(':').Last()), typeof(TId));
            }
            if (typeof(TId).IsAssignableFrom(typeof(int)))
            {
                return (TId)Convert.ChangeType(int.Parse(key.Split(':').Last()), typeof(TId));
            }
            throw new UnsupportedIdTypeException($"No deconstruction logic for type {typeof(TId).Name}");
        }
    }

Now, by defining which Postgres models we want to have this Redis Model linked to, we can dynamically build the full key, with conjunction of provided top-level prefix.

Top-level prefix is provided by configuration, that should be shared across all models, as it is used by Redis Cluster to provide namespace isolation, which enables having multiple APIs using the same cluster without interfering with each other through ACL. One of the issues with the DocumentAttribute approach is that, if we need to change that top-level prefix, we would need to do a full rebuild and redeployment, which is a sub-optimal solution.

Additionally, we want client application to be agnostic to Redis Keys structure, and rather send standard Guid/Long Ids that are generated by Postgres, then, IdGenerationStrategy would manage the construction of full Redis keys.

This whole concept is inspired mainly by Entity Framework Core Fluent API, and how entities can be configured through registration of configuration classes.

I would say that, this project can support both approaches, as for simple use cases, developers can use attributes, where for more advanced functionality, a different dynamic approach might be supported.

What are your thoughts? I am willing to provide more details, if above explanation is not sufficient.