search

thepirat000 / Audit.NET

An extensible framework to audit executing operations in .NET and .NET Core.
MIT License
2.27k stars 324 forks source link

Owned entites not tracked. #696

Open hassanrazakhalid opened 5 days ago

hassanrazakhalid commented 5 days ago

Describe the bug Owned entities when mapped to Json not showing in EventEntry.Changes callback. Is there a way to override changes implementation or wo could a json in oldValue and newValue for complex types.

To Reproduce

  1. Use the below setup
  2. Now whenever i try to update or add ShipperPricing
  3. EventEntry.Changes contain other props but not owned prop.

Expected behavior OwnedEntities should be visible in Changes.

Libraries (specify the Audit.NET extensions being used including version): For example:

Target .NET framework: For example:

Additional context

public enum FulfillmentOfferedService
{
    Pick_And_Pack,
    Storage_Per_CBM_Pallet_Shelves,
}
public sealed class Shipper {

}
public sealed class ShipperPricing
{
    public long Id { get; init; }
    public FulfillmentOfferedService OfferedService { get; set; }
    public List<SkuRangePricing> PriceRanges { get; private set; } = new ();
    public decimal? ExtraPerSku { get; set; }

    public Shipper Shipper { get; set; } = null!;
    public long ShipperId { get; set; }
}

/// <summary>
/// Will be used as JSON column to avoid extra JOIN.
/// </summary>
public record SkuRangePricing
{
    public string Range { get; set; } = string.Empty;
    /// <summary>
    /// Fee will in AED.
    /// </summary>
    public decimal Fee { get; set; }
}

public sealed class ShipperPricingConfigurationConfiguration : IAuditableEntityConfiguration<ShipperPricing>
{
    public override void ConfigureEntity(EntityTypeBuilder<ShipperPricing> builder)
    {
        builder.Property(x => x.Id)
            .ValueGeneratedNever();
        builder.Property(x => x.OfferedService)
            .HasConversion<string>()
            .HasMaxLength(StringLength.StandardMax);

        builder.OwnsMany(x => x.PriceRanges, skuRange =>
        {
            skuRange.ToJson();
            skuRange.Property(x => x.Fee)
                .HasColumnType("decimal(9, 2)");
        });

        builder.Property(x => x.ExtraPerSku)
            .HasColumnType("decimal(9, 2)");
    }
}

Audit.Core.Configuration.Setup()
            .UseEntityFramework(ef => ef
                .AuditTypeExplicitMapper(m => m
                    .Map<Shipper, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.Shippers);
                        return true;
                    })
                    .Map<ShipperCarrier, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.ShipperCarriers);
                        auditLog.Reference.Add((entry.GetEntry().Entity as ShipperCarrier)!.ShipperId.ToString());
                        return true;
                    })
                    .Map<ShipperPricing, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.ShipperPricings);
                        auditLog.Reference.Add((entry.GetEntry().Entity as ShipperPricing)!.ShipperId.ToString());
                        return true;
                    })
                    .AuditEntityAction<AuditLog>((ev, entry, entity) =>
                    {
                        // To fix audit infinity cycle in case of DbUpdateException
                        // if (ev.Environment.Exception != null)
                        // {
                        //     return Task.FromResult(false);
                        // }

                        // If only created at or updated at is modified and not prop is change ignoring that event
                        if (entry.GetEntry().Entity is IAuditableEntity auditableEntity)
                        {
                            if (entry.Changes != null && entry.Changes.Count == 1)
                            {
                                if (entry.Changes.First().ColumnName == nameof(IAuditableEntity.CreatedAt) ||
                                    entry.Changes.First().ColumnName == nameof(IAuditableEntity.UpdatedAt))
                                {
                                    return false;
                                }
                            }
                        }

                        entity.Action = entry.Action;
                        entity.Exception = ev.GetEntityFrameworkEvent().ErrorMessage;
                        entity.Data = entry.ToJson();
                        entity.Changes = JsonSerializer.Serialize(entry.Changes ?? []);
                        entity.EntityType = entry.EntityType.Name;
                        entity.AuditDate = DateTime.UtcNow;

                        entity.TablePk = entry.PrimaryKey.First().Value.ToString()!;
                        var userId = ev.CustomFields[AuditKeys.UserId];
                        var email = ev.CustomFields[AuditKeys.Email];
                        if (userId is not null)
                        {
                            entity.ByUserId = userId.ToString()!;
                            entity.ByUserName = email.ToString()!;
                            // ev.CustomFields["User"] = userId;
                        }

                        return true;
                    }))
                .IgnoreMatchedProperties(true));

LogicTosave() {
foreach (ShipperPricingDto pricingDto in request.PricingDtos)
        {
            ShipperPricing? shipperPricing;
            if (string.IsNullOrEmpty(pricingDto.Id))
            {
                shipperPricing = new ShipperPricing
                {
                    Id = _idGenerator.LongId,
                    OfferedService = pricingDto.OfferedService,
                    ShipperId = request.ShipperId
                };
                _dbContext.ShipperPricings.Add(shipperPricing);
            }
            else
            {
                shipperPricing = dbShipperPricing.Single(x => x.Id.ToString() == pricingDto.Id);
                _dbContext.ShipperPricings.Update(shipperPricing);
            }

            shipperPricing.PriceRanges.Clear();
            List<SkuRangePricing> ranges = _mapper.Map<List<SkuRangePricing>>(pricingDto.PriceRanges) ?? [];
            shipperPricing.PriceRanges.AddRange(ranges);
            shipperPricing.ExtraPerSku = pricingDto.ExtraPerSku;
        }
}

See the below screenshot. Are owned entries should be treated as separate entries. Also the state is Added during execution, which is why i guess its not added to value changes

Screenshot from 2024-09-25 08-23-54

thepirat000 commented 5 days ago

Could you clarify which property you don't see in EventEntry.Changes for your example? I couldn't understand the screenshot. Please provide what you are getting and what your expectation is

hassanrazakhalid commented 5 days ago

yes. What i think should be like this Changes structure similar to

{
    "Changes": [
               // ommiting other properities
        {
            "PropertyName": "PriceRanges"
            "OldValue": "some-json-string",
            "NewValue": "some-json-string"
        }
    ]
}

since "PriceRanges" is defined as Owned and cannot exist independently, When i modify entity "ShipperPricing" and modify PriceRanges, then i expect to see is value changes along with non-complex properties. Right now

.AuditEntityAction<AuditLog> is missing "PriceRanges" in changes when i save parent entity "ShipperPricing"

thepirat000 commented 5 days ago

I was unable to reproduce. You should see the changes to the owned property by the owner's entity.

Could you provide a minimal project that replicates the issue, or at the very least, share all relevant code and configuration details?

Note you probably want to set the ReloadDatabaseValues configuration to true, since you use DbContext.Update (see this)

For example:

Audit.EntityFramework.Configuration.Setup()
    .ForAnyContext(cfg => cfg
        .ReloadDatabaseValues());
hassanrazakhalid commented 4 days ago

Adding ReloadDatabaseValues() gives exception

An unhandled exception has occurred while executing the request.
System.Collections.Generic.KeyNotFoundException: The given key 'Property: SkuRangePricing.Id (no field, int) Shadow Required PK AfterSave:Throw ValueGenerated.OnAdd' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.EntityFrameworkCore.Query.StructuralTypeProjectionExpression.BindProperty(IProperty property)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.BindProperty(StructuralTypeReferenceExpression typeReference, IProperty property)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.TryBindMember(Expression source, MemberIdentity member, Expression& expression, IPropertyBase& property)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.TryBindMember(Expression source, MemberIdentity member, Expression& expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerSqlTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerSqlTranslatingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression)
   at Microsoft.EntityFrameworkCore.SqlServer.Query.Internal.SqlServerSqlTranslatingExpressionVisitor.VisitBinary(BinaryExpression binaryExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.TranslateInternal(Expression expression, Boolean applyDefaultTypeMapping)
   at Microsoft.EntityFrameworkCore.Query.RelationalSqlTranslatingExpressionVisitor.Translate(Expression expression, Boolean applyDefaultTypeMapping)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateExpression(Expression expression, Boolean applyDefaultTypeMapping)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateLambdaExpression(ShapedQueryExpression shapedQueryExpression, LambdaExpression lambdaExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateWhere(ShapedQueryExpression source, LambdaExpression predicate)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at Microsoft.EntityFrameworkCore.Query.QueryableMethodTranslatingExpressionVisitor.Translate(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryableMethodTranslatingExpressionVisitor.Translate(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.QueryCompilationContext.CreateQueryExecutor[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult](Expression query, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult](IDatabase database, Expression query, IModel model, Boolean async)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass9_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Internal.EntityFinder`1.GetDatabaseValues(InternalEntityEntry entry)
   at Microsoft.EntityFrameworkCore.ChangeTracking.EntityEntry.GetDatabaseValues()
   at Audit.EntityFramework.DbContextHelper.CreateAuditEvent(IAuditDbContext context)
   at Audit.EntityFramework.DbContextHelper.BeginSaveChangesAsync(IAuditDbContext context, CancellationToken cancellationToken)
   at Audit.EntityFramework.AuditSaveChangesInterceptor.SavingChangesAsync(DbContextEventData eventData, InterceptionResult`1 result, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Diagnostics.Internal.SaveChangesInterceptorAggregator.CompositeSaveChangesInterceptor.SavingChangesAsync(DbContextEventData eventData, InterceptionResult`1 result, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken)
   at Application.Features.Shippers.Commands.SaveShipperPricing.SaveShipperPricingCommandHandler.Handle(SaveShipperPricingCommand request, CancellationToken cancellationToken) in /home/hassan/projects/uruk/fulfillment-web/src/Fulfilment.Application/Features/Shippers/Commands/SaveShipperPricing/SaveShipperPricingCommandHandler.cs:line 62
   at Application.Commons.Behaviors.ValidationBehavior`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken) in /home/hassan/projects/uruk/fulfillment-web/src/Fulfilment.Application/Commons/Behaviors/ValidationBehavior.cs:line 23
   at Application.Commons.Behaviors.LoggingBehavior`2.Handle(TRequest request, RequestHandlerDelegate`1 next, CancellationToken cancellationToken) in /home/hassan/projects/uruk/fulfillment-web/src/Fulfilment.Application/Commons/Behaviors/LoggingBehavior.cs:line 25
   at Fulfillment.Sys.Controllers.Shippers.ShipperController.SavePricingDetails(SaveShipperPricingCommand command) in /home/hassan/projects/uruk/fulfillment-web/src/Fulfillment.Sys/Controllers/Shippers/ShipperController.cs:line 261
   at lambda_method487(Closure, Object)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.AwaitableObjectResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Logged|12_1(ControllerActionInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
hassanrazakhalid commented 4 days ago

ShipperPrice Configuration

public sealed class ShipperPricingConfigurationConfiguration : IAuditableEntityConfiguration<ShipperPricing>
{
    public override void ConfigureEntity(EntityTypeBuilder<ShipperPricing> builder)
    {
        builder.Property(x => x.Id)
            .ValueGeneratedNever();
        builder.Property(x => x.OfferedService)
            .HasConversion<string>()
            .HasMaxLength(StringLength.StandardMax);

        // builder.Property(x => x.SkuRanges)
        //     
        //     .HasMaxLength(StringLength.StandardMax);
        builder.OwnsMany(x => x.PriceRanges, skuRange =>
        {
            skuRange.ToJson();
            skuRange.Property(x => x.Fee)
                .HasColumnType("decimal(9, 2)");
        });

        builder.Property(x => x.ExtraPerSku)
            .HasColumnType("decimal(9, 2)");
    }
}

The configuration of Audit.Net i'm using

Audit.EntityFramework.Configuration.Setup()
            .ForContext<AppDbContext>(config => config
                    .IncludeEntityObjects()
                .ForEntity<ShipperPricing>(_ => _
                    .Ignore(x => x.UpdatedAt)
                    .Override(x => x.PriceRanges, value =>
                    {
                        return value.Entity as List<ShipperPricing>;
                    })
                    .Format(x => x.PriceRanges, value =>
                    {
                        return JsonSerializer.Serialize(value);
                    })
                )
                // .ForEntity<ShipperPricing>(_ => _
                //     .Format(sp => sp.PriceRanges, pr => JsonSerializer.Serialize(pr))))
                )
            .UseOptOut()
            ;

        Audit.Core.Configuration.Setup()
            .UseEntityFramework(ef => ef
                .AuditTypeExplicitMapper(m => m
                    .Map<Shipper, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.Shippers);
                        return true;
                    })
                    .Map<ShipperCarrier, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.ShipperCarriers);
                        auditLog.Reference.Add((entry.GetEntry().Entity as ShipperCarrier)!.ShipperId.ToString());
                        return true;
                    })
                    // .Map<SkuRangePricing, AuditLog>((shipper, entry, auditLog) =>
                    // {
                    //     return true;
                    // })
                    .Map<ShipperPricing, AuditLog>((shipper, entry, auditLog) =>
                    {
                        auditLog.TableName = nameof(AppDbContext.ShipperPricings);
                        auditLog.Reference.Add((entry.GetEntry().Entity as ShipperPricing)!.ShipperId.ToString());
                        return true;
                    })
                    .AuditEntityAction<AuditLog>((ev, entry, entity) =>
                    {
                        // To fix audit infinity cycle in case of DbUpdateException
                        if (ev.Environment.Exception != null)
                        {
                            return false;
                        }

                        // If only created at or updated at is modified and not prop is change ignoring that event
                        if (entry.GetEntry().Entity is IAuditableEntity auditableEntity)
                        {
                            if (entry.Changes == null || !entry.Changes.Any()) return false;

                            if (entry.Changes != null && entry.Changes.Count == 1)
                            {
                                if (entry.Changes.First().ColumnName == nameof(IAuditableEntity.CreatedAt) ||
                                    entry.Changes.First().ColumnName == nameof(IAuditableEntity.UpdatedAt))
                                {
                                    return false;
                                }
                            }
                        }

                        entity.Action = entry.Action;
                        entity.Exception = ev.GetEntityFrameworkEvent().ErrorMessage;
                        entity.Data = entry.ToJson();
                        entity.Changes = JsonSerializer.Serialize(entry.Changes ?? []);
                        entity.EntityType = entry.EntityType.Name;
                        entity.AuditDate = DateTime.UtcNow;

                        entity.TablePk = entry.PrimaryKey.First().Value.ToString()!;
                        var userId = ev.CustomFields[AuditKeys.UserId];
                        var email = ev.CustomFields[AuditKeys.Email];
                        if (userId is not null)
                        {
                            entity.ByUserId = userId.ToString()!;
                            entity.ByUserName = email.ToString()!;
                            // ev.CustomFields["User"] = userId;
                        }

                        return true;
                    }))
                .IgnoreMatchedProperties(true));

I have a few questions

  1. Do i need both configurations in case i want to override serialization of complex property, or it can be done in single configuration, also that overrides of "Format" and "Override" of a complex callbacks are never called.
  2. see this chunk of code. It is commented in the above configuration. When trying to figure the issue, if i uncomment this code. It gets called although its a complex property and not an independent Entity. Why is it getting called on a complex property 
// .Map<SkuRangePricing, AuditLog>((shipper, entry, auditLog) =>
                    // {
                    //     return true;
                    // })
hassanrazakhalid commented 4 days ago

I'm sharing my findings, may be it will give you a clue where things are going wrong. If still, a minimal project is required. Let me know.

thepirat000 commented 1 day ago

The Override and Format methods are not applied to navigation properties (or owned/complex properties of the owner entity) because they rely on EF Core change tracker entries. In your scenario, the EF change tracker will create separate entries for ShipperPricing and SkuRangePricing, even though the latter is defined as an owned entity in the same table. Internally, EF treats them as distinct entities.

The Map<SkuRangePricing, AuditLog> action is executed for the same reason.

To address the issue mentioned in your comment // To fix audit infinity cycle in case of DbUpdateException, you can resolve it by using a separate instance of your DbContext.

You can invoke UseDbContext to define a new DbContext instance like this:

Audit.Core.Configuration.Setup()
   .UseEntityFramework(ef => ef
       .UseDbContext(ctx => ctx.GetEntityFrameworkEvent().GetDbContext().GetService<AppDbContext>())
   );

Ultimately, the solution depends on your specific implementation and requirements, which are not entirely clear. If you need further assistance, please provide a minimal sample project that reproduces the issue, along with details about your expected audit output.