Not saving operations

[e9li]

Hello, I just updated from the 'old' audit which was working perfectly with Symfony 5.2. The configuration is default and really standard with one DB. With the old everything worked fine but now nothing gets logged. Someone has an idea what could be wrong?

Symfony 5.2.1 damienharper/auditor: damienharper/auditor-bundle: doctrine/annotations: *

Current configuration for "DoctrineBundle"
==========================================

doctrine:
    dbal:
        default_connection: default
        connections:
            default:
                url: '%env(resolve:DATABASE_URL)%'
                driver: pdo_mysql
                server_version: '5.7'
                charset: utf8mb4
                default_table_options:
                    collate: utf8mb4_unicode_ci
                host: localhost
                port: null
                user: root
                password: null
                logging: true
                profiling: true
                profiling_collect_backtrace: false
                profiling_collect_schema_errors: true
                options: {  }
                mapping_types: {  }
                slaves: {  }
                replicas: {  }
                shards: {  }
        types: {  }
    orm:
        auto_generate_proxy_classes: true
        default_entity_manager: default
        entity_managers:
            default:
                naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
                auto_mapping: true
                mappings:
                    App:
                        is_bundle: false
                        type: annotation
                        dir: xyz/src/Entity
                        prefix: App\Entity
                        alias: App
                        mapping: true
                query_cache_driver:
                    type: null
                metadata_cache_driver:
                    type: null
                result_cache_driver:
                    type: null
                class_metadata_factory_name: Doctrine\ORM\Mapping\ClassMetadataFactory
                default_repository_class: Doctrine\ORM\EntityRepository
                quote_strategy: doctrine.orm.quote_strategy.default
                entity_listener_resolver: null
                repository_factory: doctrine.orm.container_repository_factory
                hydrators: {  }
                filters: {  }
        proxy_dir: '%kernel.cache_dir%/doctrine/orm/Proxies'
        proxy_namespace: Proxies
        resolve_target_entities: {  }

dh_auditor:
    enabled: true
    timezone: 'Europe/Zurich'
    providers:
        doctrine:
            table_prefix: ~
            table_suffix: '_audit'
            entities:
                App\Entity\Action: ~
                App\Entity\ActionUser: ~
                App\Entity\Address: ~
                App\Entity\AddressContactNote: ~
                App\Entity\AddressLog: ~
                App\Entity\Alert: ~

Originally posted by @e9li in https://github.com/DamienHarper/auditor-bundle/issues/133#issuecomment-751665031

[e9li]

At the moment I do not use annotations for audits. But I tried annotations in an entity but that's also not logging anything. To update guide in the docs are clear. bin/console audit:schema:update --force worked as well and finds/updates all the tables from the config file.

[N4IR0]

I have almost the same problem, only with the difference that it works for me in the development environment, but as soon as I change the environment to prod it does not work anymore. The audit viewer is working and table schema is up to date.

Installed packages:

Symfony: 5.2.1
damienharper/auditor-bundle: 4.0.3

Doctrine configuration:

Current configuration for "DoctrineBundle"
==========================================

doctrine:
    dbal:
        default_connection: default
        connections:
            default:
                driver: pdo_pgsql
                host: postgres
                port: null
                dbname: symfony
                user: symfony
                password: dev
                charset: UTF8
                logging: false
                profiling: false
                profiling_collect_backtrace: false
                profiling_collect_schema_errors: true
                options: {  }
                mapping_types: {  }
                default_table_options: {  }
                slaves: {  }
                replicas: {  }
                shards: {  }
        types: {  }
    orm:
        auto_generate_proxy_classes: false
        default_entity_manager: default
        entity_managers:
            default:
                naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
                auto_mapping: true
                mappings:
                    App:
                        is_bundle: false
                        type: annotation
                        dir: /code/src/Entity
                        prefix: App\Entity
                        alias: App
                        mapping: true
                metadata_cache_driver:
                    type: service
                    id: doctrine.system_cache_provider
                query_cache_driver:
                    type: service
                    id: doctrine.system_cache_provider
                result_cache_driver:
                    type: service
                    id: doctrine.result_cache_provider
                class_metadata_factory_name: Doctrine\ORM\Mapping\ClassMetadataFactory
                default_repository_class: Doctrine\ORM\EntityRepository
                quote_strategy: doctrine.orm.quote_strategy.default
                entity_listener_resolver: null
                repository_factory: doctrine.orm.container_repository_factory
                hydrators: {  }
                filters: {  }
        proxy_dir: '%kernel.cache_dir%/doctrine/orm/Proxies'
        proxy_namespace: Proxies
        resolve_target_entities: {  }

dh_auditor.yaml:

dh_auditor:
    enabled: true
    timezone: 'Europe/Berlin'
    providers:
        doctrine:
            table_prefix: ~
            table_suffix: '_audit'
            ignored_columns:
                - id
                - createdAt
                - updatedAt
            entities:
                App\Entity\Instance: ~
                App\Entity\Server: ~
                App\Entity\Task:
                    ignored_columns:
                        - callback
                        - payload
                        - returnPayload
                        - taskQueue
                        - startedAt
                        - finishedAt

[DamienHarper]

@N4IR0 that's strange, did you clear your cache in prod environment? @e9li does it work in dev env? @e9li @N4IR0 do you have any other bundle extending/depending on doctrine? (post the result of composer info might be helpful)

[e9li]

In my case it does not log operations at all, not in dev nor in prod.

doctrine/annotations                 1.11.1
doctrine/cache                       1.10.2
doctrine/collections                 1.6.7
doctrine/common                      3.1.0
doctrine/dbal                        2.12.1
doctrine/doctrine-bundle             2.2.2
doctrine/doctrine-migrations-bundle  3.0.2 
doctrine/event-manager               1.1.1
doctrine/inflector                   2.0.3
doctrine/instantiator                1.4.0
doctrine/lexer                       1.2.1
doctrine/migrations                  3.0.1
doctrine/orm                         2.8.1
doctrine/persistence                 2.1.0
doctrine/sql-formatter               1.1.1
gedmo/doctrine-extensions            v3.0.0 

composer.json is like this

"require": {
        "php": "^7.4",
        "ext-ctype": "*",
        "ext-exif": "*",
        "ext-fileinfo": "*",
        "ext-iconv": "*",
        "ext-intl": "*",
        "ext-json": "*",
        "damienharper/auditor": "*",
        "damienharper/auditor-bundle": "*",
        "doctrine/annotations": "*",
        "gedmo/doctrine-extensions": "*",
        "hakimio/pdfmerger": "dev-master",
        "imagine/imagine": "*",
        "knplabs/knp-paginator-bundle": "*",
        "knplabs/knp-snappy-bundle": "*",
        "oneup/uploader-bundle": "*",
        "phpoffice/phpspreadsheet": "*",
        "sensio/framework-extra-bundle": "*",
        "symfony/apache-pack": "*",
        "symfony/asset": "*",
        "symfony/console": "*",
        "symfony/dotenv": "*",
        "symfony/expression-language": "*",
        "symfony/filesystem": "*",
        "symfony/finder": "*",
        "symfony/flex": "*",
        "symfony/form": "*",
        "symfony/framework-bundle": "*",
        "symfony/http-client": "*",
        "symfony/mailer": "*",
        "symfony/monolog-bundle": "*",
        "symfony/orm-pack": "*",
        "symfony/process": "*",
        "symfony/security-bundle": "*",
        "symfony/serializer-pack": "*",
        "symfony/translation": "*",
        "symfony/twig-bundle": "5.*",
        "symfony/uid": "5.*",
        "symfony/validator": "*",
        "symfony/web-link": "*",
        "symfony/yaml": "*",
        "twig/cssinliner-extra": "*",
        "twig/extra-bundle": "*",
        "twig/inky-extra": "*",
        "twig/twig": "*",
        "vich/uploader-bundle": "*"
    },

[DamienHarper]

@e9li thanks, I need some more time to investigate

[N4IR0]

Yes, I've cleared the cache multiple times to ensure that this is not the issue. Additionally I changed the Symfony environment of my local development environment to production to ensure that this is not an issue with the production setup.

Here is the output of composer info with all bundles extending doctrine/entities:

damienharper/auditor                 1.0.1     The missing audit log library.
damienharper/auditor-bundle          4.0.3     Integrate auditor library in your Symfony projects.
doctrine/annotations                 1.11.1    Docblock Annotations Parser
doctrine/cache                       1.10.2    PHP Doctrine Cache library is a popular cache implementation that supports many different drivers such as redis, memcache, apc, mongodb and others.
doctrine/collections                 1.6.7     PHP Doctrine Collections library that adds additional functionality on top of PHP arrays.
doctrine/common                      3.1.0     PHP Doctrine Common project is a library that provides additional functionality that other Doctrine projects depend on such as better reflection support, proxie...
doctrine/dbal                        2.12.1    Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection and management.
doctrine/doctrine-bundle             2.2.2     Symfony DoctrineBundle
doctrine/doctrine-migrations-bundle  3.0.2     Symfony DoctrineMigrationsBundle
doctrine/event-manager               1.1.1     The Doctrine Event Manager is a simple PHP event system that was built to be used with the various Doctrine projects.
doctrine/inflector                   2.0.3     PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and singular/plural forms of words.
doctrine/instantiator                1.4.0     A small, lightweight utility to instantiate objects in PHP without invoking their constructors
doctrine/lexer                       1.2.1     PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
doctrine/migrations                  3.0.2     PHP Doctrine Migrations project offer additional functionality on top of the database abstraction layer (DBAL) for versioning your database schema and easily de...
doctrine/orm                         2.8.1     Object-Relational-Mapper for PHP
doctrine/persistence                 2.1.0     The Doctrine Persistence project is a set of shared interfaces and functionality that the different Doctrine object mappers share.
doctrine/sql-formatter               1.1.1     a PHP SQL highlighting library
friendsofsymfony/rest-bundle         3.0.4     This Bundle provides various tools to rapidly develop RESTful API's with Symfony
jms/metadata                         2.4.0     Class/method/property metadata management in PHP
jms/serializer                       3.10.0    Library for (de-)serializing data of any complexity; supports XML, JSON, and YAML.
jms/serializer-bundle                3.7.0     Allows you to easily serialize, and deserialize data of any complexity
justinrainbow/json-schema            5.2.10    A library to validate a json schema.
knplabs/knp-components               v3.0.0    Knplabs component library
knplabs/knp-paginator-bundle         v5.4.2    Paginator bundle for Symfony to automate pagination and simplify sorting and other features
nelmio/api-doc-bundle                v4.1.1    Generates documentation for your REST API from annotations
predis/predis                        v1.1.6    Flexible and feature-complete Redis client for PHP and HHVM
snc/redis-bundle                     3.2.4     A Redis bundle for Symfony
symfony/doctrine-bridge              v5.2.1    Symfony Doctrine Bridge
symfony/doctrine-messenger           v5.2.1    Symfony Doctrine Messenger Bridge

Additionally I have an own event subscriber which subscribes preUpdate and postUpdate event in doctrine. This subscriber is still working in production.

[N4IR0]

I have now tried to log all events which are fired. Unfortunately the prod environment does not log events unless APP_DEBUG is set to true, but as soon as I set this to true, the audit is working again (even in production).

Edit: If I register the DoctrineSubscriber manually in the services.yaml of my application it works in production, so I think it's an issue with the event subscriber registration in production:

    DH\Auditor\Provider\Doctrine\Auditing\Transaction\TransactionManager:
        arguments: ['@DH\Auditor\Provider\Doctrine\DoctrineProvider']

    DH\Auditor\Provider\Doctrine\Auditing\Event\DoctrineSubscriber:
        arguments: ['@DH\Auditor\Provider\Doctrine\Auditing\Transaction\TransactionManager']
        tags:
            - { name: doctrine.event_subscriber }

[e9li]

If I register the DoctrineSubscriber manually as @N4IR0 describes, everything seams to work in dev and prod.

[DamienHarper]

@N4IR0 @e9li thanks for the feedback, I'll have a look at this issue asap

[benoit-hash]

Hello Everyone, I really find this bundle wonderful. I have the same problem with no entity modification getting logged since the new version. But with the workaround from @N4IR0, it works.

[ikappas]

@DamienHarper I have run into this issue as well. Using manual service registration as @N4IR0 suggested gets logging working.

[DamienHarper]

@N4IR0 @e9li @benoit-hash @ikappas This should be fixed in master, could you please confirm me that it works for you in production environment, without manually registering DoctrineSubscriber. If it does, I'll release 4.0.4 shortly.

[N4IR0]

@DamienHarper Unfortunately with the master branch it's also not working for me. I've also verified that your changes from PR #233 has been deployed.

[DamienHarper]

@N4IR0 that's weird, I can't reproduce that issue, it works for me in every environment, whatever value is set to APP_DEBUG, still trying to figure out what's going on. Manually registering DoctrineSubscriber should not be required as long as it should be dynamically done when the bundle is loaded 🤔

[DamienHarper]

@e9li @N4IR0 @ikappas @benoit-hash I still can't reproduce the issue so far :/ Can any one of you could share me a project which can reproduce the issue?

Could you also please post your compiled container as a gist for example? (it should be located in a path similar to var/cache/XXX/ContainerYYY/App_KernelXXXContainer.php where XXX should match the environment such as dev or prod and YYY is some random chars)

[Upgreydd]

@DamienHarper - I've checked EasyAdmin3 problem with @N4IR0 solution - works fine, so it's related to this issue.

[DamienHarper]

@N4IR0 @e9li @Upgreydd well I feel like I'm going around in circles and still can't reproduce this issue. It does not seem to appear in the demo of this bundle (https://github.com/DamienHarper/auditor-bundle-demo), It does not seem to appear when using the Symfony demo project. I also use this bundle in several apps in production without this issue so it might be the consequence of using other specific bundles that could affect/register doctrine event listener/subscriber.

It would be really helpful if one of you could share a project in which the issue happens. Thanks by advance.

[Alarich]

Apologies for adding to the existing muck, but I think the issue I'm experiencing is similar and what I found might help somewhat.

PHP 7.3, Symfony 4.4.18.

It seems to me that the Doctrine EntityManager that is described by the Injector Definition in the compilerpass is not the same EntityManager that is loaded in the system later. At some point after the DoctrineProvider obtains its' EntityManager, a new one is generated that does not contain the onFlush event in the EventManager. Thus - nothing is updated.

What the fix, or bypass does, is probably registering everything later, so the EntityManager that the service has is the correct one that is used in the system later.

Without the "fix":

With the "fix" (Notice the 2 DoctrineSubscriber events - I can't explain this):

Here are the kernels you asked for (Some domain things were removed): https://gist.github.com/Alarich/3ced6d5ea97b17ddc10b567643b46f09

Some notable differences:

1. DefaultConnection Event subscribers:
2. DoctrineSubscriber does not exist in the kernel without the "fix", so I'm unsure where the first EntityManager is from.

Alas I'm not well versed in DI and injecting from packages, but I'll provide more information if asked.

[e9li]

@DamienHarper I use this bundle only in one large project and not a public one. I am sorry, unfortunately I can't share it with you. With the fix of @N4IR0 logging works for me. If I can provide you with some further informations let me know.

[DamienHarper]

@N4IR0 thanks for all these info, very valuable. Unfortunately I still can't reproduce it myself :(

[N4IR0]

@DamienHarper Unfortunately I also use this bundle in an internal project, but I try to build an application which reproduces this issue, as soon as possible. Maybe I forgot something while upgrading Symfony from 3.4 -> 4.4 -> 5.2.

[N4IR0]

@DamienHarper While I tried to reproduce this issue in an own application I've found out that version 5.2.2 of symfony/doctrine-bridge fixes this issue. @e9li can you also verify that in your application?

[DamienHarper]

@N4IR0 that sounds interesting. Does this mean you can reproduce the issue with your app if downgrading doctrine-bridge bundle?

[e9li]

@DamienHarper I can confirm that in version 5.2.2 logging works fine without the workaround from @N4IR0. At least in my project. Thank you very much for your great support!

[DamienHarper]

Thanks @e9li @N4IR0! Now we have to find if the problem was introduced in 5.2.1 or if it was already here earlier. What bugs me is that I'm pretty sure it worked in my own projects on 5.2.1, I have to double check.

[N4IR0]

@DamienHarper I have now pushed the app which reproduces the bug. Please note that the creation is getting logged, only modifications are not logged: https://github.com/N4IR0/auditor-bundle-test/

It uses sqlite by default, so just run the migration, afterwards the app should run. I hope this will help you :see_no_evil:

[DamienHarper]

@N4IR0 thanks a lot, I'll have a look at it asap!

[DamienHarper]

@N4IR0 it looks like 5.2.0 and 5.2.1 are the only affected versions of doctrine-bridge Anyway, thanks a lot for your help!

[DamienHarper]

@N4IR0 @e9li @Alarich @Upgreydd I finally excluded both 5.2.0 and 5.2.1 from the composer.json so everything should be fined now in master branch.

Kudos to @N4IR0 for his help!

[Alarich]

@DamienHarper My symfony/doctrine-bridge was 4.4.18, however, 4.4.19 was released just a few days ago and updating to that fixes it for me. https://github.com/symfony/doctrine-bridge/releases/tag/v4.4.19

[Viktor-Bredihin]

@DamienHarper I don't think the issue is fixed, prePersist/postPersist events works now, but preUpdate/postUpdate doesn't. Should I post a new issue?

[Viktor-Bredihin]

if I comment this row - listeners work

[DamienHarper]

@Viktor-Bredihin that's very weird because the line you comment in your screenshot is in a file only used in tests

[mathieu-gilloots]

@DamienHarper I've the same issue. @N4IR0 fixes the issue.

Q	A
auditor-bundle version	4.1.0
PHP version	7.4.15
Symfony version	3.4
Database	MySQL

I think this issue should be re-open.

[Viktor-Bredihin]

@mathieu-gilloots check this out https://github.com/DamienHarper/auditor-bundle/issues/248#issuecomment-786662276 do u have the same problem?

[mathieu-gilloots]

@Viktor-Bredihin No, same problem in prod environement. It's work with @N4IR0 listener.

@DamienHarper any clues ? Thanks

[DamienHarper]

@DamienHarper I've the same issue. @N4IR0 fixes the issue.

@mathieu-gilloots I'm not sure what issue you are talking about (the one from @Viktor-Bredihin or the one from @e9li)? Also, could you please test again using auditor's master branch?