search

opengento / magento2-gdpr

Magento 2 GDPR module is a must have extension for the largest e-commerce CMS used in the world. The module helps to be GDPR compliant. Actually it allows the customers to erase, or export their personal data. As a merchant you have powerful tools to customize the extension capabilities and apply the finest privacy rules.
https://opengento.fr/magento2-gdpr
MIT License
140 stars 39 forks source link

[BUG] my system.log fills up with the same error message from the module Impossible to process the erasure: One or more input exceptions have occurred #143

Closed ioweb-gr closed 2 months ago

ioweb-gr commented 7 months ago

Description

I'm getting constantly the folllowing error

[2024-03-22T09:51:15.802108+00:00] main.ERROR: Impossible to process the erasure: One or more input exceptions have occurred. [{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/opengento/module-gdpr/Cron/EraseEntity.php","line":71,"function":"process","class":"Opengento\\Gdpr\\Model\\EraseEntityManagement","type":"->"},{"function":"execute","class":"Opengento\\Gdpr\\Cron\\EraseEntity","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":368,"function":"call_user_func_array"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":879,"function":"_runJob","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":840,"function":"tryRunJob","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Observer/ProcessCronQueueObserver.php","line":280,"function":"processPendingJobs","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Invoker/InvokerDefault.php","line":88,"function":"execute","class":"Magento\\Cron\\Observer\\ProcessCronQueueObserver","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Invoker/InvokerDefault.php","line":74,"function":"_callObserverMethod","class":"Magento\\Framework\\Event\\Invoker\\InvokerDefault","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Event/Manager.php","line":65,"function":"dispatch","class":"Magento\\Framework\\Event\\Invoker\\InvokerDefault","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/generated/code/Magento/Framework/Event/Manager/Proxy.php","line":95,"function":"dispatch","class":"Magento\\Framework\\Event\\Manager","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/App/Cron.php","line":86,"function":"dispatch","class":"Magento\\Framework\\Event\\Manager\\Proxy","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/module-cron/Console/Command/CronCommand.php","line":126,"function":"launch","class":"Magento\\Framework\\App\\Cron","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Command/Command.php","line":298,"function":"execute","class":"Magento\\Cron\\Console\\Command\\CronCommand","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":58,"function":"run","class":"Symfony\\Component\\Console\\Command\\Command","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":138,"function":"___callParent","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Interception/Interceptor.php","line":153,"function":"Magento\\Framework\\Interception\\{closure}","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/generated/code/Magento/Cron/Console/Command/CronCommand/Interceptor.php","line":23,"function":"___callPlugins","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":1040,"function":"run","class":"Magento\\Cron\\Console\\Command\\CronCommand\\Interceptor","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":301,"function":"doRunCommand","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/magento/framework/Console/Cli.php","line":116,"function":"doRun","class":"Symfony\\Component\\Console\\Application","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/vendor/symfony/console/Application.php","line":171,"function":"doRun","class":"Magento\\Framework\\Console\\Cli","type":"->"},{"file":"/var/www/vhosts/domain.gr/httpdocs/bin/magento","line":23,"function":"run","class":"Symfony\\Component\\Console\\Application","type":"->"}] []

Prerequisites

PHP Version:

Magento Version:

Module Version:

Desktop (if applicable):

Smartphone (if applicable):

Issue Details

Steps to reproduce the behavior I constantly see this error spamming without any special action item occurring. It just spams there

Expected behavior
The error is gone

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
It turns out my log will fill up into hundreds of gigabytes in days.

ioweb-gr commented 5 months ago

I've been trying to debug this and it seems the cron job that's running to process erasures, had to go through a list of 981 customers. There are a couple of things I notice

  1. There's no date to indicate when the deletion was requested. These requests should be logged.
  2. It seems in some cases it's trying to anonymize the user, but it cannot process the address properly as it's not validating extra address attributes which are required.
  3. It will repeat the process constantly without notifying the admin about the error and will never be able to complete.
  4. It's highly unlikely that half our customers requested deletion. We cannot process deletion requests without logging the actual submission of the request.
  5. There's an abnormally high count of rows in the table for 1641 unique customers and 1400 unique orders image

The actual error in most of the cases for me is this one

Any ideas how to proceed with this?

I'm going to clear up the table just in case and start fresh, but I really need to see how to log every request for deletion in a manageable way.

ioweb-gr commented 4 months ago

Hi,

any news on what might be causing it and how I can mitigate this issue?

thomas-kl1 commented 4 months ago

Hello, yes it seems it's related to customer validation that is not disabled in this case. $customer->set data('ignore_validation_flag') seems to be missing. Btw the module is going to be reworked in order to remove some abstraction and simplicity some cases.

ioweb-gr commented 4 months ago

I see. Also any ideas why half of our customers appeared in the deletion list? I doubt it half of them requested deletion.

Is it possible the module is being exploited somehow?

thomas-kl1 commented 4 months ago

There's a config to remove automatically users after being idle for a certain amount of time. You can check their last updated date and see if it match.

ioweb-gr commented 4 months ago

Oh I see. Is the config enabled by default?

thomas-kl1 commented 4 months ago

If you have enabled erasure, yes.

See https://github.com/opengento/magento2-gdpr/blob/5618a8c86efb22ac99b4c8fd66a7b54b36f0e075/etc/config.xml#L126

Value is in days GDPR requires data to be erased when user is idle.

ioweb-gr commented 4 months ago

I see it's impossible to disable the feature or set a virtually unlimited entity_max_age. We basically want our customers to be able to request deletion, but not automatically delete them just because they're old customers.

thomas-kl1 commented 2 months ago

Is fixed in 5.0.0-beta1