tehraven commented 6 months ago

Problem: Whats wrong?

The CharacterAssets and CorporationAssets jobs were made to delete all assets that were not "touched" by the execution of a assets job. The change that introduced this logic is here: https://github.com/eveseat/eveapi/pull/391/files#diff-8720b696b5950adb2e6a8cfe1a3b249b0d5bc0ef43abfed94eabff7d8927bd47L133-R124

The ->save() call that occurs inside the assets->each() call doesn't actually affect the updated_at column because the Model class from Laravel gaurds that operation with isDirty checks -- The row in question has to actually be different then the DB value, else it won't perform the operation at all.

Due to this, every iteration of the Character/Corporation Assets job deletes all assets that were not actually changed in this iteration. Example: You have 5 items. First run, it adds 5 rows. Before the second run, you add a 6th item to your inventory in-game. After that run, one new record is added, and the preexisting 5 rows will not have their updated_at columns updated, so they will be deleted. The third run will then find 5 new assets, and purge the 6th because it wasn't changed. Your runs will now flip-flop between 1 and 5 assets.

Expected: What did you expect to happen?

I expected the jobs to run, find all assets, and purge assets no longer returned by the ESI

Logs / Screenshots / Proof: The more information you give the better!

Version Info: PHP Version, SeAT Version, Operating System etc.

SEAT v5 https://github.com/recursivetree/eveapi/blob/5.0.x/src/Jobs/Assets/Character/Assets.php#L113

kibawulf commented 6 months ago

After updating an existing test environment to check out this change, it looks like this may break asset jobs currently due to data constraints. Asset jobs no longer seem to function and get stuck. Reverting to the previous version takes it back to a functional state. I'll certainly see if leaving it alone for a bit causes it to recover - but I have some doubts. Each time an asset job attempts to run the following stack trace is thrown.

`count(): Argument #1 ($value) must be of type Countable|array, stdClass given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, stdClass given at /var/www/seat/vendor/softcreatr/jsonpath/src/Filters/SliceFilter.php:17) [stacktrace]

0 /var/www/seat/vendor/softcreatr/jsonpath/src/JSONPath.php(51): Flow\JSONPath\Filters\SliceFilter->filter(Object(stdClass))

1 /var/www/seat/vendor/eveseat/eveapi/src/Mapping/DataMapping.php(66): Flow\JSONPath\JSONPath->find('2024-05-22 15:4...')

2 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Assets/Character/Assets.php(114): Seat\Eveapi\Mapping\DataMapping::make(Object(Seat\Eveapi\Models\Assets\CharacterAsset), Object(stdClass), Array)

3 /var/www/seat/vendor/laravel/framework/src/Illuminate/Collections/Traits/EnumeratesValues.php(240): Seat\Eveapi\Jobs\Assets\Character\Assets->Seat\Eveapi\Jobs\Assets\Character\{closure}(Object(stdClass), 0)

4 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Assets/Character/Assets.php(103): Illuminate\Support\Collection->each(Object(Closure))

5 /var/www/seat/vendor/laravel/framework/src/Illuminate/Collections/Traits/EnumeratesValues.php(240): Seat\Eveapi\Jobs\Assets\Character\Assets->Seat\Eveapi\Jobs\Assets\Character\{closure}(Object(Illuminate\Support\Collection), 0)

6 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Assets/Character/Assets.php(101): Illuminate\Support\Collection->each(Object(Closure))

7 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Seat\Eveapi\Jobs\Assets\Character\Assets->handle()

8 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()

9 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))

10 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))

11 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)

12 /var/www/seat/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(128): Illuminate\Container\Container->call(Array)

13 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

14 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

15 /var/www/seat/vendor/laravel/framework/src/Illuminate/Bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then(Object(Closure))

16 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(123): Illuminate\Bus\Dispatcher->dispatchNow(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), false)

17 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(144): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

18 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Middleware/WithoutOverlapping.php(78): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

19 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Illuminate\Queue\Middleware\WithoutOverlapping->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

20 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Middleware/CheckTokenVersion.php(54): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

21 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Seat\Eveapi\Jobs\Middleware\CheckTokenVersion->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

22 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Middleware/CheckTokenScope.php(82): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

23 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Seat\Eveapi\Jobs\Middleware\CheckTokenScope->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

24 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Middleware/CheckEsiRouteStatus.php(80): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

25 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Seat\Eveapi\Jobs\Middleware\CheckEsiRouteStatus->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

26 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Middleware/CheckServerStatus.php(55): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

27 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Seat\Eveapi\Jobs\Middleware\CheckServerStatus->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

28 /var/www/seat/vendor/eveseat/eveapi/src/Jobs/Middleware/CheckEsiRateLimit.php(57): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

29 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(183): Seat\Eveapi\Jobs\Middleware\CheckEsiRateLimit->handle(Object(Seat\Eveapi\Jobs\Assets\Character\Assets), Object(Closure))

30 /var/www/seat/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(119): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

31 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(122): Illuminate\Pipeline\Pipeline->then(Object(Closure))

32 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\Queue\Jobs\RedisJob), Object(Seat\Eveapi\Jobs\Assets\Character\Assets))

33 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Jobs/Job.php(102): Illuminate\Queue\CallQueuedHandler->call(Object(Illuminate\Queue\Jobs\RedisJob), Array)

34 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(439): Illuminate\Queue\Jobs\Job->fire()

35 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(389): Illuminate\Queue\Worker->process('redis', Object(Illuminate\Queue\Jobs\RedisJob), Object(Illuminate\Queue\WorkerOptions))

36 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(176): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\RedisJob), 'redis', Object(Illuminate\Queue\WorkerOptions))

37 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(137): Illuminate\Queue\Worker->daemon('redis', 'characters', Object(Illuminate\Queue\WorkerOptions))

38 /var/www/seat/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(120): Illuminate\Queue\Console\WorkCommand->runWorker('redis', 'characters')

39 /var/www/seat/vendor/laravel/horizon/src/Console/WorkCommand.php(51): Illuminate\Queue\Console\WorkCommand->handle()

40 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(36): Laravel\Horizon\Console\WorkCommand->handle()

41 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/Util.php(41): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()

42 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))

43 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Foundation\Application), Array, Object(Closure))

44 /var/www/seat/vendor/laravel/framework/src/Illuminate/Container/Container.php(662): Illuminate\Container\BoundMethod::call(Object(Illuminate\Foundation\Application), Array, Array, NULL)

45 /var/www/seat/vendor/laravel/framework/src/Illuminate/Console/Command.php(211): Illuminate\Container\Container->call(Array)

46 /var/www/seat/vendor/symfony/console/Command/Command.php(326): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))

47 /var/www/seat/vendor/laravel/framework/src/Illuminate/Console/Command.php(180): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))

48 /var/www/seat/vendor/symfony/console/Application.php(1096): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

49 /var/www/seat/vendor/symfony/console/Application.php(324): Symfony\Component\Console\Application->doRunCommand(Object(Laravel\Horizon\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

50 /var/www/seat/vendor/symfony/console/Application.php(175): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

51 /var/www/seat/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(201): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

52 /var/www/seat/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))

53 {main}

"}`