Silverstripe dataObject not found inside AbstractQueuedJob extending class

[elmar001]

Module version(s) affected

5.0.3

Description

Any newly created Silverstripe dataobject can not be initialized or used inside custom class which is extending AbstractQueuedJob and used as a cronjob processor. However, dataobjects which are created before installing symbiote/silverstripe-queuedjobs and have exact same namespaces are fine and work without any issue. There seem to be cached class datamap or something that will not allow new models - dataobjects to be used. This is the error: [2024-01-28 05:52:04][ERROR] Class "revi\models\Ego" not found {"exception":"[object] (Error(code: 0): Class \"revi\\models\\Ego\" not found at /usr/share/nginx/html/s1/app/src/queueHandlers/SendNotificationJob.php:60)"} []

Now, to narrow down the issue, I have already checked:

• namespace correctness;
• dumping classes via $classes = get_declared_classes();
• using direct class name like: "revi\models\Ego"::get()->byID(1);
• using back slash in front of the class reference like: "\revi\models\Ego"::get()->byID(1); or \Ego::get()->byID(1);
• using php require to directly import the class (It worked but Silverstripe related parts like ID column etc would not work so not practical;
• Checked and confirmed that old dataobject class which created before adding symbiote/silverstripe-queuedjobs package are working perfectly in the same context;
• Checked to made sure it is not related to file ownership or permissions: Newly created dataobjects file would not work even after giving exact same ownership and permission. Plus deleting old working dataObject class and adding again. That dataObject still works (potentially indicating symbiote/silverstripe-queuedjobs has cached class datamap or something that only old classes can be used)
• Checked and confirmed that exact same dataObjects work perfectly inside any other controller classes without any issue
• Checked and made sure that it is not related to namespace, file, folder names plural naming or case sensitivity
• Checked this way: $targetCategory = new Ego(); echo $targetCategory->byID(1); - still not working
• Checked this way: $injector = Injector::inst(); $targetCategory = $injector->get(Ego::class); - still not working
• Checked this way: $targetCategory = Ego::singleton()::get()->byID(1); - still not working

In the added example code below: ApiLog, Task, CustomLogger are old dataobjects and each oof them works perfectly in the same context with the exact same namespaces. Test class Ego would not work at all.

Plus, I tried to uninstall and reinstall symbiote/silverstripe-queuedjobs as a last option and it would not work due to dependencies:

composer remove symbiote/silverstripe-queuedjobs

./composer.json has been updated
Running composer update symbiote/silverstripe-queuedjobs --with-all-dependencies
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - silverstripe/queuedjobs dev-ss24 requires silverstripe/framework ~2.4 -> found silverstripe/framework[2.4.9, ..., 2.5.0] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[dev-ss3.0, 1.0.x-dev] require silverstripe/framework ~3.0 -> found silverstripe/framework[3.0.2.1, ..., 3.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[dev-fix-health-check-time, dev-feature-php-7-4, 2.0.1-rc1, ..., 2.10.x-dev] require silverstripe/framework ~3.1 -> found silverstripe/framework[3.1.0-beta1, ..., 3.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/multivaluefield dev-master requires silverstripe/framework ^4 -> found silverstripe/framework[4.0.0-alpha1, ..., 4.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[dev-master, 3.1.2, 4.0.5, ..., 4.5.x-dev] require silverstripe/framework ^4 -> found silverstripe/framework[4.0.0-alpha1, ..., 4.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[3.0.0, ..., 3.x-dev] require silverstripe/framework ^3.1 -> found silverstripe/framework[3.1.0-beta1, ..., 3.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[4.0.0, ..., 4.0.4] require silverstripe/framework ^4@dev -> found silverstripe/framework[4.0.0-alpha1, ..., 4.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs[dev-nglasl-patch-1, 4.6.0-rc1, ..., 4.8.x-dev] require php ^7.1 -> your php version (8.1.2) does not satisfy that requirement.
    - silverstripe/queuedjobs[4.9.0, ..., 4.x-dev] require silverstripe/framework ^4.10 -> found silverstripe/framework[4.10.0-beta1, ..., 4.x-dev] but these were not loaded, likely because it conflicts with another require.
    - silverstripe/queuedjobs dev-ss30 requires silverstripe/multivaluefield dev-master -> satisfiable by silverstripe/multivaluefield[dev-master].
    - Root composer.json requires silverstripe/queuedjobs * -> satisfiable by silverstripe/queuedjobs[dev-fix-health-check-time, dev-master, dev-ss3.0, dev-ss24, dev-ss30, dev-nglasl-patch-1, dev-feature-php-7-4, 1.0.x-dev, 2.0.1-rc1, ..., 2.10.x-dev, 3.0.0, ..., 3.x-dev, 4.0.0, ..., 4.x-dev].

Removal failed, reverting ./composer.json to its original content.
Removal failed, symbiote/silverstripe-queuedjobs is still present, it may be required by another package. See `composer why symbiote/silverstripe-queuedjobs`.

composer why symbiote/silverstripe-queuedjobs

silverstripe/installer           dev-main requires symbiote/silverstripe-queuedjobs (^5.0) 
symbiote/silverstripe-queuedjobs 5.0.3    replaces silverstripe/queuedjobs (self.version)  

composer remove symbiote/silverstripe-queuedjobs --with-all-dependencies

Still not working, same output as above

How to reproduce

/* 
This is the structure inside app/src
controllers
models
admins
middlewares
errorHandlers
pages
util
queueHandlers
 */

/* 
Note: Every time I change anything in dataobjects, I run '/dev/build?flush=1' and plus 'composer dump-autoload' just to be safe
 */

// Custom AbstractQueuedJob extender class inside queueHandlers:
<?php

namespace revi\queueHandlers;

use revi\models\Ego;
use revi\models\ApiLog;
use revi\models\Task;
use Symbiote\QueuedJobs\Services\AbstractQueuedJob;
use revi\util\CustomLogger;

class SendNotificationJob extends AbstractQueuedJob

{

    protected $messages = [];
    protected $jobData = [];
    protected $totalSteps = 1;
    protected $currentStep = 0;
    protected $isComplete = false;

    public function __construct($taskID = null, $notificationType = null, $userType = null, $targetSubType = null, $targetClass = null)
    {
        $this->setJobParameter('taskID', $taskID);
        $this->setJobParameter('notificationType', $notificationType);
        $this->setJobParameter('userType', $userType);
        $this->setJobParameter('targetSubType', $targetSubType);
        $this->setJobParameter('targetClass', $targetClass);

    }

    public function getTitle()
    {
        return json_encode($this->getJobData());

    }

    public function process()
    {
        try {
            $this->totalSteps = 1;
            $this->currentStep = 1;
            $this->messages = [];
            $this->isComplete = true;      
            $data = json_decode($this->getTitle(), true);         

            // Get the task
            $task = Task::get()->byID($data['jobData']['taskID']);
             if(!$task) {
                $this->isComplete = true;
                $this->messages[] = 'Error inside processing SendNotificationJob: Task does not exist';
                return;          
            }  

            $targetCategory = Ego::get()->byID(1);  
/* Problem is here: This is the error: [2024-01-28 05:52:04][ERROR] Class "revi\models\Ego" not found {"exception":"[object] (Error(code: 0): Class \"revi\\models\\Ego\" not found at /usr/share/nginx/html/s1/app/src/queueHandlers/SendNotificationJob.php:60)"} []
 */

            $this->totalSteps = 1;
            $this->currentStep = 1;
            $this->isComplete = true;

        } catch (\Exception $e) {
            $this->totalSteps = 1;
            $this->currentStep = 1;
            $this->isComplete = true;
            $this->messages[] = 'Exception inside processing SendNotificationJob: ' . $e->getMessage();
            // $customLogger = new CustomLogger();
            // $customLogger->error('Exception inside processing SendNotificationJob: ' . $e->getMessage());

        }
    }

    public function getJobData()
    {
        $data = new \stdClass();    
        $data->jobData = $this->jobData;
        $data->totalSteps = $this->totalSteps;
        $data->currentStep = $this->currentStep;
        $data->isComplete = $this->isComplete;
        $data->messages = $this->messages;

        return $data;
    }

    public function setJobParameter($key, $value)
    {

        $this->jobData[$key] = $value;

}

}

// Test class Ego inside models [This is the one which newly created and would not work at all]:
<?php

namespace Revi\Models;

use SilverStripe\ORM\DataObject;

class Ego extends DataObject
{
    private static $table_name = 'Egos';

    private static $db = [
        'Name' => 'Varchar(45)',
        'Type' => "Enum('Tester, User, Stranger, Tester')",
    ];

// Another class ApiLogs [This is old class and works perfectly in the same context with the exact same namespace]:

<?php

namespace Revi\Models;

use SilverStripe\ORM\DataObject;
use SilverStripe\Security\Member;

class ApiLog extends DataObject
{
    private static $table_name = 'ApiLogs';

    private static $db = [
        'URL' => 'Varchar(255)',
        'UserID' => 'Int',
    ];

    private static $has_one = [
        'User' => Member::class
    ];

}

}

Possible Solution

Assuming there should be a built-in way to update class map, if it is indeed saved in cache or trigger it some way

Additional Context

This the output of composer info: composer info

asyncphp/doorman 4.0.0 Child process management composer/ca-bundle 1.4.0 Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle. composer/installers v2.2.0 A multi-framework Composer library installer doctrine/instantiator 2.0.0 A small, lightweight utility to instantiate objects in PHP without invoking their constructors doctrine/lexer 3.0.0 PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers. egulias/email-validator 4.0.2 A library for validating emails against several RFCs embed/embed v4.4.10 PHP library to retrieve page info using oembed, opengraph, etc guzzlehttp/guzzle 7.8.1 Guzzle is a PHP HTTP client library guzzlehttp/promises 2.0.2 Guzzle promises library guzzlehttp/psr7 2.6.2 PSR-7 message implementation that also provides common utility methods intervention/image 2.7.2 Image handling and manipulation library with support for Laravel integration league/csv 9.14.0 CSV data manipulation made easy in PHP league/flysystem 3.23.0 File storage abstraction for PHP league/flysystem-local 3.23.0 Local filesystem adapter for Flysystem. league/mime-type-detection 1.14.0 Mime-type detection for Flysystem m1/env 2.2.0 Env is a lightweight library bringing .env file parser compatibility to PHP. In short - it enables you to read .env files with PHP. marcj/topsort 2.0.0 High-Performance TopSort/Dependency resolving algorithm masterminds/html5 2.8.1 An HTML5 parser and serializer. ml/iri 1.1.4 IRI handling for PHP ml/json-ld 1.2.1 JSON-LD Processor for PHP monolog/monolog 3.5.0 Sends your logs to files, sockets, inboxes, databases and various web services myclabs/deep-copy 1.11.1 Create deep copies (clones) of your objects nikic/php-parser v4.18.0 A PHP parser written in PHP oscarotero/html-parser v0.1.8 Parse html strings to DOMDocument phar-io/manifest 2.0.3 Component for reading phar.io manifest information from a PHP Archive (PHAR) phar-io/version 3.2.1 Library for handling version information and constraints phpunit/php-code-coverage 9.2.30 Library that provides collection, processing, and rendering functionality for PHP code coverage information. phpunit/php-file-iterator 3.0.6 FilterIterator implementation that filters files based on a list of suffixes. phpunit/php-invoker 3.1.1 Invoke callables with a timeout phpunit/php-text-template 2.0.4 Simple template engine. phpunit/php-timer 5.0.3 Utility class for timing phpunit/phpunit 9.6.16 The PHP Unit Testing framework. psr/cache 3.0.0 Common interface for caching libraries psr/container 2.0.2 Common Container Interface (PHP FIG PSR-11) psr/event-dispatcher 1.0.0 Standard interfaces for event handling. psr/http-client 1.0.3 Common interface for HTTP clients psr/http-factory 1.0.2 Common interfaces for PSR-7 HTTP message factories psr/http-message 1.1 Common interface for HTTP messages psr/log 3.0.0 Common interface for logging libraries psr/simple-cache 3.0.0 Common interfaces for simple caching ralouphie/getallheaders 3.0.3 A polyfill for getallheaders. sebastian/cli-parser 1.0.1 Library for parsing CLI options sebastian/code-unit 1.0.8 Collection of value objects that represent the PHP code units sebastian/code-unit-reverse-lookup 2.0.3 Looks up which function or method a line of code belongs to sebastian/comparator 4.0.8 Provides the functionality to compare PHP values for equality sebastian/complexity 2.0.3 Library for calculating the complexity of PHP code units sebastian/diff 4.0.5 Diff implementation sebastian/environment 5.1.5 Provides functionality to handle HHVM/PHP environments sebastian/exporter 4.0.5 Provides the functionality to export PHP variables for visualization sebastian/global-state 5.0.6 Snapshotting of global state sebastian/lines-of-code 1.0.4 Library for counting the lines of code in PHP source code sebastian/object-enumerator 4.0.4 Traverses array structures and object graphs to enumerate all referenced objects sebastian/object-reflector 2.0.4 Allows reflection of object attributes, including inherited and non-public ones sebastian/recursion-context 4.0.5 Provides functionality to recursively process PHP variables sebastian/resource-operations 3.0.3 Provides a list of PHP built-in functions that operate on resources sebastian/type 3.2.1 Collection of value objects that represent the types of the PHP type system sebastian/version 3.0.2 Library that helps with managing the version number of Git-hosted PHP projects silverstripe/admin 2.0.13 SilverStripe admin interface silverstripe/asset-admin 2.0.4 Asset management for the SilverStripe CMS silverstripe/assets 2.0.6 SilverStripe Assets component silverstripe/campaign-admin 2.0.2 SilverStripe campaign admin interface silverstripe/closure 3.99.99 A library that can be used to serialize closures (anonymous functions) and arbitrary objects. silverstripe/cms 5.0.7 The SilverStripe Content Management System silverstripe/config 2.0.0 SilverStripe configuration based on YAML and class statics silverstripe/errorpage 2.0.2 ErrorPage component for SilverStripe CMS silverstripe/event-dispatcher 1.0.0 Publish and subscribe to events in Silverstripe CMS or your Silverstripe application silverstripe/framework 5.0.23 The SilverStripe framework silverstripe/graphql 5.0.3 GraphQL server for SilverStripe models and other data silverstripe/login-forms 5.0.0 A collection of templates for the CMS login screens silverstripe/mimevalidator 3.0.0 Checks uploaded file content roughly matches a known MIME type for the file extension. silverstripe/recipe-cms 5.0.0 SilverStripe recipe for fully featured page and asset content editing silverstripe/recipe-core 5.0.0 SilverStripe framework-only core recipe silverstripe/recipe-plugin 2.0.0 Helper plugin to install SilverStripe recipes silverstripe/reports 5.0.3 Reports module for SilverStripe CMS silverstripe/session-manager 2.0.2 Allow users to manage and revoke access to multiple login sessions across devices. silverstripe/siteconfig 5.0.2 Site wide settings administration. silverstripe/vendor-plugin 2.0.2 Allows vendor modules to expose directories to the webroot silverstripe/versioned 2.0.2 SilverStripe Versioned component silverstripe/versioned-admin 2.0.4 SilverStripe versioned admin interface sminnee/callbacklist 0.1.1 PHP class that manages a list of callbacks stripe/stripe-php v10.21.0 Stripe PHP Library symbiote/silverstripe-queuedjobs 5.0.3 A framework for defining and running background jobs in a queued manner symfony/cache v6.4.2 Provides extended PSR-6, PSR-16 (and tags) implementations symfony/cache-contracts v3.4.0 Generic abstractions related to caching symfony/config v6.4.0 Helps you find, load, combine, autofill and validate configuration values of any kind symfony/deprecation-contracts v3.4.0 A generic function and convention to trigger deprecation notices symfony/dom-crawler v6.4.0 Eases DOM navigation for HTML and XML documents symfony/event-dispatcher v6.4.2 Provides tools that allow your application components to communicate with each other by dispatching events and listening to them symfony/event-dispatcher-contracts v3.4.0 Generic abstractions related to dispatching event symfony/filesystem v6.4.0 Provides basic utilities for the filesystem symfony/finder v6.4.0 Finds files and directories via an intuitive fluent interface symfony/mailer v6.4.2 Helps sending emails symfony/mime v6.4.0 Allows manipulating MIME messages symfony/polyfill-ctype v1.28.0 Symfony polyfill for ctype functions symfony/polyfill-intl-idn v1.28.0 Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions symfony/polyfill-intl-normalizer v1.28.0 Symfony polyfill for intl's Normalizer class and related functions symfony/polyfill-mbstring v1.28.0 Symfony polyfill for the Mbstring extension symfony/polyfill-php72 v1.28.0 Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions symfony/service-contracts v3.4.1 Generic abstractions related to writing services symfony/translation v6.4.2 Provides tools to internationalize your application symfony/translation-contracts v3.4.1 Generic abstractions related to translation symfony/var-exporter v6.4.2 Allows exporting any serializable PHP data structure to plain PHP code symfony/yaml v6.4.0 Loads and dumps YAML files theseer/tokenizer 1.2.2 A small library for converting tokenized PHP source code into XML and potentially other formats ua-parser/uap-php v3.9.14 A multi-language port of Browserscope's user agent parser. webonyx/graphql-php v15.9.1 A PHP port of GraphQL reference implementation

Validations

• [X] Check that there isn't already an issue that reports the same bug
• [X] Double check that your reproduction steps work in a fresh installation of silverstripe/installer (with any code examples you've provided)

[kinglozzer]

How are you triggering the queued jobs, via CLI? If so, try running vendor/bin/sake dev/build before running vendor/bin/sake dev/tasks/ProcessJobQueuetask. I wonder if it could be because your webserver user and CLI users are different

[elmar001]

It is runned by cronjob every minute, not manual. Web server and CLI users are indeed different. However, I have already tested changing all possible permission and ownership and it did not make any difference.

Interesting, now I used vendor/bin/sake dev/build from CLI and all the issue seems to be resolved. All classes can be initialized now. Thanks heaps man! Previously, I relied on running IP+/dev/build?flush=1 from the browser only. It succeded and did not show any error. My understanding was that there is no difference between them? Or why vendor/bin/sake dev/build from CLI works but IP+/dev/build?flush=1 from the browser would not?

[kinglozzer]

It’s because in Silverstripe, caches are maintained per-user. So if your webserver user is www-data and your CLI user is myuser you’ll have two cache directories - if you create a silverstripe-cache directory in your document root you’ll see:

/path/to/docroot/silverstripe-cache/www-data/
/path/to/docroot/silverstripe-cache/myuser/

I think this is to prevent issues where, for example, www-data writes a cache file but myuser can’t overwrite it as it doesn’t own it. It’s possible to allow that in unix, but it means Silverstripe would need to make assumptions about environment, permissions etc.

We run our cronjobs as the same user as the webserver (e.g. crontab -e -u www-data), if that’s an option for you it might save future headaches.