Can't execute PDO exec() method from workers

Hi I'm having a problem inserting values in a table. I'm constantly getting php PHP Fatal error: Call to a member function exec() on a non-object.

Here are the codes which should be reproducible

ImportManager.php


<?php

date_default_timezone_set ( 'UTC' ) ;
include './ImportStacker.php' ;
include './ImportWorker.php' ;

class ImportManager {

    /**
     * Static variable to hold a single instance of the program
     * 
     * @var satic Single Instance of the program
     */
    private static $_instance = NULL ;

    /**
     * Array of workers
     * 
     * @var array 
     */
    protected $worker_pool ;

    /**
     * Array of stackables
     * 
     * @var type 
     */
    protected $work_pool ;

    /**
     * Array containing worker status
     * 
     * @var array 
     */
    protected $worker_status ;

    /**
     * Maximum number of worker
     * 
     * @var int 
     */
    protected $worker_max ;

    /**
     * The current worker number pointer
     * 
     * @var int
     */
    protected $worker_number ;

    /**
     * Initialize the class properties, read the settings and store it in an array
     * 
     * @param int $loop_interval
     * @param string $log_file
     * @param string $setting_file
     */
    private function __construct () {

    }

    /**
     * Get the single self instance of the class
     * 
     * @return object Self instance
     * 
     */
    public static function getInstance () {

        if ( !(self::$_instance instanceof ImportManager) ) {
            self::$_instance = new ImportManager() ;
        }

        return self::$_instance ;
    }

    /**
     * Main event loop of the program
     * 
     */
    public function execute () {

            $this->worker_max = 4;

            $this->submitJobToPool();

    }

    /**
     * 
     * Create a worker pool and submit stackables in it
     * 
     * @param Stackable $stackable
     * @return \Stackable
     */
    protected function submit ( Stackable $stackable ) {

        /**
         * Reset the worker pointer
         */
        if ( $this->worker_number === $this->worker_max ) {
            $this->worker_number = 0 ;
        }

        /**
         * If there are less workers than the max number of workers allowed
         * then spawn a new worker and stack the work in it
         */
        if ( count ( $this->worker_pool ) < $this->worker_max ) {

            $id = count ( $this->worker_pool ) ;

            /**
             * Create the workers
             */
            $this->worker_pool[ $id ] = new ImportWorker() ;

            /**
             * start the workers
             */
            $worker_retval = $this->worker_pool[ $id ]->start ( PTHREADS_INHERIT_NONE ) ;

            /**
             * if an error has occured then shutdown the program
             */
            if ( strlen ( $worker_retval ) > 5 ) {
                $this->supervisor->log ( posix_getpid () ,
                                         $worker_retval ) ;
                exit ( 1 ) ;
            }

            if ( $this->worker_pool[ $id ]->stack ( $stackable ) ) {
                return $stackable ;
            } else {
                $this->supervisor->log ( posix_getpid () ,
                                         sprintf ( "failed to stack onto selected worker %s" ,
                                                   $this->worker_pool[ $id ]->name ) .
                        E_USER_WARNING ) ;
            }
        }

        /**
         * If no more worker can be spawned then select one from the available pool
         */
        if ( ($select = $this->worker_pool[ $this->worker_number++ ] ) ) {

            if ( $select->stack ( $stackable ) ) {
                return $stackable ;
            } else {
                $this->supervisor->log ( posix_getpid () ,
                                         sprintf ( "failed to stack onto selected worker %s" ,
                                                   $select->name ) .
                        E_USER_WARNING ) ;
            }
        } else {
            $this->supervisor->log ( posix_getpid () ,
                                     "Failed to select available worker" . E_USER_WARNING ) ;
        }
    }

    /**
     * Shutdown the pool of workers cleanly, retain the exit status locally
     * 
     */
    protected function shutdown () {

        foreach ( $this->worker_pool as
                    $worker ) {

            $this->worker_status[ $worker->getThreadId () ] = $worker->shutdown () ;
        }
    }

    protected function submitJobToPool (  ) {

        $this->worker_number = 0 ;
        $this->work_pool     = array () ;

        $symbolList   = "" ;
        $loop_counter = 0 ;

        $this->work_pool[] = $this->submit ( new ImportStacker (  ) ) ;
        $this->work_pool[] = $this->submit ( new ImportStacker (  ) ) ;        
        $this->work_pool[] = $this->submit ( new ImportStacker ( ) ) ;        
        $this->work_pool[] = $this->submit ( new ImportStacker (  ) ) ;

        /**
         * Synchronize workers
         */
        $this->shutdown () ;

    }

}

?>

Here is the ImportWorker.php


<?php

date_default_timezone_set ( 'UTC' ) ;

class ImportWorker
        extends Worker {

    /**
     * Unique name or id of the worker
     * 
     * @var string
     */
    public $name ;

    /**
     * Database connection handle
     * 
     * @var object 
     */
    public static $connection ;

    /**
     * Initialize the worker thread
     * 
     */
    public function __construct () {

    }

    /**
     * Start the worker thread from here
     * 
     */
    public function run () {

        $this->name = sprintf ( "%s (%lu)" ,
                                __CLASS__ ,
                                $this->getThreadId () ) ;

        try {
            self::$connection = new PDO ( 'mysql:host=localhost;dbname=myDb' ,
                                          'user' ,
                                          'pass' ) ;

            /* self::$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); */
        } catch ( PDOException $exceptionObject ) {

            return "DATABASE CONNECTION ERROR : And error occured while making a connection to the database.
                                  In Class " . __CLASS__ . "
                Error Code : " . $exceptionObject->getCode () . ". Error Message : " . $exceptionObject->getMessage () ;
        }
    }

    public function insertValues ( $query ) {

        try {

            self::$connection->exec ( $query ) ;
            return "Values Inserted Successfully." ;
        } catch ( PDOException $exceptionObject ) {

            return "DATABASE CONNECTION ERROR : Sorry cannot insert values into table 
                Error Code : " . $exceptionObject->getCode () . ". Error Message : " . $exceptionObject->getMessage () ;
        }
    }

    public function getConnection () {

        return self::$connection ;
    }

}

Here is ImportStacker.php


<?php

class ImportStacker
        extends Stackable {

    public $food;

    public function __construct (
    ) {

        $this->food = "APPLE";

    }

    /**
     * Execute the stackable
     */
    public function run () {

        date_default_timezone_set ( 'UTC' ) ;

        $query = $this->insertTableValues();

        $this->worker->insertValues($query);

    }

    public static function insertTableValues (  ) {

        return "INSERT INTO `test1` (id,
                                    symbol)
                                     VALUES (1, 'mo'), (2, 'do'), (3, 'lo') " ;
    }

}

And lastly here is Run.php


#!/usr/bin/php
<?php

require_once './ImportManager.php';

ImportManager::getInstance()->execute();

Here is the schema for the table test1


+-------+-------------+------+-----+---------+-------+                                                                                                                  
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(3)      | NO   | PRI | 0       |       |
| name  | varchar(10) | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+

CREATE TABLE `test1` (
  `id` int(3) NOT NULL DEFAULT '0',
  `name` varchar(10) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8

When i run the program using

php Run.php

I'm getting an error

PHP Fatal error:  Call to a member function exec() on a non-object in /var/www/html/test/ImportWorker.php on line 76
PHP Fatal error:  Call to a member function exec() on a non-object in /var/www/html/test/ImportWorker.php on line 76
PHP Fatal error:  Call to a member function exec() on a non-object in /var/www/html/test/ImportWorker.php on line 76
PHP Fatal error:  Call to a member function exec() on a non-object in /var/www/html/test/ImportWorker.php on line 76

Please provide any advice on how to solve this error. I've tried removing all the statics but if i make the connection object non-static then it can't access it either.

I would really appreciate any help.

Thanks, Maxx

So can anyone provide any help on this. I'm really stuck with this error. :-( . And the code that i've posted should be reproducible.

Sorry Maxx, I have a list of a million things to do, I'll get to it today or tomorrow, for sure ...

@chibisuke, @bwoebi any input ? (sorry for the ping)

Sure no problem. I totally understand your concern. Have a look at it when you get any spare time, , don't worry.

And i really appreciate your concern, afterall managing the whole pthreads community single handed is a mammoth task for just one guy.

Your reproduction code is incomplete. Its accessing $this->supervisor where is was never defined, and the class isn't even part of your code.

Try reducing the footprint of your reproduction code to whats strictly nescessary. If it happens with one worker / one stack item, reduce it to that. if it doesn't - mention it (important to see if there's a race somewhere). In the best case, try to get ride of that ImportManager class. it adds complexity to the reproduction code.

The basic idea of reproduction code is to be as small as possible, while still showing the problem you need help with. Also try nailing it down to a single problem. What happens if you replace PDO by some other random object? is the problem still occuring? if yes - get ride of the DB connection. if you remove something and the problem is gone... readd it, and remove something else, until only parts that make the problem disappear are left.

if reproduction code exceeds what is about a page in your browser - don't inline it in your post. make a gist from it and post the gist. if you inline larger code it makes it hard for ppl to get an overview.

I had a look at the code anyway and there are a few things I'm seeing, None of those look like a root cause of the problem however.

1.) in your submit() method you're calling exit() The way pthreads is implemented exit() only terminates the current thread, but not the entire process.

2.) you're using PTHREADS_INHERIT_NONE to create a minimal execution environment. Try removing the argument and start with the normal environment. If it works you can later on still try reducing the memory footprint.

3.) While its technically perfectly fine to do ++ inside of the [] of an array ($this->worker_pool[ $this->worker_number++ ]) it makes your codes readability bad. try to avoid something like that if you expect your code to not work on first try.

4.) Decide for a propper coding guideline and then follow it. Also ask ppl that work with you to do so. You're mixing like 3 or 4 different coding styles.

@chibisuke Really sorry for the inconvenience. I've trimmed most of the unnecessary tidbits from the reproduction code now. But you know if i remove the PDO declaration altogether then there is no more errors anymore. Here is a slightly trimmed version of ImportManager.php


<?php

date_default_timezone_set ( 'UTC' ) ;
include './ImportStacker.php' ;
include './ImportWorker.php' ;

class ImportManager {

    private static $_instance = NULL ;

    protected $worker_pool ;

    protected $work_pool ;

    protected $worker_status ;

    protected $worker_max ;

    protected $worker_number ;

    private function __construct () {

    }

    public static function getInstance () {

        if ( !(self::$_instance instanceof ImportManager) ) {
            self::$_instance = new ImportManager() ;
        }

        return self::$_instance ;
    }

   public function execute () {
          $this->worker_max = 4;

            $this->submitJobToPool();

    }
    protected function submit ( Stackable $stackable ) {

        if ( $this->worker_number === $this->worker_max ) {
            $this->worker_number = 0 ;
        }

        if ( count ( $this->worker_pool ) < $this->worker_max ) {

            $id = count ( $this->worker_pool ) ;

            $this->worker_pool[ $id ] = new ImportWorker() ;

            $worker_retval = $this->worker_pool[ $id ]->start (  ) ;

            /**
             * if an error has occured then shutdown the program
             */
            if ( strlen ( $worker_retval ) > 5 ) {
                var_dump ( posix_getpid () ,
                                         $worker_retval ) ;

            }

            if ( $this->worker_pool[ $id ]->stack ( $stackable ) ) {
                return $stackable ;
            } else {
                var_dump( posix_getpid () ,
                                         sprintf ( "failed to stack onto selected worker %s" ,
                                                   $this->worker_pool[ $id ]->name ) .
                        E_USER_WARNING ) ;
            }
        }

        /**
         * If no more worker can be spawned then select one from the available pool
         */
        if ( ($select = $this->worker_pool[ $this->worker_number ] ) ) {

            $this->worker_number ++;

            if ( $select->stack ( $stackable ) ) {

                return $stackable ;
            } else {

                var_dump ( posix_getpid () ,
                                         sprintf ( "failed to stack onto selected worker %s" ,
                                                   $select->name ) .
                        E_USER_WARNING ) ;
            }
        } else {
            var_dump ( posix_getpid () ,
                                     "Failed to select available worker" . E_USER_WARNING ) ;
        }
    }

    protected function shutdown () {

        foreach ( $this->worker_pool as
                    $worker ) {

            $this->worker_status[ $worker->getThreadId () ] = $worker->shutdown () ;
        }
    }

    protected function submitJobToPool (  ) {

        $this->worker_number = 0 ;
        $this->work_pool     = array () ;

        $symbolList   = "" ;
        $loop_counter = 0 ;

        $this->work_pool[] = $this->submit ( new ImportStacker (  ) ) ;

        /**
         * Synchronize workers
         */
        $this->shutdown () ;

    }

}

?>

The error is still there but its showing only one line now , as i've stacked only one worker this time. Which shows that the error is something other than race conditions.

PHP Fatal error:  Call to a member function exec() on a non-object in /var/www/test/ImportWorker.php on line 69

And of course thanks for the advice. You know the thing is that i'm really a novice programmer. Still learning and making mistakes :-). So i'm still very much unaccustomed to standard coding guidelines, as i thought i can remain aloof from it. But it looks like i was wrong. Its about time to learn some

Anyways thanks a lot for the help. I really appreciate it.

I have to close this issue, I have just committed a fix for the use of static variables in general to help improve stability, the example code was never a reproduce script; all I can tell from reading it is that it uses a lot of statics which were probably the root cause of the behaviour. This should not happen any longer if you stick to the rules.

If you still feel you have a bug, please do provide reproducing code that is executable and concise in order that I can investigate further.

Sure no problem , i understand your point . I have fixed that problem by using static only once in storing the database connection and now it doesn't crashes. Although i get quite a number of php free malloc memory corruption but it doesn't crash the program anymore.

On Wed, Sep 10, 2014 at 5:58 PM, Joe Watkins notifications@github.com wrote:

I have to close this issue, I have just committed a fix for the use of static variables in general to help improve stability, the example code was never a reproduce script; all I can tell from reading it is that it uses a lot of statics which were probably the root cause of the behaviour. This should not happen any longer if you stick to the rules.

If you still feel you have a bug, please do provide reproducing code that is executable and concise in order that I can investigate further.

— Reply to this email directly or view it on GitHub https://github.com/krakjoe/pthreads/issues/222#issuecomment-55107672.

sent via 100% recycled electrons from my mobile command center.

You should pull from master, test again, might be better ;)

I will once i update the codebase which will happen in a few weeks

On Wed, Sep 10, 2014 at 6:25 PM, Joe Watkins notifications@github.com wrote:

You should pull from master, test again, might be better ;)

— Reply to this email directly or view it on GitHub https://github.com/krakjoe/pthreads/issues/222#issuecomment-55110477.

sent via 100% recycled electrons from my mobile command center.

I have very similar situation as @MaxxAstral . But instead of creating new PDO connection, I want to use Laravel DB:connection (based on PDO only) as my app is written with Laravel with ORM based queries. which are written in Models. So this native way will not work. @krakjoe could you please share insight on how can it be done. Appreciate it.

krakjoe / pthreads

Can't execute PDO exec() method from workers #222