how to install version 4 with composer?

[vesper8]

I've tried every possible combination inside the composer.json and it always comes up with this error:

• Installation request for zoopcommerce/shard 4.0.3 -> satisfiable by zoopcommerce/shard[4.0.3].
• zoopcommerce/shard 4.0.3 requires doctrine/mongodb-odm dev-hotfix719 as 1.0.0-BETA9 -> no matching package found.

I've tried all these different combinations: { "require": { "doctrine/mongodb-odm": "dev-hotfix719", "zoopcommerce/shard": "4.0.3", },

{ "require": { "doctrine/mongodb-odm": "dev-master", "zoopcommerce/shard": "4.0.3", },

{ "require": { "zoopcommerce/shard": "4.0.3", },

{ "require": { "doctrine/mongodb-odm": "dev-hotfix719 as 1.0.0-BETA9", "zoopcommerce/shard": "4.0.3", },

Any help please?

[joshystuart]

Hi @vesper8,

I believe there have been some changes with some of the Shard dependancies, in particular with the doctrine/mongodb-odm repo. I'll take a look and get back to you asap!

[joshystuart]

@vesper8,

Can you please paste your whole composer.json?

I think you may need to add:

"repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/superdweebie/mongodb-odm"
        }
    ],

That's because we have a fork of the mongodb-odm repo which includes some necessary updates.

[vesper8]

Thanks a lot! That did the trick. I got a lot of suggestions that I wasn't getting before though.

Here's my complete composer.json

{
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/superdweebie/mongodb-odm"
        }
    ],
    "require": {
        "zoopcommerce/shard": "4.0.3",
        "ritero/twitch-sdk": "dev-master",
        "predis/predis": "dev-master"
    },
    "minimum-stability": "dev",
    "autoload": {
        "psr-0": {
            "FOG":        "fog/"
        }
    }
}

And here are the suggestions I got from composer:

zendframework/zend-stdlib suggests installing zendframework/zend-eventmanager (To support aggregate hydrator usage)
zendframework/zend-stdlib suggests installing zendframework/zend-serializer (Zend\Serializer component)
zendframework/zend-servicemanager suggests installing zendframework/zend-di (Zend\Di component)
zendframework/zend-math suggests installing ircmaxell/random-lib (Fallback random byte generator for Zend\Math\Rand if OpenSSL/Mcrypt extensions are unavailable)
zendframework/zend-math suggests installing ext-gmp (If using the gmp functionality)
zendframework/zend-json suggests installing zendframework/zend-server (Zend\Server component)
zendframework/zend-json suggests installing zendframework/zend-http (Zend\Http component)

I don't use Zend Framework btw, I use a custom php framework.

If you think I'd be wise to add additional stuff to my composer.json and pay due to those suggestions please let me know.

And... if you have a minute more @crimsonronin, I noticed a few "issues" on the previous repo and this where you were looking for a way to pass the config / connection string to the manifest and I am having the same problem.. I've literally spent all day trying to get this to work and I can't ever get the $documentManager = $manifest->getModelManager(); to return anything but null.

More details as to what I've done and what I'm trying to do here: https://github.com/superdweebie/doctrine-extensions/issues/12#issuecomment-39136175

I'd really appreciate if you could help me be on my way and get a working document manager again... if possible without the need to create a custom factory.. or if that really is the best way.. could you possibly add a working example on pastebin or something?

Thank you very much

[joshystuart]

It's weird that composer isn't using our custom repositories out of the box (I'll look into that!).

Don't worry about the composer suggestions, it will work fine without them. Shard uses some parts of Zend Framwork 2, most notably the ServiceManager, which is why we are getting some suggestions in the first place.

A fair few things have changed since you were using the superdweebie/doctrine-extensions. I believe you should be able to create a shard manifest like so:

$manifest = new Manifest(
            [
                'models' => [
                    'Your\Project\DocumentModel' => __DIR__ . '/Your/Project/DocumentModel'
                ],
                'extension_configs' => [
                    'extension.serializer' => true,
                    'extension.odmcore' => true
                ],
            ]
        );

Then to get the modelManager you now use:

$manifest->getServiceManager()->get('modelmanager');

To get the serializer or unserializer you use

$unserializer = $manifest->getServiceManager()->get('unserializer');
$serializer = $manifest->getServiceManager()->get('serializer');

What really helped me understand everything was by looking at the examples in the tests eg https://github.com/zoopcommerce/shard/blob/master/tests/Zoop/Shard/Test/Serializer/SerializerTest.php

Let me know how you go!

[joshystuart]

By the way, when you use code samples on github comments you can use 3 backticks to format it nicely. eg.

\`\`\`
//some code remove the \ from the ticks....they are just there to escape them
\`\`\`

[vesper8]

Thanks! Funny I was googling exactly that when you sent the message.. you read my mind!

The problem I'm having is that I don't know how to pass the connection string and the custom folders for my proxies/hydrators.

From what I understood.. if I use Shard then I don't need to set up a seperate connection to doctrine odm like I am currently doing like this:

use Doctrine\MongoDB\Connection;
use Doctrine\ODM\MongoDB\Configuration;
use Doctrine\ODM\MongoDB\DocumentManager;
use Doctrine\ODM\MongoDB\Mapping\Driver\AnnotationDriver;

if ( ! file_exists($file = $config["framework_extensions_dir"] . "SHARED" . '/vendor/autoload.php')) {
    throw new RuntimeException('Install dependencies to run this script.');
}

$doctrineLoader = require_once $file;

$doctrineLoader->add('Documents', $config['doctrine_data_folder']);
$doctrineLoader->register();

$doctrineConnection = new Connection("mongodb://" . $config['DATABASE']['USER'][$config['mongodb']] . ":" . $config['DATABASE']['USER'][$config['mongodb']] . "@" . $config['DATABASE']['READ_LIST'][$config['mongodb']][0] . ":27017/" . $config['mongodb']);

$doctrineConfig = new Configuration();

$doctrineConfig->setProxyDir($config['doctrine_cache_folder'] . 'Proxies');
$doctrineConfig->setProxyNamespace('Proxies');
$doctrineConfig->setHydratorDir($config['doctrine_cache_folder'] . 'Hydrators');
$doctrineConfig->setHydratorNamespace('Hydrators');
$doctrineConfig->setDefaultDB($config['mongodb']);

$doctrineConfig->setMetadataDriverImpl(AnnotationDriver::create($config['doctrine_data_folder'] . 'Documents'));

AnnotationDriver::registerAnnotationClasses();

$dm = DocumentManager::create($doctrineConnection, $doctrineConfig);

Assuming I correctly understand that I can replace the above code completely now that I've opted to use Shard... then how do I pass my custom config (default DB, username/password for the mongodb instance, custom hydrators/proxies location) to Shard?

It seems that following your default manifest code can't possibly work since I use custom settings and a secured mongodb instance.

[superdweebie]

@crimsonronin thanks for stepping in and helping out @vesper8.

With the creation of a document manager, shard just wants to you config a factory that will return the document manager you want to use. So, you can do something as simple as:

use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class MyDocumentManagerFactory implements FactoryInterface
{
    public function createService(ServiceLocatorInterface $serviceLocator)
    {
        //do something to create an instance of $myDocumentManager

        return $myDocumentManager;
    }
}

Then you can grab it from the service manager as shown by @crimsonronin.

In regard to composer. Custom repositories will only be used when defined in the root composer.json. We'll get rid of that need when doctrine/mongodb-odm beta 10 is tagged. At present shard needs fixes in doctrine/mongodb-odm added since beta 9, but we don't want to just use master because of instability problems.

[joshystuart]

Ah yes;

To explain further:

In the manifest config do:

$manifest = new Manifest([
    'models' => [
        'Your\Project\DocumentModel' => __DIR__ . '/Your/Project/DocumentModel'
    ],
    'extension_configs' => [
        'extension.serializer' => true,
        'extension.odmcore' => true
    ],
    'service_manager_config' => [
        'factories' => [
            'modelmanager' => 'My\DocumentManager\Factory'
        ]
    ]
]);

Then in My\DocumentManager\Factory.php

use Doctrine\MongoDB\Connection;
use Doctrine\ODM\MongoDB\Configuration;
use Doctrine\ODM\MongoDB\DocumentManager;
use Doctrine\ODM\MongoDB\Mapping\Driver\AnnotationDriver;
use Zend\ServiceManager\FactoryInterface;
use Zend\ServiceManager\ServiceLocatorInterface;

class MyDocumentManagerFactory implements FactoryInterface
{

    public function createService(ServiceLocatorInterface $serviceLocator)
    {
        if (!file_exists($file = $config["framework_extensions_dir"] . "SHARED" . '/vendor/autoload.php')) {
            throw new RuntimeException('Install dependencies to run this script.');
        }

        $doctrineLoader = require_once $file;

        $doctrineLoader->add('Documents', $config['doctrine_data_folder']);
        $doctrineLoader->register();

        $doctrineConnection = new Connection("mongodb://" . $config['DATABASE']['USER'][$config['mongodb']] . ":" . $config['DATABASE']['USER'][$config['mongodb']] . "@" . $config['DATABASE']['READ_LIST'][$config['mongodb']][0] . ":27017/" . $config['mongodb']);

        $doctrineConfig = new Configuration();

        $doctrineConfig->setProxyDir($config['doctrine_cache_folder'] . 'Proxies');
        $doctrineConfig->setProxyNamespace('Proxies');
        $doctrineConfig->setHydratorDir($config['doctrine_cache_folder'] . 'Hydrators');
        $doctrineConfig->setHydratorNamespace('Hydrators');
        $doctrineConfig->setDefaultDB($config['mongodb']);

        $doctrineConfig->setMetadataDriverImpl(AnnotationDriver::create($config['doctrine_data_folder'] . 'Documents'));

        AnnotationDriver::registerAnnotationClasses();

        return DocumentManager::create($doctrineConnection, $doctrineConfig);
    }

}

Or something to that effect. (I haven't tested your code there).

[joshystuart]

@superdweebie since @vesper8 isn't using ZF2, maybe he will have to just use a closure instead so that he can pass in his config?

[joshystuart]

@vesper8 if you want you could do something like this to get your custom config in without zf2:

$manifest = new Manifest([
    'models' => [
        'Your\Project\DocumentModel' => __DIR__ . '/Your/Project/DocumentModel'
    ],
    'extension_configs' => [
        'extension.serializer' => true,
        'extension.odmcore' => true
    ],
    'service_manager_config' => [
        'factories' => [
            'modelmanager' => function() use ($config) {
                if (!file_exists($file = $config["framework_extensions_dir"] . "SHARED" . '/vendor/autoload.php')) {
                    throw new RuntimeException('Install dependencies to run this script.');
                }

                $doctrineLoader = require_once $file;

                $doctrineLoader->add('Documents', $config['doctrine_data_folder']);
                $doctrineLoader->register();

                $doctrineConnection = new Connection("mongodb://" . $config['DATABASE']['USER'][$config['mongodb']] . ":" . $config['DATABASE']['USER'][$config['mongodb']] . "@" . $config['DATABASE']['READ_LIST'][$config['mongodb']][0] . ":27017/" . $config['mongodb']);

                $doctrineConfig = new Configuration();

                $doctrineConfig->setProxyDir($config['doctrine_cache_folder'] . 'Proxies');
                $doctrineConfig->setProxyNamespace('Proxies');
                $doctrineConfig->setHydratorDir($config['doctrine_cache_folder'] . 'Hydrators');
                $doctrineConfig->setHydratorNamespace('Hydrators');
                $doctrineConfig->setDefaultDB($config['mongodb']);

                $doctrineConfig->setMetadataDriverImpl(AnnotationDriver::create($config['doctrine_data_folder'] . 'Documents'));

                AnnotationDriver::registerAnnotationClasses();

                return DocumentManager::create($doctrineConnection, $doctrineConfig);
            }
        ]
    ]
]);

You could clean that up by moving most of the DocumentManager creation into a factory of it's own and just pass in the config as a param.

[superdweebie]

A closure is possible, but personally I think it makes the manifest config look very messy.

It is also fairly straight forward to add a config service to the service manager. Eg:

$manifest = new Manifest([
    'models' => [
        'Your\Project\DocumentModel' => __DIR__ . '/Your/Project/DocumentModel'
    ],
    'extension_configs' => [
        'extension.serializer' => true,
        'extension.odmcore' => true
    ],
    'service_manager_config' => [
        'factories' => [
            'modelmanager' => 'My\DocumentManager\Factory',
            'myconfig'          => 'My\Config\Factory'
        ]
    ]
]);

Then write the config factory, pulling values from a config file/class, whatever. Then config can be accessed with $serviceLocator->get('myconfig') in the document manager factory, and also in other locations as needed.

[joshystuart]

That's much more elegant. The closure will get things running quickly, then tidy up with factories.

Btw, ```php gives some nice code coloring in the comment ;)

[superdweebie]

Thanks fort he ```php tip

[vesper8]

Thanks so much for the help... I know have a working documentManager again.

I hope you don't mind but I've immediately stumbled onto another error.

When trying to apply toArray or toJson on a doctrine document I get this error:

Fatal error: Call to undefined method Doctrine\ODM\MongoDB\Mapping\ClassMetadata::getSerializer() in /var/www/_framework_extensions/SHARED/vendor/zoopcommerce/shard/lib/Zoop/Shard/Serializer/Serializer.php on line 86

looking at the function

    public function isSerializableField($field, ClassMetadata $metadata)
    {
        $serializerMetadata = $metadata->getSerializer();

        if (isset($serializerMetadata['fields'][$field]['serializeIgnore']) &&
            $serializerMetadata['fields'][$field]['serializeIgnore']
        ) {
            return false;
        }

        return true;
    }

I can print_r the $metadata and it contains my document class for the document I am trying to serialize. But it has no function getSerializer... why would it? Should it? I didn't see anything about having to add some special annotation to my classes in order for them to be able to use serializer? And this much was working before.. on version 3.0

Thanks again!

[superdweebie]

@vesper8 I'm pretty sure I know what the problem is. Shard extends class metadata, so you're document manager needs to configured to use an alternate ClassMetadataFactory. An example of the magic line is here: https://github.com/zoopcommerce/shard/blob/master/lib/Zoop/Shard/ODMCore/DevDocumentManagerFactory.php#L47

The value should be set to https://github.com/zoopcommerce/shard/blob/master/lib/Zoop/Shard/ODMCore/Extension.php#L51

Hope that fixes it.

(The reason it was changed is because without extending metadata, if you are caching metadata, which should do in production, then the values would get dropped when waking up from cache).

[vesper8]

That was it! Excellent guess! Thanks!

I will have to figure out how to make my own factories because even though things are working.. I am getting some errors in my error log complaining about

Argument 1 passed to Zoop\Shard\Core\BootstrappedEventArgs::__construct() must be an instance of Zoop\Shard\Core\ModelManager, instance of Doctrine\ODM\MongoDB\DocumentManager given

I'm also running into a problem where when I unserialize using fromJson it causes an error because it says it doesn't have the class name for the embedded document. Anyway I don't have the exact error but I think it may have to do because in my annotation for my embedded documents it was

/** @ODM\ReferenceMany(targetDocument="MyDocument", simple=true) */

I think it's the "simple=true" which might be causing the problem? Haven't been able to confirm yet.

Oh and.. you mentionned that I should cache metadata... is this something I would have to do manually by adding a caching layer? Or is that something I can enable in doctrine?

Last question... I'm confused about what it says in your unserializer methods

mode:
Must be either unserialize_update or unserialize_patch. If unserialize_update is used, the unserialized document will replace any existing document in the db with the same id. If unserialize_patch is used the unserialized data will be merged with any existing document in the db.

I don't want it to do either an update or a merge when unserializing... the whole reason I am trying to implement Shard's serializing is so I can store/retrieve my documents from Redis without losing the references to my embedded documents. I don't want it to affect the database upon unserializing... is this going to happen now? The whole point was to NOT hit the mongodb... if it's going to hit it every time I unserialize then I have failed at picking the right solution here...

[joshystuart]

Just a quick point on the embedded document error, I found that you have to specify full qualified names eg.

/** @ODM\ReferenceMany(targetDocument="My\DocumentFolder\MyDocument", simple=true) */

Also, @superdweebie can talk about this more, but shard won't alter the document in the DB unless you persist and flush it. The difference between update and patch just tells the unserializer how to treat the document eg either merge, or replace.

[superdweebie]

@vesper8 about the first error. Your document manager should implement the ModelManager interface. This is the beginning of efforts to abstract shard out so it is not so closely tied to mongodb-odm. You can use this class: https://github.com/zoopcommerce/shard/blob/master/lib/Zoop/Shard/ODMCore/ModelManager.php

With unserialize, it has been designed primarility for taking a serialized document from a client (eg, browser) and hydrating that document so it can be managed by the ODM. As part of that, it will normally check to see if the document already exists in the db. The patch and update modes tell the unserializer how to resolve any differeces between the unserialized document and one that might already exist in the db. If you want to make sure the db isn't hit at all, then pass the $document parameter to unserizlize. Eg:

$doc = $unserializer->fromJson('{"my": "json string"}', null, new My\Document\Class)

This will fill the new instance of your document class with the data from the json string without touching the db. Note, however, that the document will not be managed by the ODM. You'll need to call $dm->persist($doc) for that to happen.

[vesper8]

Thanks a lot for the clarification there, I understand much better now.

I was able to get rid of my funky connection and figure out how to create my own DM factory. Thanks a lot for providing the ModelManager class.. it certainly saved me time and helped me understand.

Unfortunately I'm still having the same error as before.. here's the full error text this time

Fatal error: Uncaught exception 'InvalidArgumentException' with message 'Association name expected, 'images' is not an association.' in /vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Mapping/ClassMetadataInfo.php:1749 Stack trace: 
#0 /vendor/zoopcommerce/shard/lib/Zoop/Shard/Serializer/Unserializer.php(235): Doctrine\ODM\MongoDB\Mapping\ClassMetadataInfo->getAssociationTargetClass('images') 
#1 /vendor/zoopcommerce/shard/lib/Zoop/Shard/Serializer/Unserializer.php(191): Zoop\Shard\Serializer\Unserializer->getTargetClass(Object(Zoop\Shard\ODMCore\ClassMetadata), Array, 'images') 
#2 /vendor/zoopcommerce/shard/lib/Zoop/Shard/Serializer/Unserializer.php(147): Zoop\Shard\Serializer\Unserializer->unserializeCollection(Array, Object(Zoop\Shard\ODMCore\ClassMetadata), Object(Documents\ViewFacesOfGaming), 'images', 'unserliaze_patc...') 
#3 in /vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Mapping/ClassMetadataInfo.php on line 1749

I'm using the full qualified name (I think?) in my document class:

    /** @ODM\ReferenceMany(targetDocument="Documents\FogImage") */
    protected $images = array();

Most of my classes extend "master classes".. for example I have a class for images, one for videos and one for files and they all extend a class named FogUpload

<?
namespace Documents;

use Doctrine\ODM\MongoDB\Mapping\Annotations as ODM;

require_once("FogUpload.php");

/** @ODM\Document(collection="Documents\FogImage") */
class FogImage extends FogUpload
{
    /** @ODM\Int */
    protected $filesize;

    /** @ODM\String */
    protected $type;

    /** @ODM\Int */
    protected $width;

    /** @ODM\Int */
    protected $height;

    /** @ODM\Int */
    protected $thumbsCreated;

    function __construct() {

    }

}

If you have any idea what the problem is here well.. you've already been so helpful.. thank you very much!!

[superdweebie]

@vesper8 does your parent document have the annotation @MappedSuperclass? See http://doctrine-mongodb-odm.readthedocs.org/en/latest/reference/annotations-reference.html#mappedsuperclass

Also, I'd suggest you use PSR0 directory structure and let composer's autoloader handle things rather than using require.

[vesper8]

right on the money again! thanks!

and.. would you mind showing me an example of what you mean with using psr0 instead of using requires? I am currently using PSR4 to specify my own custom namespace for my own files.. but I don't know how to set it to replace the requires? I thought psrX was only to map folders and namespaces and not to actually download packages

[superdweebie]

Yep, PSR0 maps folders/namespaces, which can then be used by an autoloader, such as the composer autoloader. To configure your own namespaces for the composer autoloader, add to your composer.json file. eg: https://github.com/zoopcommerce/shard/blob/master/composer.json#L42

or you can add them at runtime, eg: https://github.com/zoopcommerce/shard/blob/master/tests/bootstrap.php#L21

You'll find proper docs on the composer site.