schmittjoh / JMSSerializerBundle

Easily serialize, and deserialize data of any complexity (supports XML, JSON, YAML)
http://jmsyst.com/bundles/JMSSerializerBundle
MIT License
1.8k stars 311 forks source link

How to warmup cache for built-in types #801

Open pdugas opened 4 years ago

pdugas commented 4 years ago

I've included my src/Entity path under jms_serializer>metadata>warmup>paths>included but at runtime with a read-only filesystem, I find I'm getting an error that var/cache/prod/jms_serialzer isn't writeable. Turns out it's trying to cache metadata for the DateTime class. Is there a trick to pre-cache metadata for built-in classes?

goetas commented 4 years ago

The built-in types do not use metadata as they are managed by handlers. I thing your issue might be somewhere else, like a class that does not have a handler.

pdugas commented 4 years ago

Curious... If I adjust permissions to allow it write access to var/cache/prod/jms_serializer/ and hit the API in a browser, it produces DateTime.cache.php (content below) in that folder.

<?php return unserialize('C:37:"JMS\\Serializer\\Metadata\\ClassMetadata":427:{a:23:{i:0;a:0:{}i:1;a:0:{}i:2;a:0:{}i:3;N;i:4;N;i:5;a:0:{}i:6;N;i:7;N;i:8;b:0;i:9;N;i:10;N;i:11;N;i:12;a:0:{}i:13;a:0:{}i:14;s:72:"a:5:{i:0;s:8:"DateTime";i:1;a:0:{}i:2;a:0:{}i:3;a:0:{}i:4;i:1589986975;}";s:19:"discriminatorGroups";a:0:{}s:25:"xmlDiscriminatorAttribute";b:0;s:21:"xmlDiscriminatorCData";b:1;s:15:"usingExpression";b:0;s:25:"xmlDiscriminatorNamespace";N;s:13:"xmlRootPrefix";N;s:6:"isList";b:0;s:5:"isMap";b:0;}}');
pdugas commented 4 years ago

I guess I need to go do some reading up on handlers to understand what I'm missing.

goetas commented 4 years ago

yeah, it is weird.. would be great if you could investigate more the issue.

goetas commented 4 years ago

After a lot of investigation I manged to reproduce the issue. In my case was https://github.com/nelmio/NelmioApiDocBundle trying to get metadata for it. Do you have installed that library?

pdugas commented 4 years ago

Yes, I do. I've not gotten time to spend on it. Sorry.

patrickbussmann commented 4 years ago

I have the same issue @goetas . But its a bit more special here. 😆

I'm using Bref PHP because the software is running on Amazon Web Services Lambda. So I cant write on the disk because its locked. And to speed up everything its needed to warm up the cache. So I have this productive configuration.

jms_serializer:
    visitors:
        json_serialization:
            options:
                - JSON_UNESCAPED_SLASHES
                - JSON_PRESERVE_ZERO_FRACTION

    metadata:
        cache: file
        warmup:
            paths:
                included:
                    - '%kernel.project_dir%/src/Entity'
                    - '%kernel.project_dir%/vendor/doctrine/collections/lib/Doctrine/Common/Collections'
                    - '%kernel.project_dir%/vendor/knplabs/knp-paginator-bundle/src/Pagination'
                    - '%kernel.project_dir%/vendor/knplabs/knp-components/src/Knp/Component/Pager/Pagination'

But when I disable writing to the cache (vendor/jms/metadata/src/Cache/FileCache.php::put) by adding:

    public function put(ClassMetadata $metadata): void
    {
        echo 'NOT ALLOWED TO WRITE ANY CACHE (for ' . $metadata->name . ')';
        die();
        if (!is_writable($this->dir)) {
            throw new \InvalidArgumentException(sprintf('The directory "%s" is not writable.', $this->dir));
        }

Then I'm getting this message for:

So I solved this issue with my own CacheWarmer.

# config/services.yaml

services:
    _defaults:
        autowire: true
        autoconfigure: true

    Metadata\MetadataFactoryInterface: '@jms_serializer.metadata_factory'

    App\Cache\:
        resource: '../src/Cache'
        tags: ['kernel.cache_warmer']
<?php
// src/Cache/CacheWarmer.php
namespace App\Cache;

use Doctrine\ORM\PersistentCollection;
use Metadata\MetadataFactoryInterface;
use Symfony\Component\HttpKernel\CacheWarmer\CacheWarmerInterface;

/**
 * Class CacheWarmer
 * @package App\Cache
 */
class CacheWarmer implements CacheWarmerInterface
{
    /**
     * @var MetadataFactoryInterface
     */
    private $metadataFactory;

    public function __construct(MetadataFactoryInterface $metadataFactory)
    {
        $this->metadataFactory = $metadataFactory;
    }

    public function warmUp($cacheDir)
    {
        $classes = [
            \DateTime::class,
            PersistentCollection::class
        ];

        foreach($classes as $class)
        {
            $this->metadataFactory->getMetadataForClass($class);
        }
    }

    public function isOptional()
    {
        return true;
    }
}

And now the cache warms up and I not hitting the put method of the cache anymore. 🥳

goetas commented 4 years ago

Indeed that is a nice soltion. If you are interested in contributing it back, we could do something as

jms_serializer:
    metadata:

        warmup:
            paths:
                included:
                    - '%kernel.project_dir%/src/Entity'
                    - '%kernel.project_dir%/vendor/doctrine/collections/lib/Doctrine/Common/Collections'
            classes:
              - 'DateTime'
              - 'SomeOtherClass'

(note the new classes option)

kevincerro commented 3 years ago

Indeed that is a nice soltion. If you are interested in contributing it back, we could do something as

jms_serializer:
    metadata:

        warmup:
            paths:
                included:
                    - '%kernel.project_dir%/src/Entity'
                    - '%kernel.project_dir%/vendor/doctrine/collections/lib/Doctrine/Common/Collections'
            classes:
              - 'DateTime'
              - 'SomeOtherClass'

(note the new classes option)

Any advance on this?

I have the same issue. I want to warmup some clases inside a folder, not all.