search

ramsey / uuid

:snowflake: A PHP library for generating universally unique identifiers (UUIDs).
https://uuid.ramsey.dev
MIT License
12.46k stars 501 forks source link

Improve preloading compatibility on 8.x #386

Closed pentium10 closed 2 years ago

pentium10 commented 3 years ago

We have a preloading compiled and this is the only library that is reported at runtime errors. 

2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Provider\Time\SystemTimeProvider: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Provider/Time/SystemTimeProvider.php</b> on line <b>25</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Provider\Node\FallbackNodeProvider: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Provider/Node/FallbackNodeProvider.php</b> on line <b>25</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Lazy\LazyUuidFromString: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Lazy/LazyUuidFromString.php</b> on line <b>53</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Provider\Dce\SystemDceSecurityProvider: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Provider/Dce/SystemDceSecurityProvider.php</b> on line <b>36</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Uuid: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Uuid.php</b> on line <b>41</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
Default
2021-09-01 21:05:00.742 EEST<b>Warning</b>: Can't preload unlinked class Ramsey\Uuid\Converter\Time\PhpTimeConverter: Unknown type dependencies in <b>/var/www/html/vendor/ramsey/uuid/src/Converter/Time/PhpTimeConverter.php</b> on line <b>43</b><br />
Default
2021-09-01 21:05:00.742 EEST<br />
ramsey commented 3 years ago

@pentium10 I'm not sure how to replicate this or what to do to fix it. Can you provide some instructions on how your preloading is configured and/or pointers on what I can do to improve compatibility with it?

pentium10 commented 3 years ago

Hi @ramsey I create a repo https://github.com/pentium10/uuid386 with the exact setup to see and investigate.

for me the warnings show up also on Cloud Run, and also locally when using docker build

as a guidance, I don't know what improvement must be done, but if you check the composer.lock file, you will see that other repos are fine, and only yours provide these warnings, somehow the PHP autoloader cannot figure out the right files to import.

mallardduck commented 3 years ago

@pentium10 can you include all dependencies in the composer.json this example repo isn't quite "complete" for anyone to replicate it. Seems to be missing these dependencies, and the preload relies on the paths within docker once built. It'd be more portable if it used __DIR__ to find the vender files to preload.

Edit: Maybe it's just missing ramsey/uuid tho, hard to say without making a guess that could be wrong tho.

gquemener commented 3 years ago

FYI I've been able to replicate the mentioned issue with the provided repository :+1:

ramsey/uuid is an indirect dependency, still it is required, so no worry on this side.

$ composer why ramsey/uuid
google/cloud-bigquery  v1.22.2  requires  ramsey/uuid (^3.0|^4.0)      
ramsey/uuid            4.2.1    replaces  rhumsaa/uuid (self.version)

I'm trying to reduce the scenario (so far, there are 19 dependencies).

lcobucci commented 3 years ago

@pentium10 it seems you have missing items on your generated list (e.g. TimeProviderInterface depends on the class Ramsey\Uuid\Type\Time).

You have to have every single dependency on your preload file if you want to use opcache_compile_file().

There is another alternative (include/require) as documented in https://www.php.net/manual/en/opcache.preloading.php

mallardduck commented 3 years ago

Good catch there! Long week....😮‍💨 I was able to build the image and can see the errors on my side.

mallardduck commented 3 years ago

Yeah - looking things over I agree with @lcobucci. There appears to be items either missing, or potentially out of order, in the preload script being used.

lcobucci commented 3 years ago

Yeah - looking things over I agree with @lcobucci. There appears to be items either missing, or potentially out of order, in the preload script being used.

Out of order is not a problem with that strategy but missing items is definitely not okay 😁

exussum12 commented 3 years ago

Yeah looks like missing items. https://github.com/Ayesh/Composer-Preload is a useful plugin which uses the composer autoloader to make sure everything is resolved

mallardduck commented 3 years ago

Hmm. For some reason I thought order mattered. Good to know I guess.

Even still though @lcobucci , that poses a question of the theory about missing ones. Because that seems wrong now too - if you check one of the errors...

2021-09-01 21:05:00.742 EESTWarning: Can't preload unlinked class Ramsey\Uuid\Provider\Time\SystemTimeProvider: Unknown type dependencies in /var/www/html/vendor/ramsey/uuid/src/Provider/Time/SystemTimeProvider.php on line 25

So the file it's having an issue with: https://github.com/ramsey/uuid/blob/main/src/Provider/Time/SystemTimeProvider.php#L25

This line is just the class with an implements, which should mean that the TimeProviderInterface is missing from their preload script. However that does exist already, so 🤔

SystemTimeProvider: https://github.com/pentium10/uuid386/blob/master/includes/preload.php#L154 TimeProviderInterface: https://github.com/pentium10/uuid386/blob/master/includes/preload.php#L151

gquemener commented 3 years ago

I don't have much knowledge about opcache preloading (I just understand the basic), however I've noticed that symfony doesn't rely on opcache_compile_file() to preload classes : https://github.com/symfony/symfony/blob/5.4/src/Symfony/Component/DependencyInjection/Dumper/Preloader.php#L40.

However, I've been able to setup a more minimal environment with ramsey/uuid which doesn't produce warning : https://github.com/pentium10/uuid386/pull/1

lcobucci commented 3 years ago

The example I gave was the dependency of TimeProviderInterface (Type\Time). That one needs to be loaded.

mallardduck commented 3 years ago

@lcobucci - Right that's the same example I'm pointing to as well. And the lines I references account for that interface needing to be loaded.

Edit: Ohh - I see what you mean now.

TimeProviderInterface itself needs Ramsey\Uuid\Type\Time which isn't being preloaded.

It's unfortunate the error isn't very clear about that being the real issue if that's indeed the case.

ramsey commented 3 years ago

Preloading with opcache_compile_file() doesn't invoke the autoloader. What I've done in preload scripts is require Composer's auto-generated classmap and loop through everything, calling class_exists() on the class name. This is probably a highly inefficient way of doing it, but class_exists() does invoke the autoloader, and if you have opcache turned on, it'll cache the opcodes (without needing to use opcache_compile_file()).

It also means the order you load them doesn't matter. 😄

pentium10 commented 3 years ago

I've added opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/Time.php'); but no difference.

gquemener commented 3 years ago

IIUC, you must be missing a file in your preload.php file.

To be sure, could you remove all line from your preload.php that contains ramsey and add the following lines (which should contain all the ramsey/uuid php files) :

opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/AbstractArray.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/AbstractCollection.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/AbstractSet.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/ArrayInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Collection.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/CollectionInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/DoubleEndedQueue.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/DoubleEndedQueueInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/CollectionMismatchException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/InvalidArgumentException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/InvalidSortOrderException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/NoSuchElementException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/OutOfBoundsException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/UnsupportedOperationException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Exception/ValueExtractionException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/GenericArray.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/AbstractMap.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/AbstractTypedMap.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/AssociativeArrayMap.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/MapInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/NamedParameterMap.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/TypedMap.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Map/TypedMapInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Queue.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/QueueInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Set.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Tool/TypeTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Tool/ValueExtractorTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/collection/src/Tool/ValueToStringTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/BinaryUtils.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Builder/BuilderCollection.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Builder/DefaultUuidBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Builder/DegradedUuidBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Builder/FallbackBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Builder/UuidBuilderInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/CodecInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/GuidStringCodec.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/OrderedTimeCodec.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/StringCodec.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/TimestampFirstCombCodec.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Codec/TimestampLastCombCodec.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Number/BigNumberConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Number/DegradedNumberConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Number/GenericNumberConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/NumberConverterInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Time/BigNumberTimeConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Time/DegradedTimeConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Time/GenericTimeConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/Time/PhpTimeConverter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Converter/TimeConverterInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/DegradedUuid.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/DeprecatedUuidInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/DeprecatedUuidMethodsTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/BuilderNotFoundException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/DateTimeException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/DceSecurityException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/InvalidArgumentException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/InvalidBytesException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/InvalidUuidStringException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/NameException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/NodeException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/RandomSourceException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/TimeSourceException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/UnableToBuildUuidException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/UnsupportedOperationException.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Exception/UuidExceptionInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/FeatureSet.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Fields/FieldsInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Fields/SerializableFieldsTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/CombGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/DceSecurityGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/DceSecurityGeneratorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/DefaultNameGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/DefaultTimeGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/NameGeneratorFactory.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/NameGeneratorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/PeclUuidNameGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/PeclUuidRandomGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/PeclUuidTimeGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/RandomBytesGenerator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/RandomGeneratorFactory.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/RandomGeneratorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/RandomLibAdapter.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/TimeGeneratorFactory.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Generator/TimeGeneratorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Guid/Fields.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Guid/Guid.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Guid/GuidBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Lazy/LazyUuidFromString.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Math/BrickMathCalculator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Math/CalculatorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Math/RoundingMode.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Nonstandard/Fields.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Nonstandard/Uuid.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Nonstandard/UuidBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Nonstandard/UuidV6.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Dce/SystemDceSecurityProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/DceSecurityProviderInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Node/FallbackNodeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Node/NodeProviderCollection.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Node/RandomNodeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Node/StaticNodeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Node/SystemNodeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/NodeProviderInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Time/FixedTimeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/Time/SystemTimeProvider.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Provider/TimeProviderInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/Fields.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/FieldsInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/NilTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/NilUuid.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidBuilder.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidV1.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidV2.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidV3.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidV4.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/UuidV5.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/Validator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/VariantTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Rfc4122/VersionTrait.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/Decimal.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/Hexadecimal.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/Integer.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/NumberInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/Time.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Type/TypeInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Uuid.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/UuidFactory.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/UuidFactoryInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/UuidInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Validator/GenericValidator.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/Validator/ValidatorInterface.php');
opcache_compile_file('/var/www/html/vendor/ramsey/uuid/src/functions.php');

Have you tried to rely on composer classmap to dynamically find these files (instead of having a static preload.php)?

pentium10 commented 3 years ago

BTW we run this in a serverless environment, and we use the preload script cached, prebuilt.

I went into another error:

<b>Warning</b>: Can't preload class Ramsey\Uuid\Math\BrickMathCalculator with unresolved initializer for constant ROUNDING_MODE_MAP in <b>/var/www/html/vendor/ramsey/uuid/src/Math/BrickMathCalculator.php</b> on line <b>32</b><br />

lcobucci commented 3 years ago

@pentium10 it seems you might still have missing dependencies on your file (dependencies of dependencies).

There's nothing wrong in this library that prevents preloading, it just looks like the strategy you picked won't work for you. I'd encourage you to try to use a different one (composer autoloader + class_exist() may help the generation of the preload file).

I'd suggest closing this issue as way, @ramsey, as there is no action required from your end IMHO.