HydratorFactory.php blows up when running in Docker

[eluzgin]

Multiple developers are experiencing the same issue when using this framework in Docker - it blows up application with an error: ContextErrorException: Warning: rename(/var/www/trackingfirst/app/cache/dev/doctrine/odm/mongodb/Hydrators/TrackingFirstMainBundleDocumentUserHydrator.php.5901663c46c199.48165620,/var/www/trackingfirst/app/cache/dev/doctrine/odm/mongodb/Hydrators/TrackingFirstMainBundleDocumentUserHydrator.php): Operation not permitted in /var/www/trackingfirst/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Hydrator/HydratorFactory.php line 415

Fix it already!

[eluzgin]

Here is the fix for you:

 -            rename($tmpFileName, $fileName);
 +            copy($tmpFileName, $fileName);
 +            unlink($tmpFileName);

[alcaeus]

Operation not permitted

That looks like a permission issue on your end. Could you share a little bit of information about your docker container? Of interest would be whether the cache folder used for doctrine is added to the Docker image while copying the sources into the container.

As for the suggested "fix", as I've previously indicated in #1495 that rename has the advantage of being an atomic operation, while copy can lead to processes reading a partially written file, blowing the application up with a parse error.

[malarzm]

Maybe jumping the gun a bit, but @eluzgin

Fix it already!

Is not the best attitude towards something you probably make money with and don't pay yourself for. "Could you please fix this?" sounds way better and is friendlier.

[eluzgin]

@malarzm sorry guys, no hard feelings. I'm just a fellow developer here and was getting frustrated with that error, which kept preventing me from getting my work done on a project.

It doesn't seem to be permission related, just Docker related. And this is not an isolated issue since it was reported by other developers as well. The local workaround is to replace single replace operation by copy, unlink operations. It fixes this problem.

Now I understand your concern for atomic operations and single transaction but it's better that it works then doesn't.

[alcaeus]

@eluzgin As I mentioned, please share some information about your docker container. I'm running a couple of ODM applications in with Docker as well and I don't have the issue at all. It's quite simple: if the system can touch('/some/path/to/file.tmp') but not rename('/some/path/to/file.tmp', '/some/path/to/file'), the culprit is somewhere else. This is not something we should fix by replacing rename with copy.

[alcaeus]

Closed due to lack of feedback. @eluzgin please reopen with the info I asked for if you still need help.

[eluzgin]

I don't have time for this. The decision have been made to move away from this php framework any way. However you might still see this issue reported by other users since Docker environment is gaining popularity.

-Eugene

On Wed, May 17, 2017 at 2:01 AM, Andreas notifications@github.com wrote:

Closed due to lack of feedback. @eluzgin https://github.com/eluzgin please reopen with the info I asked for if you still need help.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/doctrine/mongodb-odm/issues/1583#issuecomment-302016015, or mute the thread https://github.com/notifications/unsubscribe-auth/AAiR0k6nCgzE0A8bm6uA7bvjHzOej3y6ks5r6qlDgaJpZM4NJsL0 .

[soderluk]

Hi, I'm experiencing the same issue, but without Docker. We're using Vagrant on Virtualbox with a PHP Lumen API as back-end and React front-end. This is totally random, and happens once in a while. This is really annoying, since the folder is writable, and every hydrator class can be written to it, so it's not a permission issue on the server.

Using doctrine/mongodb-odm: 1.1.5 and alcaeus/mongo-php-adapter: 1.0.9 with laravel/lumen-framework: v5.4.5

The filename seems to be some kind of cached file. The original hydrator file exists already, so I think that might have something to do with it, but I'm not sure.

Here's the stack trace:

[2017-08-31 04:57:11] lumen.ERROR: ErrorException: rename(/path/to/api/storage/doctrine/hydrators/NordLumenOAuth2DoctrineODMDocumentsAccessTokenHydrator.php.59a797279da171.87882666,/path/to/api/storage/doctrine/hydrators/NordLumenOAut
h2DoctrineODMDocumentsAccessTokenHydrator.php): Operation not permitted in /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Hydrator/HydratorFactory.php:422
Stack trace:
#0 [internal function]: Laravel\Lumen\Application->Laravel\Lumen\Concerns\{closure}(2, 'rename(/path/to/api...', '/path/to/api/vendor...', 422, Array)
#1 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Hydrator/HydratorFactory.php(422): rename('/path/to/api/storag...', '/path/to/api/storag...')
#2 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Hydrator/HydratorFactory.php(146): Doctrine\ODM\MongoDB\Hydrator\HydratorFactory->generateHydratorClass(Object(Doctrine\ODM\MongoDB\Mapping\ClassMetadata), 'Nord
LumenOAuth2...', '/path/to/api/storag...')
#3 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Hydrator/HydratorFactory.php(460): Doctrine\ODM\MongoDB\Hydrator\HydratorFactory->getHydratorFor('Nord\\Lumen\\OAut...')
#4 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/UnitOfWork.php(2692): Doctrine\ODM\MongoDB\Hydrator\HydratorFactory->hydrate(Object(Nord\Lumen\OAuth2\Doctrine\ODM\Documents\AccessToken), Array, Array)
#5 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Persisters/DocumentPersister.php(649): Doctrine\ODM\MongoDB\UnitOfWork->getOrCreateDocument('Nord\\Lumen\\OAut...', Array, Array, Object(Nord\Lumen\OAuth2\Doctri
ne\ODM\Documents\AccessToken))
#6 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/Persisters/DocumentPersister.php(514): Doctrine\ODM\MongoDB\Persisters\DocumentPersister->createDocument(Array, NULL, Array)
#7 /path/to/api/vendor/doctrine/mongodb-odm/lib/Doctrine/ODM/MongoDB/DocumentRepository.php(186): Doctrine\ODM\MongoDB\Persisters\DocumentPersister->load(Array)
#8 /path/to/api/vendor/nordsoftware/lumen-oauth2-doctrine-odm/src/Repositories/AccessTokenRepository.php(18): Doctrine\ODM\MongoDB\DocumentRepository->findOneBy(Array)
#9 /path/to/api/vendor/nordsoftware/lumen-oauth2-doctrine-odm/src/Storages/AccessTokenStorage.php(49): Nord\Lumen\OAuth2\Doctrine\ODM\Repositories\AccessTokenRepository->findByToken('IodHVZRlX0vwr8y...')
#10 /path/to/api/vendor/league/oauth2-server/src/ResourceServer.php(114): Nord\Lumen\OAuth2\Doctrine\ODM\Storages\AccessTokenStorage->get('IodHVZRlX0vwr8y...')
#11 /path/to/api/vendor/nordsoftware/lumen-oauth2/src/OAuth2Service.php(55): League\OAuth2\Server\ResourceServer->isValidRequest(true, NULL)
#12 /path/to/api/vendor/nordsoftware/lumen-oauth2/src/OAuth2Service.php(65): Nord\Lumen\OAuth2\OAuth2Service->validateAccessToken()
#13 /path/to/api/Access/Services/UserService.php(73): Nord\Lumen\OAuth2\OAuth2Service->getResourceOwnerId()
#14 /path/to/api/Access/Guards/OAuth2Guard.php(37): App\Access\Services\UserService->getCurrentUser()
#15 /path/to/api/vendor/illuminate/auth/GuardHelpers.php(49): App\Access\Guards\OAuth2Guard->user()
#16 /path/to/api/vendor/illuminate/auth/GuardHelpers.php(59): App\Access\Guards\OAuth2Guard->check()
#17 /path/to/api/Access/Http/Middleware/Authenticate.php(44): App\Access\Guards\OAuth2Guard->guest()
#18 /path/to/api/vendor/illuminate/pipeline/Pipeline.php(148): App\Access\Http\Middleware\Authenticate->handle(Object(Illuminate\Http\Request), Object(Closure))
#19 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#20 /path/to/api/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(32): call_user_func(Object(Closure), Object(Illuminate\Http\Request))
#21 /path/to/api/vendor/illuminate/pipeline/Pipeline.php(102): Laravel\Lumen\Routing\Pipeline->Laravel\Lumen\Routing\{closure}(Object(Illuminate\Http\Request))
#22 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(778): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#23 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(624): Laravel\Lumen\Application->sendThroughPipeline(Array, Object(Closure))
#24 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(598): Laravel\Lumen\Application->handleFoundRoute(Array)
#25 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(532): Laravel\Lumen\Application->handleDispatcherResponse(Array)
#26 [internal function]: Laravel\Lumen\Application->Laravel\Lumen\Concerns\{closure}(Object(Illuminate\Http\Request))
#27 /path/to/api/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(52): call_user_func(Object(Closure), Object(Illuminate\Http\Request))
#28 /path/to/api/vendor/nordsoftware/lumen-fractal/src/FractalMiddleware.php(42): Laravel\Lumen\Routing\Pipeline->Laravel\Lumen\Routing\{closure}(Object(Illuminate\Http\Request))
#29 /path/to/api/vendor/illuminate/pipeline/Pipeline.php(148): Nord\Lumen\Fractal\FractalMiddleware->handle(Object(Illuminate\Http\Request), Object(Closure))
#30 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#31 /path/to/api/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(32): call_user_func(Object(Closure), Object(Illuminate\Http\Request))
#32 /path/to/api/vendor/nordsoftware/lumen-cors/src/CorsService.php(114): Laravel\Lumen\Routing\Pipeline->Laravel\Lumen\Routing\{closure}(Object(Illuminate\Http\Request))
#33 /path/to/api/vendor/nordsoftware/lumen-cors/src/CorsMiddleware.php(45): Nord\Lumen\Cors\CorsService->handleRequest(Object(Illuminate\Http\Request), Object(Closure))
#34 /path/to/api/vendor/illuminate/pipeline/Pipeline.php(148): Nord\Lumen\Cors\CorsMiddleware->handle(Object(Illuminate\Http\Request), Object(Closure))
#35 [internal function]: Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Illuminate\Http\Request))
#36 /path/to/api/vendor/laravel/lumen-framework/src/Routing/Pipeline.php(32): call_user_func(Object(Closure), Object(Illuminate\Http\Request))
#37 /path/to/api/vendor/illuminate/pipeline/Pipeline.php(102): Laravel\Lumen\Routing\Pipeline->Laravel\Lumen\Routing\{closure}(Object(Illuminate\Http\Request))
#38 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(778): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#39 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(534): Laravel\Lumen\Application->sendThroughPipeline(Array, Object(Closure))
#40 /path/to/api/vendor/laravel/lumen-framework/src/Concerns/RoutesRequests.php(475): Laravel\Lumen\Application->dispatch(NULL)
#41 /path/to/api/public/index.php(28): Laravel\Lumen\Application->run()
#42 {main}

[malarzm]

Reopening due to @soderluk's update

[alcaeus]

This may be related to https://github.com/doctrine/doctrine2/issues/6713#issuecomment-333325870 - duplicate generation of these files would definitely cause the error described by @soderluk. There's nothing we can do, except recommend a different generation strategy (which might be inconvenient in dev, but really the only solution for an operating system issue).

[soderluk]

OK, good to know. I thought as much, that there's probably not much that can be done to this issue. The files are all there, the directories/files are writable, and 99.99999999% of the time everything just works. I just wanted to report the issue, if anyone else has the same problem without Docker.

[TomCan]

Facing a similar problem using mongodb-odm in Symfony through docker. Also fails on the rename, but in my case he's giving a "No such file or directory" instead of the "Operation not permitted". It seems like some sort of locking issue with docker, caused by thing that rename does under the hood.

What I did to overcome this problem, was to set the cache folder of Symfony to a folder that was not part of the volume shared with the docker container (so instead of /var/cache, I've set it /dev/shm in kernel.php)

I've previously ran into a similar problem with docker when trying to run MongoDB through docker-compose with the mongodb data folder set to a folder inside my project root. This caused MongoDB to crash as it could not create the files (although it actually did create the files). My best guess is that it has got something to do with the way docker exposes the volume/filesystem to the container.

[stale[bot]]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in a week if no further activity occurs. Thank you for your contributions.

[binary-data]

@TomCan Thanks! Set Hydrators directory to /tmp (outside of a shared folder) on Virtualbox VM and the error was gone!