search

burzum / cakephp-file-storage

Abstract file storage and upload plugin for CakePHP. Write to local disk, FTP, S3, Dropbox and more through a single interface. It's not just yet another uploader but a complete storage solution.
MIT License
197 stars 84 forks source link

Upgrading from v1 to v2 #190

Closed lorro closed 4 years ago

lorro commented 5 years ago

Hi

I'm looking for some help in upgrading from v1 to v2 :) Could you elaborate a little bit about the steps that need to be done to start using v2 of your plugin. I've been using the older v1 branch for a while now but decided to update to v2 because of compatibility with the newest versions of cakePHP3.

Though I think the docs are rather confusing/vague regarding the migration part, with lots of v1 code snippets. The migration part in the docs only mention which events, classes are removed but don't say anything about how it should be done now.

The ImageStorageTable class has been removed apparently, but what replaces its behavior now?

Thanks

lorro commented 5 years ago

I got some additional related question. In the docs you mention: FileStorage::deleteOldFileOnSave() is no longer called automatically in the FileStorage::afterSave() callback

What is the actual motivation behind that? :-) I actually need to call this as now my file_storage contains a lot of records with older images. It should be better they got deleted on a new upload.

I tried calling it by listening to the afterSave event, but that throws some errors unfortunately.

EventManager::instance()->on('FileStorage.afterSave', function ($event, $entity) {
    TableRegistry::get('Burzum/FileStorage.FileStorage')->deleteOldFileOnSave($entity);
});

Something obvious I'm missing?

code: 500
file: "/home/public_html/vendor/burzum/file-storage/src/Storage/Listener/AbstractListener.php"
line: 366
message: "StorageManager::get() first arg must be a non empty string!"

It has something to do with the deleting where the adapter is not passed:

Caused by: [InvalidArgumentException] StorageManager::get() first arg must be a non empty string!
#0 /home/public_html/vendor/burzum/file-storage/src/Storage/StorageTrait.php(44): Burzum\FileStorage\Storage\StorageManager::get(NULL, false)
#1 /home/public_html/vendor/burzum/file-storage/src/Model/Behavior/FileStorageBehavior.php(161): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->getStorageAdapter(NULL)
#2 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(353): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->afterDelete(Object(Cake\Event\Event), Object(Burzum\FileStorage\Model\Entity\FileStorage), Object(ArrayObject))
#3 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(330): Cake\Event\EventManager->_callListener(Array, Object(Cake\Event\Event))
#4 /home/public_html/vendor/cakephp/cakephp/src/Event/EventDispatcherTrait.php(114): Cake\Event\EventManager->dispatch(Object(Cake\Event\Event))
#5 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2379): Cake\ORM\Table->dispatchEvent('Model.afterDele...', Array)
#6 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2291): Cake\ORM\Table->_processDelete(Object(Burzum\FileStorage\Model\Entity\FileStorage), Object(ArrayObject))
#7 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1634): Cake\ORM\Table->Cake\ORM\{closure}()
#8 /home/public_html/vendor/cakephp/cakephp/src/Database/Connection.php(738): Cake\ORM\Table->Cake\ORM\{closure}(Object(Cake\Database\Connection))
#9 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1635): Cake\Database\Connection->transactional(Object(Closure))
#10 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2292): Cake\ORM\Table->_executeTransaction(Object(Closure), true)
#11 /home/public_html/vendor/burzum/file-storage/src/Model/Behavior/FileStorageBehavior.php(190): Cake\ORM\Table->delete(Object(Burzum\FileStorage\Model\Entity\FileStorage))
#12 [internal function]: Burzum\FileStorage\Model\Behavior\FileStorageBehavior->deleteOldFileOnSave(Object(Cake\ORM\Entity))
#13 /home/public_html/vendor/cakephp/cakephp/src/ORM/BehaviorRegistry.php(253): call_user_func_array(Array, Array)
#14 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2505): Cake\ORM\BehaviorRegistry->call('deleteoldfileon...', Array)
#15 /home/public_html/config/file_storage.php(23): Cake\ORM\Table->__call('deleteOldFileOn...', Array)
#16 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(353): Cake\Core\Configure\Engine\PhpConfig->{closure}(Object(Cake\Event\Event), Object(Cake\ORM\Entity), Object(Gaufrette\Filesystem))
#17 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(330): Cake\Event\EventManager->_callListener(Object(Closure), Object(Cake\Event\Event))
#18 /home/public_html/vendor/cakephp/cakephp/src/Event/EventDispatcherTrait.php(114): Cake\Event\EventManager->dispatch(Object(Cake\Event\Event))
#19 /home/public_html/vendor/burzum/file-storage/src/Model/Behavior/FileStorageBehavior.php(121): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->dispatchEvent('FileStorage.aft...', Array, Object(App\Model\Table\UserAvatarsTable))
#20 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(353): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->afterSave(Object(Cake\Event\Event), Object(Cake\ORM\Entity), Object(ArrayObject))
#21 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(330): Cake\Event\EventManager->_callListener(Array, Object(Cake\Event\Event))
#22 /home/public_html/vendor/cakephp/cakephp/src/Event/EventDispatcherTrait.php(114): Cake\Event\EventManager->dispatch(Object(Cake\Event\Event))
#23 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2047): Cake\ORM\Table->dispatchEvent('Model.afterSave', Array)
#24 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2013): Cake\ORM\Table->_onSaveSuccess(Object(Cake\ORM\Entity), Object(ArrayObject))
#25 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1920): Cake\ORM\Table->_processSave(Object(Cake\ORM\Entity), Object(ArrayObject))
#26 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1634): Cake\ORM\Table->Cake\ORM\{closure}()
#27 /home/public_html/vendor/cakephp/cakephp/src/Database/Connection.php(738): Cake\ORM\Table->Cake\ORM\{closure}(Object(Cake\Database\Connection))
#28 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1635): Cake\Database\Connection->transactional(Object(Closure))
#29 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1921): Cake\ORM\Table->_executeTransaction(Object(Closure), true)
#30 /home/public_html/vendor/burzum/file-storage/src/Storage/Listener/AbstractListener.php(356): Cake\ORM\Table->save(Object(Cake\ORM\Entity), Object(ArrayObject))
#31 /home/public_html/vendor/burzum/file-storage/src/Storage/Listener/BaseListener.php(107): Burzum\FileStorage\Storage\Listener\AbstractListener->_storeFile(Object(Cake\Event\Event))
#32 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(353): Burzum\FileStorage\Storage\Listener\BaseListener->afterSave(Object(Cake\Event\Event), Object(Cake\ORM\Entity), Object(Gaufrette\Filesystem))
#33 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(330): Cake\Event\EventManager->_callListener(Array, Object(Cake\Event\Event))
#34 /home/public_html/vendor/cakephp/cakephp/src/Event/EventDispatcherTrait.php(114): Cake\Event\EventManager->dispatch(Object(Cake\Event\Event))
#35 /home/public_html/vendor/burzum/file-storage/src/Model/Behavior/FileStorageBehavior.php(121): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->dispatchEvent('FileStorage.aft...', Array, Object(App\Model\Table\UserAvatarsTable))
#36 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(353): Burzum\FileStorage\Model\Behavior\FileStorageBehavior->afterSave(Object(Cake\Event\Event), Object(Cake\ORM\Entity), Object(ArrayObject))
#37 /home/public_html/vendor/cakephp/cakephp/src/Event/EventManager.php(330): Cake\Event\EventManager->_callListener(Array, Object(Cake\Event\Event))
#38 /home/public_html/vendor/cakephp/cakephp/src/Event/EventDispatcherTrait.php(114): Cake\Event\EventManager->dispatch(Object(Cake\Event\Event))
#39 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2047): Cake\ORM\Table->dispatchEvent('Model.afterSave', Array)
#40 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(2013): Cake\ORM\Table->_onSaveSuccess(Object(Cake\ORM\Entity), Object(ArrayObject))
#41 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1920): Cake\ORM\Table->_processSave(Object(Cake\ORM\Entity), Object(ArrayObject))
#42 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1634): Cake\ORM\Table->Cake\ORM\{closure}()
#43 /home/public_html/vendor/cakephp/cakephp/src/Database/Connection.php(738): Cake\ORM\Table->Cake\ORM\{closure}(Object(Cake\Database\Connection))
#44 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1635): Cake\Database\Connection->transactional(Object(Closure))
#45 /home/public_html/vendor/cakephp/cakephp/src/ORM/Table.php(1921): Cake\ORM\Table->_executeTransaction(Object(Closure), true)
#46 /home/public_html/src/Lib/Traits/ImageUploadsTrait.php(25): Cake\ORM\Table->save(Object(Cake\ORM\Entity))
#47 /home/public_html/src/Controller/Admin/UploadsController.php(14): App\Model\Table\UserAvatarsTable->upload('1', Object(Cake\ORM\Entity))
#48 /home/public_html/vendor/cakephp/cakephp/src/Controller/Controller.php(610): App\Controller\Admin\UploadsController->image()
#49 /home/public_html/vendor/cakephp/cakephp/src/Http/ActionDispatcher.php(120): Cake\Controller\Controller->invokeAction()
#50 /home/public_html/vendor/cakephp/cakephp/src/Http/ActionDispatcher.php(94): Cake\Http\ActionDispatcher->_invoke(Object(App\Controller\Admin\UploadsController))
#51 /home/public_html/vendor/cakephp/cakephp/src/Http/BaseApplication.php(235): Cake\Http\ActionDispatcher->dispatch(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#52 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(65): Cake\Http\BaseApplication->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response), Object(Cake\Http\Runner))
#53 /home/public_html/vendor/multidots/cakephp-rest-api/src/Middleware/RestApiMiddleware.php(54): Cake\Http\Runner->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#54 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(65): RestApi\Middleware\RestApiMiddleware->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response), Object(Cake\Http\Runner))
#55 /home/public_html/vendor/cakephp/cakephp/src/Routing/Middleware/RoutingMiddleware.php(162): Cake\Http\Runner->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#56 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(65): Cake\Routing\Middleware\RoutingMiddleware->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response), Object(Cake\Http\Runner))
#57 /home/public_html/vendor/cakephp/cakephp/src/Routing/Middleware/AssetMiddleware.php(88): Cake\Http\Runner->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#58 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(65): Cake\Routing\Middleware\AssetMiddleware->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response), Object(Cake\Http\Runner))
#59 /home/public_html/vendor/cakephp/debug_kit/src/Middleware/DebugKitMiddleware.php(53): Cake\Http\Runner->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#60 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(65): DebugKit\Middleware\DebugKitMiddleware->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response), Object(Cake\Http\Runner))
#61 /home/public_html/vendor/cakephp/cakephp/src/Http/Runner.php(51): Cake\Http\Runner->__invoke(Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#62 /home/public_html/vendor/cakephp/cakephp/src/Http/Server.php(98): Cake\Http\Runner->run(Object(Cake\Http\MiddlewareQueue), Object(Cake\Http\ServerRequest), Object(Cake\Http\Response))
#63 /home/public_html/webroot/index.php(40): Cake\Http\Server->run()
#64 {main}

Thanks.

lorro commented 5 years ago

Apparently got something to do with what happens in the afterDelete callback

In FileStorageBehavior:150, if I comment out the dispatching of FileStorage.afterDelete (I reckon this handles the deletion of the physical file), the upload works fine and also the deletion of the old file in the database. So the problem lies somewhere in that direction.

    /**
     * afterDelete callback
     *
     * @param \Cake\Event\Event $event
     * @param \Cake\Datasource\EntityInterface $entity
     * @param array $options
     * @return bool
     */
    public function afterDelete(Event $event, EntityInterface $entity, $options) {
        // $this->dispatchEvent('FileStorage.afterDelete', [
        //  'entity' => $entity,
        //  'storageAdapter' => $this->getStorageAdapter($entity->get('adapter')),
        // ], $this->_table);        
    }
lorro commented 5 years ago

Sorry for the spamming :-)

I have the impression the code in the repo 2.0 doesn't reflect the following?

The DB field file_storage.model has been renamed to file_storage.identifier.
The DB field file_storage.adapter has been renamed to file_storage.adapter_config.

I had prepared for this, but as I see it the code still references these fields as model & adapter? Or am I wrong? I reverted my database fields and functions back to the old names, and that seems to work...

burzum commented 5 years ago

This is a little hard to follow without having the application locally running.

I don't remember why I'm no longer calling the method automatically, I think because it was not really transparent to the user and could lead to accidental deletions. I think it is better if the developer makes this decision intentionally. Also very often there is additional logic tied to this.

"StorageManager::get() first arg must be a non empty string!"

Debug if the data you're passing to it is empty. Obviously it expects a string and the code in question, I guess its entity data here, doesn't have the adapter config name present.

You can also create a shell and run a cron job to delete old and orphaned records. Be aware that deleteAll() doesn't run callbacks.

About the DB changes: Yes, looks like I haven't had time to finish this but did the 2.0 release. Actually @ravage84 did a lot of the work on 2.0 to get it finished because I personally haven't had a need to update to 2.0 yet. But @ravage84 already did a migration from 1.2 to 2.0. I'll update the docs based on your input. Thank you!

ravage84 commented 5 years ago

@lorro

Though I think the docs are rather confusing/vague regarding the migration part, with lots of v1 code snippets. The migration part in the docs only mention which events, classes are removed but don't say anything about how it should be done now.

I would say the docs could need a lot of improvements, in general. :upside_down_face:

In the docs you mention: FileStorage::deleteOldFileOnSave() is no longer called automatically in the FileStorage::afterSave() callback What is the actual motivation behind that? :-)

To be hoenst I do not remember. Not even sure I was involved with that.

About the 2.0 release. We pushed through with that because we needed it for a project. Since we aren't using the plugin to its fullest it's quite likely that we missed something. @lorro instead of just fighting the symptoms, I would recommend us to invest time to check and fix it thoroughly - Code & docs.

burzum commented 5 years ago

@lorro @ravage84 any more input from you guys? Or can / should we close this?

burzum commented 4 years ago

@lorro @ravage84 closing this, if there is still a need for this please re-open it.