search

AmericanCouncils / AyamelResourceApiServer

RESTful API server for managing multimedia resource objects for language learning applications.
http://api.ayamel.org
MIT License
4 stars 5 forks source link

Improve underlying filesystem handling #139

Open evillemez opened 11 years ago

evillemez commented 11 years ago

The ideal solution would allow declaring multiple filesystems, and easy migrating of files between filesystem implementations. I see this as happening like so:

ResourceFileManager - takes several "tagged" implementations of file systems, extra methods for migrating specific files across systems, and migrating entire resources. GaufretteFilesystem - Implements FilesystemInterface for any Gaufrette instance.

I would use https://github.com/KnpLabs/KnpGaufretteBundle to configure multiple storage locations, and configure them to power multiple instances of FilesystemManager.

The end result would let configuration be declared for separate pools of files, for example:

This would let me implement an interface such as:

$resourceFileManager->registerFilesystem('upload', $gaufretteLocalStorage);
$resourceFileManager->registerFilesystem('web', $gaufretteAwsS3Storage);

$resourceFileManager->migrateResource('web', $resource);
$resourceFileManager->migrateFile('transcoding', $resource->content->files[0]);

In dev and testing, all of these can be overridden to simply point to different directories on a local filesystem.

Managing what files belong where, and when they get moved would be fairly straight forward:

A few things may need to be refactored for this to work well, but none of them major. Rather than storing an internalUri for files (which assumed they were on local storage, the database should also store a boolean for managed, and internalUri will need to support multiple "schemes" that the manager understands. If the file reference in the database is "managed", then the ResourceFileManager should know how to find it, if not, we must assume it's a reference to a file not under the systems control.

evillemez commented 9 years ago

Just an update - if this ever does get implemented or addressed, use Flysystem instead of Gaufrette. It's better and more active:

http://flysystem.thephpleague.com

Aside from the underlying library, the concept remains basically the same, but Flysystem already does a lot of it. Flysystem also allows registering custom schemes, so storing the full scheme urls, such as web:///path/to/file.mp3 or cdn:///path/to/file.mp3 would probably be a viable solution.