Open a17t opened 2 years ago
hi @a17t This sounds like what I've been looking for, great to see someone has already thought about s3 compatible storage adapters. Can you point me to your fork? I'd like to give it a spin on a test server here.
Hey @dwehrmann, feel free to take a look into my fork of paperless-ng. The branch smartOpen contains my (trivial) changes.
Please be aware, its definitely in a PoC state, so do not use it for any important stuff.
To use S3 (or any storage provider supported by smart_open) just set the corresponding url in the config. For now the checks for existence are only implemented updated to allow for s3. I just tested it by setting PAPERLESS_MEDIA_ROOT to a s3u:// url, which was enough to have paperless-ng store all document data in reference s3 bucket.
@a17t have you been able to build your fork into a docker image? I'd love to run this alongside the original container to test things out. My docker machine has limited storage so I try to put everything on my Minio machine. This S3 compatibility feature is exactly what I'm looking for.
@funkybunch I will upload an image to dockerhub so you can try. Please be aware, this is in a PoC state, it might break or do strange stuff.
https://hub.docker.com/r/a17t/paperless-ngx-s3
here you go
@funkybunch did it work for you? Did you experience any issues?
Hello everyone,
in search for a "personal document storage" I stumbled upon the original paperless a few years ago and used it for a while. A few weeks ago I picked up that challenge of digitalizing personal documents once again and found paperless-ng. So far great stuff!
But I would like to abstract storage from raw filesystem storage. I have different things in mind, but as a starting point I thought about S3 compatible storage. While researching I found smart_open (https://github.com/RaRe-Technologies/smart_open) which overrides the open function to allow urls to different storage systems, one of those being S3, in that function call.
So I quickly included it into a fork of paperless-ng and voila, paperless docs are stored in a local S3 storage.
@jonaswinkler What do you think about that change? Feel free to look into my fork, it should just contain a few changes.
I would really like to bring this back to mainline paperless-ng, especially with the perspective of introducing further storage backends to paperless-ng. (e.g. ms onedrive, or other managed cloud storage)
Looking forward to some discussion, and stay healthy
Simon