Idea: Compress data before downloading

cyrus-and / httpfs

Remote FUSE filesystem via server-side script

MIT License

62 stars 10 forks source link

Idea: Compress data before downloading #3

Open infodox opened 11 years ago

infodox commented 11 years ago

Quick idea I had the other day while using httpfs to backup data from a misbehaving server. What if the data was remotely compressed (using gzip or whatever PHP supports by default) by the httpfs script prior to downloading? Would make retrieving files a hell of a lot faster. Threading the download process might also work, though that is less certain.

Just some ideas, loving this tool! So bloody useful!

cyrus-and commented 11 years ago

Glad to hear that!

It would be (relatively) easy to patch httpfs to implement that, but that's not the main bottleneck, a full rewrite of the software is planned, we aim to implement all the common optimizations (some of them also suggested by another user: Antony G.):

HTTP persistent connections;
HTTP pipelining;
HTTP ETags;
multithreaded FUSE;
data compression.

Unfortunately we cannot say when we will have time to do that... stay tuned!

frodik commented 7 years ago

Hello, any progress in promised full rewrite ? I would really love to have those features you wrote about...

cyrus-and commented 7 years ago

@frodik unfortunately not, a rewrite from scratch would require too much work for me right now and I have very little free time.