enhancement : add mount function

[limbo127]

Hello, Can you implement mount function of archive ? Regards, Nicolas Prochazka

[charles-dyfis-net]

I don't wear the maintainership hat here -- but my own expectation is that this is unlikely, unless someone (maybe you?) contributes it. A FUSE driver is a lot of work.

[limbo127]

Hello, It seems go fuse api exist, as use by https://github.com/kahing/goofys For example Regards Nicolas

Le sam. 20 janv. 2018 à 02:13, Charles Duffy notifications@github.com a écrit :

I don't wear the maintainership hat here -- but my own expectation is that this is unlikely. A FUSE driver is a lot of work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/folbricht/desync/issues/3#issuecomment-359132138, or mute the thread https://github.com/notifications/unsubscribe-auth/ABgm0ASX6-CdHh6cW5J22QdGs7SvJQSMks5tMT3LgaJpZM4RkH6j .

[charles-dyfis-net]

I actually wouldn't be using that one, but go-fuse instead.

However, the library interface is only a small piece; this would still be a much larger project than all the functionality as-yet existing.

[limbo127]

ok, our needs is about to mount a desync index file that split a big raw image. 1 - desync make index.x image.raw 2 - put chunks into remote site 3 - mount index in other local site, to acces image.raw with streaming and your local cache store

Regards, Nicolas

[charles-dyfis-net]

Ahh -- so would this be the NBD driver rather than the FUSE driver? That's actually a bit easier.

(That is to say, would using /dev/nbd0 or such to access your image work?)

[limbo127]

hello, When we index a blob image, it's not , in all case, a raw image file, so it can be unusable to mount as nbd block or device. My case is, if the image is a qcow format image, i desync make on 1.qcow, mount as nbd block does not any sense. Nicolas

[charles-dyfis-net]

Hmm -- I'd actually think if you had a nbd block device with a qcow image, you could then use the libguestfs fuse driver to mount it, or pass it directly through to qemu or kvm.

But if the goal is to expose a directory with a single file via FUSE (making it a lower-impact alternative to a NBD server + NBD client), then... yeah, that's much more doable than what I thought you were asking for at first.

[limbo127]

hello, In my point of view, fuse mount is very important for casync/desync product since this allows block streaming from remote server/cloud. Fuse via http://www.secfs.net/winfsp/blog/ can be convert to windows not the case with nbd client. Regards, Nicolas

[folbricht]

One of us is going to try implement a solution which, initially at least, allows mounting an index file via FUSE. It'll be single file (in your case image.raw), not a whole directory tree in an archive.

[limbo127]

thanks :) nicolas

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Garanti sans virus. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

2018-01-26 16:04 GMT+01:00 folbricht notifications@github.com:

One of us is going to try implement a solution which, initially at least, allows mounting an index file via FUSE. It'll be single file (in your case image.raw), not a whole directory tree in an archive.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/folbricht/desync/issues/3#issuecomment-360808324, or mute the thread https://github.com/notifications/unsubscribe-auth/ABgm0CLdgHeZb-de7_DLbRf1R4TzzyFpks5tOelngaJpZM4RkH6j .

[folbricht]

@limbo127 There's now a mount-index command in desync which could be used to FUSE mount an index file. It'll make the blob in the index available underneath the mountpoint as single file. It's possible there are issues, I quite literally only tried it once with a small index so far. But perhaps it's useful to you already.

[limbo127]

Hello, thank, first test is not concluant, can you gives me a debug version ?

time $GOPATH /bin/desync make -n 16 -s /data/nicolas/dev/store 1.index /data2/tmp/windows_base.raw

[root@retd dev]# $GOPATH/bin/desync mount-index -s store 1.index mnt/1/

[root@retd dev]# time cat mnt/1/1 > /dev/null cat: mnt/1/1: Input/output error ( after 1 minutes )

regards, Nicolas

[folbricht]

Was able to trigger the error locally as well and I know where it's thrown, not sure why yet but that shouldn't be too hard to figure out.

[folbricht]

I believe that particular issue has been addressed now, and I did test it with larger files to confirm. Please try again.

[limbo127]

it works ! I will test with remote store, local cache ... do you think performance can be improved, with fuse mount ?

store is local desync mount [root@retd dev]# time cat mnt/1/1 > /dev/null real 6m39.926s user 0m0.554s sys 0m58.317s

std cat of raw image time cat /data2/tmp/windows_base.raw >/dev/null real 5m21.976s user 0m0.482s sys 0m47.903s

[folbricht]

It won't be able to beat the performance of a local disk, there's a certain amount of overhead that can't be optimized away. Also keep in mind that copying the whole file off a FUSE mount is going to be slower than just running the extract command directly.

One more change I'm planning to make is to shortcut reading zero-byte-chunks which in your use case (large image with sections of 0-bytes) may improve performance quite a bit.

[limbo127]

ok. It's already a very interesting product for us. Local cache and http serving/proxy seems to be run very well. I'm testing a big store with all our product gold image. Regards, Nicolas.

[folbricht]

@limbo127 The optimization for indexes referencing a lot of zero chunks has been added to mount-index and may help with your large and mostly blank images.