search

ucberkeley / bce

Berkeley Common Environment provides a common Linux computational environment for classwork and research.
Apache License 2.0
13 stars 6 forks source link

download speeds can be _very_ slow in BCE #61

Open paciorek opened 9 years ago

paciorek commented 9 years ago

I've run into this earlier this summer as well in spring-2015...

Downloading a test 150 Mb file using wget in BCE downloads at ~50 KB/s when the host is a Linux machine. Natively downloading on the host Linux machine it is 100 MB/s. Similarly in BCE on a Mac laptop, downloading is ~50 KB/s.

davclark commented 9 years ago

I'm wondering if this is a network or rather a filesystem issue. I've had issues in linux in general when, for example, writing to a USB key. Does your filesystem otherwise perform close to native speed?

paciorek commented 9 years ago

I think the filesystem is fine - I just copied a big-ish file from one place to another from within the BCE filesystem and didn't notice anything awry. If anyone has suggestions of how to test filesystem performance in a standard way, let me know.

I'll note that using wget or scp, the download speeds start at a high rate (say 10 MB/s or 500 KB/s) but then quickly drop to like 30 KB/s.

On Sun, Aug 30, 2015 at 9:47 PM, Dav Clark notifications@github.com wrote:

I'm wondering if this is a network or rather a filesystem issue. I've had issues in linux in general when, for example, writing to a USB key. Does your filesystem otherwise perform close to native speed?

— Reply to this email directly or view it on GitHub https://github.com/ucberkeley/bce/issues/61#issuecomment-136250813.

aculich commented 9 years ago

Are shared folders involved? If so, try removing all shared folders and check the download speeds. The shared folders are known to have very slow performance (among other problems).

See also: Comparing Filesystem Performance in Virtual Machines

paciorek commented 9 years ago

No, I haven't enabled shared folders in the VM where I am seeing the slow downloading.

On Mon, Aug 31, 2015 at 1:58 PM, Aaron Culich notifications@github.com wrote:

Are shared folders involved? If so, try removing all shared folders and check the download speeds. The shared folders are known to have very slow performance (among other problems).

See also: Comparing Filesystem Performance in Virtual Machines http://mitchellh.com/comparing-filesystem-performance-in-virtual-machines

— Reply to this email directly or view it on GitHub https://github.com/ucberkeley/bce/issues/61#issuecomment-136497296.