rsandhu1 / datafabric-griffin

Automatically exported from code.google.com/p/datafabric-griffin
0 stars 0 forks source link

Griffin very slow on directory listings #5

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago

Hi,

I am running Griffin 0.9.0 on a slave server - and directory listings are very 
slow (about 1s per entry).

The speed was all fine with Griffin 0.8.4 - so I'm just guessing this might be 
related to the switch from Jargon Classic to Jargon Core.

Would there be an explanation as to the change of how directory listings are 
done?

And any recommendations for a fix?

Thanks a lot in advance for any help.

Cheers,
Vlad

PS: The slave and the master server are geographically separated and ping 
reports latency of about 16ms.  When doing strace on the irodsAgent process on 
the slave server, I could see it was very active while the GridFTP client was 
waiting for the directory listing.  I have tried with different directory sizes 
and the time to get the listing grows linearly at ~ 1s per entry.

Original issue reported on code.google.com by Vladimir.Mencl on 3 Apr 2013 at 10:36

GoogleCodeExporter commented 8 years ago
The old Griffin didn't check permission of each individual file when listing; 
it assumed the user can read and write all children files if the user could 
browse a directory. The new Griffin uses the new Jargon to list and also check 
permission of each file, hence adds up to overhead.
Mike Conway has started some benchmarking work on jargon, please see 
"[iROD-Chat:9753] jargon performance notes"

Original comment by shunde.p...@gmail.com on 4 Apr 2013 at 3:37

GoogleCodeExporter commented 8 years ago
Hi Vlad,
I've put up a new 0.9.0 tar ball which have fixed this issue (hopefully). Could 
you download it and give it a try?

Original comment by shunde.p...@gmail.com on 10 Apr 2013 at 12:29

GoogleCodeExporter commented 8 years ago
Hi Shunde,

Thanks - speed has MUCH improved after this updated - all fine now.

Many thanks - and sorry about the delay in getting to test this.

Cheers,
Vlad

Original comment by Vladimir.Mencl on 18 Apr 2013 at 11:40

GoogleCodeExporter commented 8 years ago

Original comment by shunde.p...@gmail.com on 24 Apr 2013 at 3:20