search

Zhoutall / stasis

Automatically exported from code.google.com/p/stasis
Other
0 stars 0 forks source link

Poor log performance #2

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Read existing log entries during normal operation.

What is the expected output? What do you see instead?

The log maintains multiple file handles, one of which is used for writes. 
Reading from the log causes the write handle to flush.  The write handle is
opened with O_SYNC.  Therefore, reading recently written log entries incurs
synchronous disk writes.

Original issue reported on code.google.com by sears.ru...@gmail.com on 13 May 2009 at 1:57

GoogleCodeExporter commented 9 years ago 
Remote log implementations will also have to deal with this problem.  Otherwise
Tabort() will need to round-trip data to the log server.

Original comment by sears.ru...@gmail.com on 1 Jun 2009 at 9:34

GoogleCodeExporter commented 9 years ago

Original comment by sears.ru...@gmail.com on 4 Nov 2009 at 3:30

GoogleCodeExporter commented 9 years ago 
There are a few related issues worth addressing:

1) Log appends do not take advantage of multiprocessors, since they grab a big 
lock.
 This could be fixed by having log append generate an entry, calculate its size, then
reserve a slot in the log buffer.  It would then memcpy into the buffer, and
decrement some reference count when done.  This should give us group commit for 
free,
since these operations could be performed even if another thread is inside of a
synchronous disk operation.

2) Log truncation is currently too expensive.  It should not need to copy data 
in
order to truncate. 

3) It would be nice if the log integrated in nicely with Stasis' I/O handle 
stuff

Original comment by sears.ru...@gmail.com on 10 Dec 2009 at 1:52

GoogleCodeExporter commented 9 years ago 
r1350 completes the refactoring that part (1) was blocked on.

Original comment by sears.ru...@gmail.com on 23 Jan 2010 at 2:38

GoogleCodeExporter commented 9 years ago 
This problem also limits Stasis' sequential log bandwidth.

Original comment by sears.ru...@gmail.com on 23 Jan 2010 at 2:40

GoogleCodeExporter commented 9 years ago 
The new file pool log better leverages multiprocessors, and greatly decreases 
the cost of log truncation.  However, it still relies upon global mutexes.

Original comment by sears.ru...@gmail.com on 2 May 2011 at 10:21