Parallel materialization

[GoogleCodeExporter]

Subj. The idea is to beat all the other frameworks on materialization test 
- the only test we lose now.

But first we must implement:
- http://code.google.com/p/dataobjectsdotnet/issues/detail?id=279
- http://code.google.com/p/dataobjectsdotnet/issues/detail?id=278
- http://code.google.com/p/dataobjectsdotnet/issues/detail?id=274

What exactly must be done:

1. We must add MaterializationMode to SessionConfiguration. MaterializationMode 
= Standard, Parallel, Auto (default). Auto = we detect 
count of CPU/cores & use Parallel if CpuCount > 1;

2. In case with parallel materialization we must split the result into 
batches, e.g. by 512 items.
- The first is enumerated as usual 
- When the next batch is starting to enumerate, its N-th successor (1st by 
default) is sent to parallel pre-processing (this will be desciribed 
later).
- If, when starting to enumerate the next batch, we don't have ready pre-
processed result for it, we add 1 to N (i.e. shift the pre-processing 
boundary to the further 512 items).

Why 512? I admit we'll be able to materialize ~ 500-600K items per second 
on my PC by this way. So 512 is ~ 1ms. It seems this must be configurable 
as well.

3. Parallel pre-processing includes:
- Key materialization - actually this is rather complex part requiring hash 
code computation & global key cache lookup.
- Exception aggregation. We must stop on the first exception thrown here, 
and this exception must be passed to the primary thread.
- Note that pre-processor doesn't enumerate the RecordSet by its own - it 
just gets a list of Tuples to pre-process and returns the result in the 
same form (or even in the same list). The RecordSet is enumerated by a 
primary thread only.

Original issue reported on code.google.com by alex.yakunin on 10 Jul 2009 at 7:28

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Original comment by alexande...@gtempaccount.com on 21 Jul 2009 at 12:12

• Changed state: Started

[GoogleCodeExporter]

Original comment by Dmitri.Maximov on 24 Jul 2009 at 7:08

• Changed state: Suspended

[GoogleCodeExporter]

Original comment by alex.yakunin on 15 Oct 2009 at 5:43

• Added labels: Priority-Medium
• Removed labels: Priority-Critical

[GoogleCodeExporter]

Original comment by alex.yakunin on 16 Oct 2009 at 10:01

• Removed labels: Milestone-Release4.1