Threading

[jferard]

Reminder:

• one should write ods files sequentially;
• LO needs the styles in content.xml to be defined before they are used (that means that a style-name attribute mut refer to a style previously defined);
• basic use case of FastODS implies threading: one thread gathers data (e.g. performs a query on a database) and put it on a bus; one thread reads data from bus and writes it to file (this is the FastODS part).

Now, under some assumptions, it is possible to split FastODS work into two threads:

• one thread builds HeavyTableRows and put it on a bus;
• one thread reads the rows from bus and writes it to file.

These assumptions are:

• all styles are defined when the first table row is written;
• a row is never modified after it was put on the bus (there is a kind of flush)

I think there is only one way to discover if it will really speed up the file creation: try!

[jferard]

The currrent design doesn't fit to threading.

The process is quite strict: the data is written at the very end of the process. All the objects (tables, rows, cells, styles) are first created by the user, then transcripted to XML format and written to the file.

The aim is to make the process more flexible, allowing to flush data at any time, but to keep the current behavior in the case of a single flush at the end.

Here's a possible kind of "producer-consumer" design:

• the table has a flush(boolean endOfTable) method that flushes the rows to the bus. If endOfTable is true, then the end of the table XML is written.
• the document has a flush() method that flushes all the document to the bus.
• the bus is read by a thread and written to the stream
• under the hood, the document is "Observable", and has a unique "Observer", OdsFileWriter, which has access to the output stream and to the document.

Note 1: all the files of the ods archive, except content.xml, must be written before content.xml. That means that the styles have to be inserted manually in the document.

Note 2: this would free some memory, if the rows are set to null in the document when sent to the bus.

[jferard]

This is the most hard feature to implement for 0.3.0, so this issue will be the first in the next iteration. I think now that the point is not speed but memory: a flush method would allow to free memory fast, e.g. after each table is complete.

The flush should be a method from OdsFileWriter:

• OdsFileWriter.flushRows to flush current rows
• OdsFileWriter.flushTables to flush current tables.

[jferard]

Here are a few steps:

• [ ] make all styles and configuration accessible to static creation in ods elements. Currently, styles are dynamically added to ods elements when used. It's a good thing, but we should be able to create styles in ods elements before using them.
• [ ] implement the flush method without freeing memory or using a distinct thread. Just create a "write on demand" feature.
• [ ] implement the memory release of flushed rows. A rolling list could be a solution here.
• [ ] move the writing in a distinct thread. A very simple Producer-Consumer-Bus design.

[jferard]

• [x] make all styles and configuration accessible to static creation in ods elements. Currently, styles are dynamically added to ods elements when used. It's a good thing, but we should be able to create styles in ods elements before using them.
• [ ] implement the flush method without freeing memory or using a distinct thread. Just create a "write on demand" feature.
• [ ] implement the memory release of flushed rows. A rolling list could be a solution here.
• [ ] move the writing in a distinct thread. A very simple Producer-Consumer-Bus design.

[jferard]

• [x] make all styles and configuration accessible to static creation in ods elements. Currently, styles are dynamically added to ods elements when used. It's a good thing, but we should be able to create styles in ods elements before using them.
• [x] implement the flush method without freeing memory or using a distinct thread. Just create a "write on demand" feature.
• [ ] implement the memory release of flushed rows. A rolling list could be a solution here.
• [ ] move the writing in a distinct thread. A very simple Producer-Consumer-Bus design.

The current implementation is very tedious:

• the user has to declare each style
• the user must call each flush method at the right time

The goal is:

1. the user chooses a number of lines in the "buffer";
2. when the lines are filled with data, the flush method is automatically called.
3. if a new style appears after the first flush, an error is sent to log.

[jferard]

This seems to be a too big step for one iteration. Let"s split it in two:

1. First step: create a flushable OdsFileWriter that reproduces the OdsDocument output;
2. Split the job into two threads: the first thread writes objects; the second thread writes a file.

I keep this issue open, but set to milestone 3.1, and open a new issue for the first step.

[jferard]

The current implementation of the flushmethod misses the point: the first thread should handle the whole text/xml production (including escape of xml chars), the second thread the writing of the text/xml into the output stream. The communication using one or several CharBuffer seems a good solution.

Let's split this issue into two new issues:

• a two threads implementation in a test case;
• a two threads implementation with a CharBuffer as communication "channel".

[jferard]

See #69