At the moment, business logic is implemented by both ILogFile implementations as well as by SingleIDataSource. This worked well for now that we only had text logs, however this is starting to break down with the introduction of other formats such as json, etc...
Instead of introducing many more IDataSource implementations for other log files, we'll instead move part of this functionality to a new ILogFile implementation with the following responsibilities instead:
Detect format and encoding of a file
Create a StreamingTextLogFile (using the encoding detected in step 1) which does nothing more than:
Offer the contents of a text file as log entries (Columns: Line Number, Raw Content)
Stream log entries into memory on-demand, keeping a fixed number of entries in memory
Does so by building an index of byte offset <==> line index
Offers no more interpretation of the data in any way
Create an ILogFile through a plugin based on the format determined in Step 1
Any such implementation is an aggregating log file which is fed a stream of log entries (from the object in Step 2) and spits out another set of log entries with potentially more columns as determined by the format
By default, an GenericTextLogFile will be created which:
Finds log level, timestamp in the format
Create a MultiLineLogFile , if the log file from Step 3 requires it
Introduce a new property to allow the log file from Step 3 require multiline parsing
Introduce a new property to allow users to enable/disable multiline parsing (some formats might only be single-line, others might always be multi-line, some might be mixed)
Extension points
Implement plugins for more text formats which have nothing more to do than to transform log entries
GenericJsonLog (Perform similar heuristics to GenericTextLog: which column is the timestamp, what's its format, etc...)
log4cpp / log4net (similar to serilog)
Various web server formats
Allow for opening binary formats by checking if the format from Step 1 is binary, skipping step 2 and then using a slightly different plugin to create the ILogFile for that binary format (there's no point in having Tailviewer stream the binary contents to the plugin)
Any expected problems concerning backwards compatibility of existing plugins?
Yes/No
Any expected problems concerning backwards compatibility of existing user settings?
First batch of work for #284
Architectural Changes
At the moment, business logic is implemented by both
ILogFileimplementations as well as bySingleIDataSource. This worked well for now that we only had text logs, however this is starting to break down with the introduction of other formats such as json, etc...Instead of introducing many more
IDataSourceimplementations for other log files, we'll instead move part of this functionality to a newILogFileimplementation with the following responsibilities instead:StreamingTextLogFile(using the encoding detected in step 1) which does nothing more than:ILogFilethrough a plugin based on the format determined in Step 1GenericTextLogFilewill be created which:MultiLineLogFile, if the log file from Step 3 requires itExtension points
Any expected problems concerning backwards compatibility of existing plugins?
Yes/No
Any expected problems concerning backwards compatibility of existing user settings?
Yes/No
Does this break existing user workflows?
Yes/No