tajmone / PBCodeArcProto

PB CodeArchiv Rebirth Indexer Prototype
4 stars 0 forks source link

What are the differences between mod_Errors and mod_Logger? #19

Open SicroAtGit opened 5 years ago

SicroAtGit commented 5 years ago

I still don't know what the difference is between mod_Errors and mod_Logger.

Is mod_Errors only for critical errors or should the module at the end be the logger module?

I created the mod_Logger module to handle all process status messages (errors, warnings, informations, debugs).

Two logger modules can cause confusion. We should merge them, if both have the same task.

tajmone commented 5 years ago

Ciao @SicroAtGit !

No, they have different purposes (and this is one of the difficulties that have slowed me down in the modularization of the code).

The Errors module serves to keep track of errors during execution of loops or other kind of iterations, in order to track the various errors types required to evaluate if the process failed or not. For example, the Archiv module scans the whole project and tracks any errors encountered, and once the whole project has been scanned it can decide if the project is in good status, and if the HTML pages can be updated or not.

Similar loops, before exiting they evaluate all the errors and create a report to send to the logger, but in some cases they might also log errors as they encounter them. The Archiv scanner does that, depending on the debug level it might or might not log in real time every error encountered, but it will always store a status report (in itself, as a string) after scanning is completed.

It's up to the tool to decide if the real-time error reports should be discarded or logged (via the Logger module settings), and if to "read" the final report or not.

The difficult part here is to keep every module both self-contained and interacting with the others. I think that we need to think of a way that each module can "register itself" with the logger according to the setting of the tool using the module — example, a CLI tool might wish the Archiv parser to log to STDERR and STDOUT all the scanning steps and their errors, while a GUI tool might instead want to show only the final report.

These are the details in which I got entangled.

But hopefully now I'll be back on track on the project, and start to dedicate full afternoons to it, and try to break down these problems into simpler and smaller problem that can be managed.

tajmone commented 5 years ago

Also, the goal is to keep separate function in different modules, which should simplify independent development of the modules, since each module would only have to know about the exposed API of the others. Keep in mind that most of the current modules are most likely going to be required by any tool working on the Archiv; so, the sole purpose of modularization is splitting the old page builder's code into independent modules in view of different usage scenarios by different tools.

Ideally, the Errors Module should handle all errors that are cross-module, including those of the logger, so that any module can access that information. It will also contain all the constants definitions of the types of errors.

The problem here is that I first started with a single standalone application to build the HTML website, and when it was almost ready started to split it into module. In the original design I didn't really consider modules at all, so I basically ended up realizing that instead of just splitting the code into multiple include files there is a need to organize how these modules should communicate with each other via the Global module. This is a key factor if we want to be able to create different tools that reuse the same modules, and the time invested now is going to save us time later on.

The cache problem has been a bit of a thorn in the side in this respect, because it plays a major role in Resources parsing module, so I took some time to experiment with caching — the solution is actually rather simple, the tricky part was to find a cross platform way to check if folders have changed, and the only way is to check the last changed attribute of every folder in the tree. The resources only need to be fingerprinted with one of the hashing functions of the Cypher library. I've tested it locally and works fine.

I should just take out of the shelves some of my old books on design patterns and look at examples of how to implement a consistent way for modules to interact with each other in a flexible manner that makes their use smooth and easy. The overall approach to including into any tool both the mandatory and optional modules should be intuitive and allow the end user to register and customize functionality like logging, interfacing to a GUI, etc., and a standardized system would be ideal.

Because of this, the Global module should take care of all the modules initialization to ensure that custom settings don't conflict, and that the correct order of initialization is taken care of (which right now is not a clear issue, since many functionalities are still in draft stage).