cpan-testers / cpantesters-backend

Backend data processing for CPAN Testers
Other
0 stars 4 forks source link

Copy "generate" task into new module that uses Beam::Runner #1

Closed preaction closed 7 years ago

preaction commented 7 years ago

In order to start working on improving the main generate task which reads from the Metabase and writes the individual statistics, we should move it into a runnable module using Beam::Wire and Beam::Runner.

The main generate task starts on https://github.com/cpan-testers/cpantesters-backend/blob/master/old/generate/lib/CPAN/Testers/Data/Generator.pm#L179. Only the "generate" task should be moved. Other tasks like rebuild, regenerate, parse, tail, and etc... will be built using other modules (one module per runnable task).

The resulting module should:

In addition to the module, there should be a Beam::Wire configuration file in the file etc/container/backend.yml. This should contain a configuration of an object called parse_reports which is the runnable class.

This is the beginning of a larger refactoring and cleaning up of the backend processes, and will require deployment to be set up for this repository, a cron job to be configured, and a BEAM_PATH environment variable set, which is another ticket.