Currently, #294 will be providing progress of block processing via intermittently writing the block number processed to a logfile. When a cluster job is launched, the user will be given a tail command pointing to the logfile, as a means to monitor the pipeline progress.
However, this is not the most modern way/solution to provide job progress, and perhaps can be improved upon.
I'm not at all familiar with messaging libraries like ZeroMQ/ActiveMQ/etc, but with the very vague notion in mind of having:
some kind of "channel" through which httomo can send "messages" regarding block/pipeline processing progress
any client then being able to "subscribe" to such messages
maybe messaging libraries are something that can provide this (if such a thing would make sense for providing job progress, maybe it's overkill/unnecessary/not ideal)?
Currently, #294 will be providing progress of block processing via intermittently writing the block number processed to a logfile. When a cluster job is launched, the user will be given a
tailcommand pointing to the logfile, as a means to monitor the pipeline progress.However, this is not the most modern way/solution to provide job progress, and perhaps can be improved upon.
I'm not at all familiar with messaging libraries like ZeroMQ/ActiveMQ/etc, but with the very vague notion in mind of having:
maybe messaging libraries are something that can provide this (if such a thing would make sense for providing job progress, maybe it's overkill/unnecessary/not ideal)?