BuildFlow, is an open source framework for building large scale systems using Python. All you need to do is describe where your input is coming from and where your output should be written, and BuildFlow handles the rest. No configuration outside of the code is required.
Describe the bug
If you use ray metrics inside of a processor the metrics do not get properly exported. I think this is cause ray deregisters metrics when they get garbage collected.
The current workaround is to inject the metric into a replica dependency to avoid it being garbage collected.
Probably the best work around is to make buildflow metrics keep a reference to the metric around, this would allow them to be properly exported.
To Reproduce
Steps to reproduce the behavior:
define a metric inside your processor or as a global metric
increment the metric inside your process
observe it not being reported
Expected behavior
I would expect the metric to be exported
Describe the bug If you use ray metrics inside of a processor the metrics do not get properly exported. I think this is cause ray deregisters metrics when they get garbage collected.
The current workaround is to inject the metric into a replica dependency to avoid it being garbage collected.
Probably the best work around is to make buildflow metrics keep a reference to the metric around, this would allow them to be properly exported.
To Reproduce Steps to reproduce the behavior:
Expected behavior I would expect the metric to be exported