Currently, we are updating the data in the Azure blob storage once Signals are triaged. This is the overall output/signals.parquet file, as well as the output/signals.csv file in the dev blob. That is the file that is pipelined into HDX, developed by @isatotun.
However, HDX should reflect alerts as soon as they have been sent out. Rather than the pipeline running on a schedule, we should ideally be able to hook into it and tell it to run the update every time we update Azure.
caldwellst
commented
4 months ago
Currently, we are updating the data in the Azure blob storage once Signals are triaged. This is the overall
output/signals.parquetfile, as well as theoutput/signals.csvfile in thedevblob. That is the file that is pipelined into HDX, developed by @isatotun.However, HDX should reflect alerts as soon as they have been sent out. Rather than the pipeline running on a schedule, we should ideally be able to hook into it and tell it to run the update every time we update Azure.