askap-vast / vast-pipeline

This repository holds the code of the Radio Transient detection pipeline for the VAST project.
https://vast-survey.org/vast-pipeline/
MIT License
8 stars 3 forks source link

Add distance-to-edge and distance-to-centre metrics to pipeline #696

Open ddobie opened 1 year ago

ddobie commented 1 year ago

During the 2023 workshop we discovered that there are some edge-of-field effects that can induce false variability. It would therefore be helpful to be able to tag, filter or remove sources that are close to the edge of the field.

ajstewart commented 1 year ago

Just a note that I remember I tried to implement this once and scrapped it because it was quite expensive.

Unfortunately I can't find the code, I thought I committed it at some point.

I think this function in tools was the result of it in the end: https://github.com/askap-vast/vast-tools/blob/3abf03e47a111821094bd4eac54fedd3b89911bd/vasttools/utils.py#L632-L646

I remember the taxicab having something to do with it.

ddobie commented 1 year ago

I did think it would be quite expensive. In that case it's something that we should be doing in the last stage of filtering with vast-tools rather than making it part of the pipeline.

ajstewart commented 1 year ago

I guess distance from centre is easy but perhaps not that useful? Distance from edge is the hard but most useful.

You could let the pipeline calculate the metric on sources only with a certain threshold of variability.

But I reckon you could put this into the image import, you only have to run it once. If it only adds a little bit of processing I'd guess it's worth it.

ddobie commented 1 year ago

Yeah distance from centre isn't useful at all, it really has to be distance from the edge.

I think if we're going to calculate it for a subset then it makes sense to do that post-pipeline (e.g. vast-tools) because otherwise we're almost certainly going to end up with a situation where someone finds a source that they want it for, but it doesn't exist. The code already partially/mostly exists in vast-tools anyway.

This is also a bit less relevant if we're cropping the fields - there's no rubbish on the image edges because that's all been removed anyway.