search

askap-vast / vast-pipeline

This repository holds the code of the Radio Transient detection pipeline for the VAST project.
https://vast-survey.org/vast-pipeline/
MIT License
8 stars 3 forks source link

Add XML to txt conversion to selavy ingest #564

Closed joshoewahp closed 3 years ago

joshoewahp commented 3 years ago

Selavy produces a VOTable XML version of selavy catalogues by default, while the pipeline currently depends on fixed width txt copies.

A simple fix is to add this adapter, which results in an identical DataFrame to pd.read_fwf with the txt files. 

from astropy.io.votable import parse
import pandas as pd

def votable_to_pandas(votable_file):
    votable = parse(votable_file)
    table = votable.get_first_table().to_table(use_names_over_ids=True)
    return table.to_pandas()
marxide commented 3 years ago

(copying from the vast-tools issue https://github.com/askap-vast/vast-tools/issues/280)

Can be simplified by using the unified table readers, e.g.

from astropy.table import Table
df = Table.read(votable_file, format="votable", use_names_over_ids=True).to_pandas()
marxide commented 3 years ago

Note to self: may as well add CSV read support since CASDA serves those too.