search

ska-sa / MeerKAT-Cookbook

GNU General Public License v3.0
12 stars 13 forks source link

download #33

Closed LKHZ closed 1 year ago

LKHZ commented 1 year ago

I have a question to inquire about. I want to download data, but I don't know how to download it. Can you help me? image

spassmoor commented 1 year ago

The telescope help pages provide a fair amount of information on data access formats what it is in and the page Accessing and examining your data will guide you through the options for data access.

LKHZ commented 1 year ago

image The second image is what I want to download, but I submitted the application two weeks ago, but I haven't received the download link yet. What is the problem causing it? Looking forward to your reply!! image

LKHZ commented 1 year ago

Isn't this file 12.31T? I clicked on the 32KW button and downloaded an. rdb file, but it only has one more GB. How do I operate this rdb file? image

bennahugo commented 1 year ago

Hi

The RDB object is just a handle in essence with some telescope state and a link to the underlying data buckets.

You can download the data with the mvftoms.py script that is installed when you pip install katdal (ie. pip install katdal[ms,s3]).

The available selection options are available under --help

Here is a basic worked example: mvftoms.py --flags cam,data_lost -f -o epoch2.ms

Note: Please log a ticket through our help desk so that operations can assist you.

Note: you may need casacore-data on your system for full ms support (available through KERNsuite). Please consult the CASA help pages on how to setup your .casarc and NRAO ephemeris correctly, manually if you are not running an Ubunu derivative.

On Fri, Jul 28, 2023 at 4:31 AM LKHZ @.***> wrote:

Isn't this file 12.31T? I clicked on the 32KW button and downloaded an. rdb file, but it only has one more GB. How do I operate this rdb file? [image: image] https://user-images.githubusercontent.com/70903631/256707497-4716e527-985c-4bfb-9595-b27d480bef3e.png

— Reply to this email directly, view it on GitHub https://github.com/ska-sa/MeerKAT-Cookbook/issues/33#issuecomment-1654886015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4RE6VR4HENEJP7RO7336TXSMQAXANCNFSM6AAAAAA22E6SXI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

--

Benjamin Hugo

LKHZ commented 1 year ago

Is the data part included in the download through mvftom? I displayed an error when downloading, 'Katdal. chunkstore. StoreUnavailable: Chunk' 1617809470 sdp-l0/correlator Data/0001 16896 00000 ': HTTPSConnectionPool (host='archive gw-1. kat. ac. za', port=443): Max retries exceeded with URL:/1617809470-sdp-l0/correlator Data/0001 16896 00000. npy (Caused by ReadTimeoutError ("HTTPSConnectionPool (host='archive gw-1. kat. ac. za ', port=443): Read timed out. (read timeout=30)")

bennahugo commented 1 year ago

Possibly a network related error. Can you open a ticket on the helpdesk - don't think it is a bug on the cookbook. Thanks! :)

LKHZ commented 1 year ago

I can only open an interface like the one above. I don't know if the ticket you mentioned is like this。 Thanks!!! image

LKHZ commented 1 year ago

Hello, I have successfully downloaded the files now, but I don't know how to convert them into fits files. Do you have any good ideas?

Thank you very much. Looking forward to your reply image

bennahugo commented 1 year ago

@LKHZ As per helpdesk - please raise specific queries via the helpdesk, not tickets here. Issues are reserved for bugs. Copied is some help:

The generation of a fits image from radio interferometer data in a measurement set file (The .ms directory with all the tables) is an involved process.

If you do not have a collection of preferred software for your workflow, we recommend starting with CASA , it has all the utilities to produce an image from radio data. There is a walkthrough with a dataset (few mb) in CASA for a user to explore:

https://drive.google.com/drive/folders/1UYiwW0ftR-ccCRd1BR2Jz2GLMJeL4RiH?usp=sharing

There is also useful material available from:

https://www.sarao.ac.za/e-learning-portal/

In the course https://www.sarao.ac.za/courses/fundamentals-of-radio-interferometry/ there is an introduction to CASA practical session and in the https://www.sarao.ac.za/courses/african-radio-interferometry-winter-school/ , there are lectures that cover many of the details of calibrating and producing an image.

More information on the TART training telescopes deployed at Rhodes University [ZA] and New Zealand can be located here: https://tart.elec.ac.nz/#about

Good luck with your data processing.