search

MITgcm / xmitgcm

Read MITgcm mds binary files into xarray
http://xmitgcm.readthedocs.io
MIT License
56 stars 65 forks source link

llcreader ecco portal no mask #190

Open apatlpo opened 4 years ago

apatlpo commented 4 years ago

I downloaded data (wind stress llc4320) from the ECCO data portal and would like to read it with llcreader as advertised in this post.

My understanding is that I need to provide mask information in order to do that. Could you please confirm?

If this is the case, I have the feeling mask information is not present on the ECCO portal for llc4320.

apatlpo commented 4 years ago

pinging again about this ...

apatlpo commented 4 years ago

My understanding is that I need to provide mask information in order to do that. Could you please confirm?

Answer is yes, you need to have a consolidated zarr archive containing mask_c, mask_w, mask_s variables

apatlpo commented 4 years ago

I am still not closing this issue because of the missing mask issue

rabernat commented 4 years ago

I am looking at this today. Sorry for the slow response.

rabernat commented 4 years ago

p.s. Would be helpful if you could provide more detail about what you have tried, i.e. specific code, and any error messages that came out.

apatlpo commented 4 years ago

you're right, sorry for not providing any code. I'll try to do that soon.

If you have time to spend on these issues however, I'd rather you take a look at the other issue I posted yesterday (191), which is much more worrying.

apatlpo commented 4 years ago

@menemenlis pushed hFac files back on the ECCO Data portal.

Before closing this issue, I'd be tempted to propose adding one or two sentences to llcreader "Manual Dataset Creation" doc referring to the necessity to generate (consolidated) a zarr mask archive. Do people think this may be useful?

rabernat commented 4 years ago

I'd be tempted to propose adding one or two sentences to llcreader "Manual Dataset Creation" doc referring to the necessity to generate (consolidated) a zarr mask archive. Do people think this may be useful?

Very useful! Would you be able to make a PR?

One complication is that the masks themselves are stored in a highly compressed format. They are not the raw hFacC.data files, which are extremely inefficient in terms of disk space usage. You can find them here: https://console.cloud.google.com/storage/browser/pangeo-ecco/llc/masks/

It might be wise to put a copy of these in the ecco data portal as well.

apatlpo commented 4 years ago

Ok, I can suggest this to Dimitris but I did not figure out how to download the mask from google cloud storage.

Do you think Dimitris will know how to do this?

If not could you please specify how one can do that?

rabernat commented 4 years ago

Ok, I can suggest this to Dimitris but I did not figure out how to download the mask from google cloud storage.

Do you think Dimitris will know how to do this?

I made this mask, not Dimitris.

If not could you please specify how one can do that?

gsutil -m cp -r gs://pangeo-ecco/llc/masks/llc_4320_masks.zarr /local/path