ncdump v4.6.3 NetCDF: Unknown file format

[abhishekBansal]

ncdump version: 4.6.3 MacOS 10.14.5, ncdump installed via homebrew

This might be a noob question, I am just starting with understanding of netcdf files and format. I am unable to open files that I am downloading from here https://tgftp.nws.noaa.gov/SL.us008001/DF.of/DC.radar/DS.p94r1/SI.kpoe/

These files open in Panoply fine. I keep getting this error sn.last: NetCDF: Unknown file format Am I doing something wrong?

[WardF]

These files don't appear to be netCDF files. Do you know how they were generated?

[abhishekBansal]

I am not sure, its govt. database I assumed it must be netCDF. Here is panoply screenshot which reads its as netCDF, its also able to export same file as CDL.

[abhishekBansal]

okay so seemed like I had to convert it to netCDF first https://www1.ncdc.noaa.gov/pub/data/radar/Radar-Decoding-EasySolution.txt

After this conversion ncdump started working for me, thanks!

[dopplershift]

For the record, those files are NEXRAD Level 3 product files, also known as NIDS files.

[DennisHeimbigner]

If the conversion to netcdf is straightforward, would it be useful to have the netcdf-c library directly read such files? =Dennis

On 9/10/2019 1:11 PM, Ryan May wrote:

For the record, those files are NEXRAD Level 3 product files, also known as NIDS files.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Unidata/netcdf-c/issues/1485?email_source=notifications&email_token=AAG47W5U4YINAFAKRGU4BXLQI7WOHA5CNFSM4IUT65D2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6MF2GY#issuecomment-530079003, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG47WYPC4TNRT24ZUKNQRTQI7WOHANCNFSM4IUT65DQ.

[dopplershift]

NIDS can describe many different product types (lines, points, polar grids, rectangular grids), so no the conversion to netcdf is not straightforward, though netcdf-java does its best.

In terms of supporting conversions from other file formats, that's duplicating the IOSP layer from netcdf-java. IMO, that's a topic that would go well-beyond the issue tracker.

[edwardhartnett]

Well we have duplicated the IOSP layer from netcdf-java, and its called user-defined formats. ;-)

[dopplershift]

That's so cute. 😉

Seriously, while I recognize a significant amount of effort in developing a useful C abstraction layer and interface, there is so much more work in:

1. Rewriting a bunch of binary I/O in C
2. Porting netcdf-java's netcdf schema for all those files into the C-API calls

I'm not saying it's a terrible idea, or one that's technically hard. But it's a massive undertaking that needs to be weighed against other on-going tasks, like Zarr and FORTRAN/C++ library updates, not to mention just basic C library maintenance.

[DennisHeimbigner]

NIDS is only worth doing if it is low-hanging fruit; but I gather that is not the case, so never mind.

[WardF]

Resolved so closing, but only for housekeeping, not to shut down the conversation. :)

[edhartnett]

@dopplershift climate science/meteorology have always pressed hardware and software to their limits.

I/O has become the bottleneck for both model development and performance. This situation is expected to continue as Earth observation assets achieve greater diversity and data rates, models greater resolution, and HPC systems more processors.

There are no scientific problem of greater importance! We need bold actions, daring design, and days, weeks, months, and years of heads-down coding. That is my usual prescription - also my usual routine. ;-)

The point of the user-defined formats is that other programmers can make these contributions. No longer are these efforts restricted to a handful of past and present Unidata programmers. Now, just as with Java IOSPs, the wider community can contribute.

Glory awaits the netCDF C programmer who writes a GRIB reader. And there are so many other possibilities!

I hope that others take up the banner of netCDF progres. There is so much to be done...

[gsjaardema]

@edhartnett I think you might be ready for a long vacation... ;-)

But I agree with your message.

[dopplershift]

@edhartnett I agree with the spirit of what you're saying. To be clear, I'm not saying they wouldn't be useful solutions. I'm just speaking to what Unidata should be prioritizing at the moment.

[edwardhartnett]

@dopplershift fortunately I'm best in a position to determine my own priorities. ;-)

@gsjaardema I'll be back in November, raring to go. Be ready!