search

Unidata / netcdf4-python

netcdf4-python: python/numpy interface to the netCDF C library
http://unidata.github.io/netcdf4-python
MIT License
757 stars 264 forks source link

can't run in celery/django #937

Open wsf1990 opened 5 years ago

wsf1990 commented 5 years ago

Hello， It works ok as normal, But when I run the netcdf in celery task/django,It can't work,and show me this error:

File "...", line 53, in ...
    with Dataset(path, format='NETCDF4') as dataset:
  File "netCDF4/_netCDF4.pyx", line 2291, in netCDF4._netCDF4.Dataset.__init__
  File "netCDF4/_netCDF4.pyx", line 1855, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -101] NetCDF: HDF error: b'/my/nc/path'

What happened when I use celery/django?

wsf1990 commented 5 years ago

I checked the source code, May be the question is at the 2232 line in file netCDF4/_netCDF4.pyx: https://github.com/Unidata/netcdf4-python/blob/master/netCDF4/_netCDF4.pyx#L2232

ierr = nc_open(path, NC_NOWRITE, &grpid)

But I don't know what's wrong with this code.

jswhit commented 5 years ago

Is '/my/nc/path' a real file?

wsf1990 commented 5 years ago

@jswhit Yes, and it can be opened in shell mode

wsf1990 commented 5 years ago

And I use ubyte.nc which is your test file,It works well.

jswhit commented 5 years ago

A permissions issues then?

wsf1990 commented 5 years ago

Maybe no, the file in same position.Can I transfor my file to you?

wsf1990 commented 5 years ago

test.zip

wsf1990 commented 5 years ago

image works well in shell mode. image Can't work in django.

jswhit commented 5 years ago

Looks like the traceback is coming from MFDataset, which won't work with this file (it can only open NETCDF3 or NETCDF4_CLASSIC formatted files).

wsf1990 commented 5 years ago

@jswhit I had checked both Dataset and MFDataset,It's all show the same error.

jswhit commented 5 years ago

haven't ever used django or celery myself, so I'm afraid I won't be of much help. Somehow the environment in which your are running the script is different than when you run in the shell - perhaps you're picking up a different version of the python module of C libs, or the user that the script is running under doesn't have permission to read the file?

jswhit commented 5 years ago

It must be possible, since this project uses netcdf4-python and django:

https://ncdjango.readthedocs.io/en/latest/

wsf1990 commented 5 years ago

Thanks, I'll have a try.

wsf1990 commented 5 years ago

I'd try set the dir's permission to 777: chmod -R 777 /my/dir/path. And I found that the 'ubyte.nc' and some other's which you provide can work well,but other's can't.

MiCurry commented 5 years ago

This project uses both Django, Celery and NetCDF4 Python: https://github.com/seacast/SharkEyes.

You are most likely not handling your environment correctly.