simonalbergel
commented
1 year ago Hello again, after investigation on my side, remote streaming is simply not implemented for SEVIRI l1b native reader, sorry for mistaking it with a bug.
Here I am sharing the work around I have found working for my usage. It is not equivalent, as with this solution I don't stream the data but store it to a temporary local file util I am done initializing the Scene object. Thank you again for the very useful package.
import os
import tempfile
import shutil
from satpy import Scene
from satpy.readers.utils import generic_open
from satpy.utils import convert_remote_files_to_fsspec
def read_native_remotely(uri_file):
"""Remotely reads native files, by creating a temporary local file to read from.
Temporary file is deleted as soon as it is not needed anymore.
Args:
- uri_file, str
Returns:
- seviri_scene, satpy.scene.Scene
"""
fsfile = convert_remote_files_to_fsspec([uri_file])[0]
with tempfile.TemporaryDirectory() as tmp_dir:
tmp_destination_path = os.path.join(tmp_dir, fsfile._file)
os.makedirs(os.path.dirname(tmp_destination_path), exist_ok=True)
with open(tmp_destination_path, "wb") as tmp_destination:
with generic_open(fsfile, "rb") as remote_source:
shutil.copyfileobj(remote_source, tmp_destination)
seviri_scene = Scene(
reader="seviri_l1b_native",
filenames=[tmp_destination_path],
reader_kwargs={"fill_disk": True},
)
return seviri_scene
Hello, thank for the great work the package is very useful. I am quite new to working with remote storage.
I am currently working with Eumetsat's geostationary imagery, in particular the product : EO_EUM_DAT_MSG_HRSEVIRI.
I have data both in HRIT and native .nat file formats that I read from a remote azure blob storage. Passing in URI for remote reading HRIT files is no issue, but I can't seem to open any .Nat files this way.
I am sure the file does exist :
Downloading and locally read the data works.
Actual results
For HRIT reading (working as expected):
For native reading, not working as intended:
Thank you in advance and sorry if I wasn't clear enough in my question.
Environment Info:
Writers
awips_tiled: ok cf: cannot find module 'satpy.writers.cf_writer' (Ensure that the netCDF4 or h5netcdf package is installed.) geotiff: ok mitiff: ok ninjogeotiff: ok ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff') simple_image: ok
Extras
cartopy: No module named 'cartopy' geoviews: No module named 'geoviews'