Open tbartholomaus opened 1 year ago
I am out for the evening but will look at this tomorrow- remind me if you don’t hear by tomorrow pm. IanIan JoughinPolar Science CenterApplied Physics LabU Washington206-221-3177On Nov 28, 2022, at 5:22 PM, Timothy Bartholomaus @.> wrote: Hi Ian, I'm somewhat loath to submit another issue here after my first, but now that I've got everything up and running on a machine at home (macbook pro 2016 intel chip) and my linux machine at work (CentOS 7), I really think this might be an actual bug. I've installed the same greenlandMapping environment on my 2021 M1 Pro macbook pro, and proceeded without issue. At a python console, I have no problem importing grimpfunc: (greenlandMapping) @. ~ % python Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:55:37) [Clang 14.0.6 ] on darwin Type "help", "copyright", "credits" or "license" for more information.
import grimpfunc
AttributeError Traceback (most recent call last) Cell In [1], line 1 ----> 1 import grimpfunc
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/init.py:4 1 all = ['boxPicker', 'cmrUrls', 'Flowlines', 'get_urls', 'GrIMPSubsetter', 2 'NASALogin', 'pointInspector'] ----> 4 from grimpfunc.boxPicker import boxPicker 5 from grimpfunc.cmrUrls import cmrUrls 6 from grimpfunc.cmr import get_urls
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/boxPicker.py:11 9 import panel as pn 10 import holoviews as hv ---> 11 import hvplot.xarray 12 import grimpfunc as grimp 13 import rioxarray
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:61 57 xr.register_dataarray_accessor(interactive)(XArrayInteractive) 59 post_patch(extension, logo) ---> 61 patch()
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:59, in patch(name, interactive, extension, logo) 56 xr.register_dataset_accessor(interactive)(XArrayInteractive) 57 xr.register_dataarray_accessor(interactive)(XArrayInteractive) ---> 59 post_patch(extension, logo)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/init.py:166, in post_patch(extension, logo) 164 def post_patch(extension='bokeh', logo=False): 165 if extension and not getattr(_hv.extension, '_loaded', False): --> 166 hvplot_extension(extension, logo=logo)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/pyviz_comms/init.py:64, in extension.new(cls, *args, *kwargs) 62 except Exception: 63 pass ---> 64 return param.ParameterizedFunction.new(cls, args, **kwargs)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/param/parameterized.py:3654, in ParameterizedFunction.new(class, *args, **params) 3652 inst = class.instance() 3653 inst.param._setname(class.name) -> 3654 return inst.call(*args,**params)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/utilities.py:45, in hvplot_extension.call(self, *args, params) 40 def call(self, *args, *params): 41 # importing e.g. hvplot.pandas always loads the bokeh extension. 42 # so hvplot.extension('matplotlib', compatibility='bokeh') doesn't 43 # require the user or the code to explicitely load bokeh. 44 compatibility = params.pop('compatibility', None) ---> 45 super().call(args, params) 46 backend = _hv.Store.current_backend 47 if compatibility in ['matplotlib', 'plotly'] and backend != compatibility:
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/holoviews/ipython/init.py:149, in notebook_extension.call(self, *args, **params) 147 completer.completions_sorting_key = self.completions_sorting_key 148 if not p.allow_jedi_completion and hasattr(IPCompleter, 'use_jedi'): --> 149 ip.run_line_magic('config', 'IPCompleter.use_jedi = False') 151 resources = self._get_resources(args, params) 153 Store.display_formats = p.display_formats
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/interactiveshell.py:2364, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth) 2362 kwargs['local_ns'] = self.get_local_scope(stack_depth) 2363 with self.builtin_trap: -> 2364 result = fn(*args, **kwargs) 2365 return result
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:163, in ConfigMagics.config(self, s) 159 from traitlets.config.loader import Config 160 # some IPython objects are Configurable, but do not yet have 161 # any configurable traits. Exclude them from the effects of 162 # this magic, as their presence is just noise: --> 163 configurables = sorted(set([ c for c in self.shell.configurables 164 if c.class.class_traits(config=True) 165 ]), key=lambda x: x.class.name) 166 classnames = [ c.class.name for c in configurables ] 168 line = s.strip()
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:164, in
AttributeError: type object 'CommManager' has no attribute 'class_traits'
As I mentioned at the top, I'm up and running on other systems, so this isn't urgent, but I thought I'd mention as a potential issue in case this comes up for anyone else, or if this is actually easily addressed or meaningful to you. Without debugging the call to import hvplot.xarray, specifically within a jupyter lab kernel, I'm not sure this is something I can get to the bottom of without a lot of effort. Thanks again.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>
Tim,
Hvplot was recently upgraded, so I expect that’s where the problem was. On my M1 Mac, it worked fine, but it was one version behind. I was able to reproduce the problem on a 2016 intel Mac Pro with a fresh environment.
I upgraded the environment to Python 10 and unpinned a couple of other programs, which all now seem to work. I have tested the new environment on both intel and M1 Macs and it seems to work fine (waiting for binder to load to finish that test).
BTW, in rebuilding the environments I ran the ipykernel command from the base env. a couple of times and it didn’t take, so my guess is you did run the command earlier but in the wrong environment.
So pull the new environment.yml file and conda env remove -n greenlandMapping then repeat the create procedure with the the new environment file.
It would be great if you could also do this on your linux machine too (an extra test of everything wouldn’t hurt).
Please let me know if you have any more problems.
Ian
Ian Joughin Polar Science Center Applied Physics Lab University of Washington 1013 NE 40th Street Seattle, WA 98105-6698
Data products: https://nsidc.org/grimp
Data Tools: https://github.com/fastice/GrIMPTools
Phone: 206-221-3177 Fax: 206-616-3142
On Nov 28, 2022, at 5:22 PM, Timothy Bartholomaus @.***> wrote:
Hi Ian,
I'm somewhat loath to submit another issue here after my first, but now that I've got everything up and running on a machine at home (macbook pro 2016 intel chip) and my linux machine at work (CentOS 7), I really think this might be an actual bug.
I've installed the same greenlandMapping environment on my 2021 M1 Pro macbook pro, and proceeded without issue. At a python console, I have no problem importing grimpfunc:
(greenlandMapping) @.*** ~ % python Python 3.9.15 | packaged by conda-forge | (main, Nov 22 2022, 08:55:37) [Clang 14.0.6 ] on darwin Type "help", "copyright", "credits" or "license" for more information.
import grimpfunc
However, when I do the same within jupyter lab, I get a fairly long set of errors, despite having run python -m ipykernel install --user --name=greenlandMapping several times, both with and without the greenlandMapping environment activated.
Within jupyter lab, with the greenlandMapping environment active and sitting idle (I can, for example, successfully import dask, geopandas, rioxarray, etc.), when I try and import grimpfunc I get an error with the boxPicker submodule that stems from the internal call to import hvplot.xarray
I'm copying this error below, and also attaching two screenshots, if they offer any more context.
AttributeError Traceback (most recent call last) Cell In [1], line 1 ----> 1 import grimpfunc
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/init.py:4 1 all = ['boxPicker', 'cmrUrls', 'Flowlines', 'get_urls', 'GrIMPSubsetter', 2 'NASALogin', 'pointInspector'] ----> 4 from grimpfunc.boxPicker import boxPicker 5 from grimpfunc.cmrUrls import cmrUrls 6 from grimpfunc.cmr import get_urls
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/grimpfunc/boxPicker.py:11 9 import panel as pn 10 import holoviews as hv ---> 11 import hvplot.xarray 12 import grimpfunc as grimp 13 import rioxarray
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:61 57 xr.register_dataarray_accessor(interactive)(XArrayInteractive) 59 post_patch(extension, logo) ---> 61 patch()
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/xarray.py:59, in patch(name, interactive, extension, logo) 56 xr.register_dataset_accessor(interactive)(XArrayInteractive) 57 xr.register_dataarray_accessor(interactive)(XArrayInteractive) ---> 59 post_patch(extension, logo)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/init.py:166, in post_patch(extension, logo) 164 def post_patch(extension='bokeh', logo=False): 165 if extension and not getattr(_hv.extension, '_loaded', False): --> 166 hvplot_extension(extension, logo=logo)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/pyviz_comms/init.py:64, in extension.new(cls, *args, *kwargs) 62 except Exception: 63 pass ---> 64 return param.ParameterizedFunction.new(cls, args, **kwargs)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/param/parameterized.py:3654, in ParameterizedFunction.new(class, *args, **params) 3652 inst = class.instance() 3653 inst.param._setname(class.name) -> 3654 return inst.call(*args,**params)
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/hvplot/utilities.py:45, in hvplot_extension.call(self, *args, params) 40 def call(self, *args, *params): 41 # importing e.g. hvplot.pandas always loads the bokeh extension. 42 # so hvplot.extension('matplotlib', compatibility='bokeh') doesn't 43 # require the user or the code to explicitely load bokeh. 44 compatibility = params.pop('compatibility', None) ---> 45 super().call(args, params) 46 backend = _hv.Store.current_backend 47 if compatibility in ['matplotlib', 'plotly'] and backend != compatibility:
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/holoviews/ipython/init.py:149, in notebook_extension.call(self, *args, **params) 147 completer.completions_sorting_key = self.completions_sorting_key 148 if not p.allow_jedi_completion and hasattr(IPCompleter, 'use_jedi'): --> 149 ip.run_line_magic('config', 'IPCompleter.use_jedi = False') 151 resources = self._get_resources(args, params) 153 Store.display_formats = p.display_formats
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/interactiveshell.py:2364, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth) 2362 kwargs['local_ns'] = self.get_local_scope(stack_depth) 2363 with self.builtin_trap: -> 2364 result = fn(*args, **kwargs) 2365 return result
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:163, in ConfigMagics.config(self, s) 159 from traitlets.config.loader import Config 160 # some IPython objects are Configurable, but do not yet have 161 # any configurable traits. Exclude them from the effects of 162 # this magic, as their presence is just noise: --> 163 configurables = sorted(set([ c for c in self.shell.configurables 164 if c.class.class_traits(config=True) 165 ]), key=lambda x: x.class.name) 166 classnames = [ c.class.name for c in configurables ] 168 line = s.strip()
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.9/site-packages/IPython/core/magics/config.py:164, in
(.0) 159 from traitlets.config.loader import Config 160 # some IPython objects are Configurable, but do not yet have 161 # any configurable traits. Exclude them from the effects of 162 # this magic, as their presence is just noise: 163 configurables = sorted(set([ c for c in self.shell.configurables --> 164 if c.class.class_traits(config=True) 165 ]), key=lambda x: x.class.name) 166 classnames = [ c.class.name for c in configurables ] 168 line = s.strip() AttributeError: type object 'CommManager' has no attribute 'class_traits' As I mentioned at the top, I'm up and running on other systems, so this isn't urgent, but I thought I'd mention as a potential issue in case this comes up for anyone else, or if this is actually easily addressed or meaningful to you. Without debugging the call to import hvplot.xarray, specifically within a jupyter lab kernel, I'm not sure this is something I can get to the bottom of without a lot of effort.
Thanks again.
https://urldefense.com/v3/__https://user-images.githubusercontent.com/24902730/204414064-a9a3d02d-c251-4dea-b2f5-51331afa2815.png__;!!K-Hz7m0Vt54!lt2OHJIhHErILy-qp8uyDVopdkGR_HBVh7Rp6sH4CZad9HXJjOw81huWzDlywKNGdnKJRz5UzYydAiiHe7oF$ https://urldefense.com/v3/__https://user-images.githubusercontent.com/24902730/204414792-b762ca04-1a7c-4998-b3a0-9736fe2b5fde.png__;!!K-Hz7m0Vt54!lt2OHJIhHErILy-qp8uyDVopdkGR_HBVh7Rp6sH4CZad9HXJjOw81huWzDlywKNGdnKJRz5UzYydAiPjNFIx$ — Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/fastice/GrIMPTools/issues/1__;!!K-Hz7m0Vt54!lt2OHJIhHErILy-qp8uyDVopdkGR_HBVh7Rp6sH4CZad9HXJjOw81huWzDlywKNGdnKJRz5UzYydApqa6NvF$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ALICGTRIDRQIG53LHSJYD6LWKVLGZANCNFSM6AAAAAASN53B2A__;!!K-Hz7m0Vt54!lt2OHJIhHErILy-qp8uyDVopdkGR_HBVh7Rp6sH4CZad9HXJjOw81huWzDlywKNGdnKJRz5UzYydAjLypoIa$. You are receiving this because you are subscribed to this thread.
Thanks for jumping right back into this, Ian. Importing grimpfunc in both M1 and linux machines, via jupyter lab all works great. But it looks like unpinning some of those packages might have led to other problems for the mac- looks like a gdal problem.
In a cell from your workbook that had previously worked fine
myTerminusUrls = grimp.cmrUrls(mode='terminus') # mode image restricts search to the image products
myTerminusUrls.initialSearch();
myTermini = {}
for url in myTerminusUrls.getURLS():
year = os.path.basename(url).split('_')[1] # Extract year from name
myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}') # Add terminus to data frame
print(f'/vsicurl/&url={url}')
I now get the following error on the gpd.read_file
from url. But! only on mac. On my linux this runs just fine.
Here's the complete error message:
---------------------------------------------------------------------------
CPLE_OpenFailedError Traceback (most recent call last)
File fiona/_shim.pyx:83, in fiona._shim.gdal_open_vector()
File fiona/_err.pyx:291, in fiona._err.exc_wrap_pointer()
CPLE_OpenFailedError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
DriverError Traceback (most recent call last)
Cell In [18], line 4
2 for url in myTerminusUrls.getURLS():
3 year = os.path.basename(url).split('_')[1] # Extract year from name
----> 4 myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}') # Add terminus to data frame
5 print(f'/vsicurl/&url={url}')
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:259, in _read_file(filename, bbox, mask, rows, engine, **kwargs)
256 path_or_bytes = filename
258 if engine == "fiona":
--> 259 return _read_file_fiona(
260 path_or_bytes, from_bytes, bbox=bbox, mask=mask, rows=rows, **kwargs
261 )
262 elif engine == "pyogrio":
263 return _read_file_pyogrio(
264 path_or_bytes, bbox=bbox, mask=mask, rows=rows, **kwargs
265 )
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:303, in _read_file_fiona(path_or_bytes, from_bytes, bbox, mask, rows, where, **kwargs)
300 reader = fiona.open
302 with fiona_env():
--> 303 with reader(path_or_bytes, **kwargs) as features:
304 crs = features.crs_wkt
305 # attempt to get EPSG code
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/env.py:408, in ensure_env_with_credentials.<locals>.wrapper(*args, **kwargs)
405 @wraps(f)
406 def wrapper(*args, **kwargs):
407 if local._env:
--> 408 return f(*args, **kwargs)
409 else:
410 if isinstance(args[0], str):
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/__init__.py:264, in open(fp, mode, driver, schema, crs, encoding, layer, vfs, enabled_drivers, crs_wkt, **kwargs)
261 path = parse_path(fp)
263 if mode in ('a', 'r'):
--> 264 c = Collection(path, mode, driver=driver, encoding=encoding,
265 layer=layer, enabled_drivers=enabled_drivers, **kwargs)
266 elif mode == 'w':
267 if schema:
268 # Make an ordered dict of schema properties.
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/collection.py:162, in Collection.__init__(self, path, mode, driver, schema, crs, encoding, layer, vsi, archive, enabled_drivers, crs_wkt, ignore_fields, ignore_geometry, **kwargs)
160 if self.mode == 'r':
161 self.session = Session()
--> 162 self.session.start(self, **kwargs)
163 elif self.mode in ('a', 'w'):
164 self.session = WritingSession()
File fiona/ogrext.pyx:540, in fiona.ogrext.Session.start()
File fiona/_shim.pyx:90, in fiona._shim.gdal_open_vector()
DriverError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp' not recognized as a supported file format.
Again, this is only a problem on mac (2021 M1 MBP), not on linux, and it looks like, from a comment in the former version of environment.yml
that this might be a known issue?
I'm copying below a section of conda list
from "identical" environments on both machines.
Mac (2021 M1) | Linux |
---|---|
``` fiona 1.8.22 py310h5a14c0a_0 conda-forge flit-core 3.8.0 pyhd8ed1ab_0 conda-forge folium 0.13.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 h5bb23bf_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 py310h90acd4f_1 conda-forge freetype 2.12.1 h3f81eb7_1 conda-forge freexl 1.0.6 hb7f2c08_1 conda-forge frozenlist 1.3.3 py310h90acd4f_0 conda-forge fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge future 0.18.2 pyhd8ed1ab_6 conda-forge gdal 3.5.2 py310he22d2f2_3 conda-forge geopandas 0.12.1 pyhd8ed1ab_1 conda-forge geopandas-base 0.12.1 pyha770c72_1 conda-forge geos 3.11.0 hb486fe8_0 conda-forge geotiff 1.7.1 ha1a2aeb_3 conda-forge geoviews 1.9.5 pyhd8ed1ab_0 conda-forge geoviews-core 1.9.5 pyha770c72_0 conda-forge gettext 0.21.1 h8a4c099_0 conda-forge giflib 5.2.1 hbcb3906_2 conda-forge glib 2.74.1 hbc0c0cd_1 conda-forge glib-tools 2.74.1 hbc0c0cd_1 conda-forge grimpfunc 0.0.4 pypi_0 pypi grimpqgis 0.0.1 pypi_0 pypi ``` | ``` fiona 1.8.22 pypi_0 pypi flit-core 3.8.0 pyhd8ed1ab_0 conda-forge folium 0.13.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 hc2a2eb6_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 pypi_0 pypi freetype 2.12.1 hca18f0e_0 conda-forge freexl 1.0.6 h166bdaf_1 conda-forge frozenlist 1.3.3 pypi_0 pypi fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge future 0.18.2 pyhd8ed1ab_6 conda-forge gdal 3.5.2 pypi_0 pypi geopandas 0.12.1 pyhd8ed1ab_1 conda-forge geopandas-base 0.12.1 pyha770c72_1 conda-forge geos 3.11.0 h27087fc_0 conda-forge geotiff 1.7.1 h4fc65e6_3 conda-forge geoviews 1.9.5 pyhd8ed1ab_0 conda-forge geoviews-core 1.9.5 pyha770c72_0 conda-forge gettext 0.21.1 h27087fc_0 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge glib 2.74.1 h6239696_1 conda-forge glib-tools 2.74.1 h6239696_1 conda-forge grimpfunc 0.0.4 pypi_0 pypi grimpqgis 0.0.1 pypi_0 pypi ``` |
Not sure what’s going on. Often the not recognized format indicates there were too many attempts to access NSIDC (you are allowed 16, but they don’t hang up right away so it's easy to get booted out). Try again a bit later and make sure you are not running it simultaneously in more than one notebook or machine (i.e. only have one file downloading from NSIDC at once). Let me know how it goes. Ian
Ian Joughin Polar Science Center Applied Physics Lab University of Washington 1013 NE 40th Street Seattle, WA 98105-6698
Data products: https://nsidc.org/grimp
Data Tools: https://github.com/fastice/GrIMPTools
Phone: 206-221-3177 Fax: 206-616-3142
On Nov 29, 2022, at 2:01 PM, Timothy Bartholomaus @.***> wrote:
Thanks for jumping right back into this, Ian. Importing grimpfunc in both M1 and linux machines, via jupyter lab all works great. But it looks like unpinning some of those packages might have led to other problems for the mac- looks like a gdal problem.
In a cell from your workbook that had previously worked fine
myTerminusUrls = grimp.cmrUrls(mode='terminus') # mode image restricts search to the image products myTerminusUrls.initialSearch();
myTermini = {} for url in myTerminusUrls.getURLS(): year = os.path.basename(url).split('_')[1] # Extract year from name myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}') # Add terminus to data frame print(f'/vsicurl/&url={url}') I now get the following error on the gpd.read_file from url. But! only on mac. On my linux this runs just fine. Here's the complete error message:
CPLE_OpenFailedError Traceback (most recent call last) File fiona/_shim.pyx:83, in fiona._shim.gdal_open_vector()
File fiona/_err.pyx:291, in fiona._err.exc_wrap_pointer()
CPLE_OpenFailedError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp https://urldefense.com/v3/__https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp__;!!K-Hz7m0Vt54!jF4gseudGo0-4x7w4w6XL0Hn0ifSuw1QDEAXe6_3baqmwZhyUezhmAh9VEjL50_2zQzMmfOMiTHQlT_HQZEH$' not recognized as a supported file format.
During handling of the above exception, another exception occurred:
DriverError Traceback (most recent call last) Cell In [18], line 4 2 for url in myTerminusUrls.getURLS(): 3 year = os.path.basename(url).split('_')[1] # Extract year from name ----> 4 myTermini[year] = gpd.read_file(f'/vsicurl/&url={url}') # Add terminus to data frame 5 print(f'/vsicurl/&url={url}')
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:259, in _read_file(filename, bbox, mask, rows, engine, kwargs) 256 path_or_bytes = filename 258 if engine == "fiona": --> 259 return _read_file_fiona( 260 path_or_bytes, from_bytes, bbox=bbox, mask=mask, rows=rows, kwargs 261 ) 262 elif engine == "pyogrio": 263 return _read_file_pyogrio( 264 path_or_bytes, bbox=bbox, mask=mask, rows=rows, **kwargs 265 )
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/geopandas/io/file.py:303, in _read_file_fiona(path_or_bytes, from_bytes, bbox, mask, rows, where, kwargs) 300 reader = fiona.open 302 with fiona_env(): --> 303 with reader(path_or_bytes, kwargs) as features: 304 crs = features.crs_wkt 305 # attempt to get EPSG code
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/env.py:408, in ensure_env_with_credentials.
.wrapper(*args, kwargs) 405 @wraps(f) 406 def wrapper(*args, *kwargs): 407 if local._env: --> 408 return f(args, kwargs) 409 else: 410 if isinstance(args[0], str): File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/init.py:264, in open(fp, mode, driver, schema, crs, encoding, layer, vfs, enabled_drivers, crs_wkt, kwargs) 261 path = parse_path(fp) 263 if mode in ('a', 'r'): --> 264 c = Collection(path, mode, driver=driver, encoding=encoding, 265 layer=layer, enabled_drivers=enabled_drivers, kwargs) 266 elif mode == 'w': 267 if schema: 268 # Make an ordered dict of schema properties.
File ~/opt/anaconda3/envs/greenlandMapping/lib/python3.10/site-packages/fiona/collection.py:162, in Collection.init(self, path, mode, driver, schema, crs, encoding, layer, vsi, archive, enabled_drivers, crs_wkt, ignore_fields, ignore_geometry, kwargs) 160 if self.mode == 'r': 161 self.session = Session() --> 162 self.session.start(self, kwargs) 163 elif self.mode in ('a', 'w'): 164 self.session = WritingSession()
File fiona/ogrext.pyx:540, in fiona.ogrext.Session.start()
File fiona/_shim.pyx:90, in fiona._shim.gdal_open_vector()
DriverError: '/vsicurl/&url=https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp https://urldefense.com/v3/__https://n5eil01u.ecs.nsidc.org/DP4/MEASURES/NSIDC-0642.002/1999.10.13/termini_1999_2000_v02.0.shp__;!!K-Hz7m0Vt54!jF4gseudGo0-4x7w4w6XL0Hn0ifSuw1QDEAXe6_3baqmwZhyUezhmAh9VEjL50_2zQzMmfOMiTHQlT_HQZEH$' not recognized as a supported file format. Again, this is only a problem on mac (2021 M1 MBP), not on linux, and it looks like, from a comment in the former version of environment.yml that this might be a known issue?
I'm copying below a section of conda list from "identical" environments on both machines.
Mac (2021 M1) Linux fiona 1.8.22 py310h5a14c0a_0 conda-forge flit-core 3.8.0 pyhd8ed1ab_0 conda-forge folium 0.13.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 h5bb23bf_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 py310h90acd4f_1 conda-forge freetype 2.12.1 h3f81eb7_1 conda-forge freexl 1.0.6 hb7f2c08_1 conda-forge frozenlist 1.3.3 py310h90acd4f_0 conda-forge fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge future 0.18.2 pyhd8ed1ab_6 conda-forge gdal 3.5.2 py310he22d2f2_3 conda-forge geopandas 0.12.1 pyhd8ed1ab_1 conda-forge geopandas-base 0.12.1 pyha770c72_1 conda-forge geos 3.11.0 hb486fe8_0 conda-forge geotiff 1.7.1 ha1a2aeb_3 conda-forge geoviews 1.9.5 pyhd8ed1ab_0 conda-forge geoviews-core 1.9.5 pyha770c72_0 conda-forge gettext 0.21.1 h8a4c099_0 conda-forge giflib 5.2.1 hbcb3906_2 conda-forge glib 2.74.1 hbc0c0cd_1 conda-forge glib-tools 2.74.1 hbc0c0cd_1 conda-forge grimpfunc 0.0.4 pypi_0 pypi grimpqgis 0.0.1 pypi_0 pypi fiona 1.8.22 pypi_0 pypi flit-core 3.8.0 pyhd8ed1ab_0 conda-forge folium 0.13.0 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 hc2a2eb6_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 pypi_0 pypi freetype 2.12.1 hca18f0e_0 conda-forge freexl 1.0.6 h166bdaf_1 conda-forge frozenlist 1.3.3 pypi_0 pypi fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge future 0.18.2 pyhd8ed1ab_6 conda-forge gdal 3.5.2 pypi_0 pypi geopandas 0.12.1 pyhd8ed1ab_1 conda-forge geopandas-base 0.12.1 pyha770c72_1 conda-forge geos 3.11.0 h27087fc_0 conda-forge geotiff 1.7.1 h4fc65e6_3 conda-forge geoviews 1.9.5 pyhd8ed1ab_0 conda-forge geoviews-core 1.9.5 pyha770c72_0 conda-forge gettext 0.21.1 h27087fc_0 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge glib 2.74.1 h6239696_1 conda-forge glib-tools 2.74.1 h6239696_1 conda-forge grimpfunc 0.0.4 pypi_0 pypi grimpqgis 0.0.1 pypi_0 pypi — Reply to this email directly, view it on GitHub https://urldefense.com/v3/__https://github.com/fastice/GrIMPTools/issues/1*issuecomment-1331376995__;Iw!!K-Hz7m0Vt54!jF4gseudGo0-4x7w4w6XL0Hn0ifSuw1QDEAXe6_3baqmwZhyUezhmAh9VEjL50_2zQzMmfOMiTHQlUi3glxd$, or unsubscribe https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ALICGTU44H2VHUC62SF5YBTWKZ4MTANCNFSM6AAAAAASN53B2A__;!!K-Hz7m0Vt54!jF4gseudGo0-4x7w4w6XL0Hn0ifSuw1QDEAXe6_3baqmwZhyUezhmAh9VEjL50_2zQzMmfOMiTHQlWiSDRVt$. You are receiving this because you commented.
I think your read on this must have been right. I tried yesterday afternoon, and again this morning, and it's working no problem. Kind of unfortunate that this is the way NSIDC declines access attempts, but it's good to know now. I think I'm in business!
Thanks again, Ian. Really, your vision of and support for community/open science is just great. Very much appreciated.
Hi Ian,
I'm somewhat loath to submit another issue here after my first, but now that I've got everything up and running on a machine at home (macbook pro 2016 intel chip) and my linux machine at work (CentOS 7), I really think this might be an actual bug.
I've installed the same
greenlandMapping
environment on my 2021 M1 Pro macbook pro, and proceeded without issue. At a python console, I have no problem importing grimpfunc:However, when I do the same within jupyter lab, I get a fairly long set of errors, despite having run
python -m ipykernel install --user --name=greenlandMapping
several times, both with and without thegreenlandMapping
environment activated.Within jupyter lab, with the
greenlandMapping
environment active and sitting idle (I can, for example, successfully import dask, geopandas, rioxarray, etc.), when I try andimport grimpfunc
I get an error with the boxPicker submodule that stems from the internal call toimport hvplot.xarray
I'm copying this error below, and also attaching two screenshots, if they offer any more context.
As I mentioned at the top, I'm up and running on other systems, so this isn't urgent, but I thought I'd mention as a potential issue in case this comes up for anyone else, or if this is actually easily addressed or meaningful to you. Without debugging the call to
import hvplot.xarray
, specifically within a jupyter lab kernel, I'm not sure this is something I can get to the bottom of without a lot of effort.Thanks again.