search

opengeos / leafmap

A Python package for interactive mapping and geospatial analysis with minimal coding in a Jupyter environment
https://leafmap.org
MIT License
3.18k stars 374 forks source link

QGIS Layer export TIF on leafmap #293

Closed Niko-La closed 1 year ago

Niko-La commented 1 year ago

Hi I am able to get the 03 COG STAC tutorial working using your provided tif files, but when I use the QGIS exported file, I am unable to see any bound values and when trying to run the leafmap it gives me this error

image

giswqs commented 1 year ago

Try the leafmap.cog_validate() function to verify that the exported file is actually a COG. https://leafmap.org/notebooks/42_create_cog/.

Niko-La commented 1 year ago

image

image

I just saved the qgis layer as geotiff. it loads fine on google earth

giswqs commented 1 year ago

Can your share the sample file for debugging?

Niko-La commented 1 year ago

https://drive.google.com/file/d/1eXhyOFy8JwY0_hX3HLHIv1IHu0UtAQ8G/view?usp=sharing

@giswqs this is the tif file outputed from qgis after using georeferencing to map the image

Niko-La commented 1 year ago

@giswqs where you able to replicate my issue. I was thinking it has something to do with bands but when i tried to do it with binary vs rgb , it still did not work.

Alternatively, i tried to load the the image into Google Earth Pro Desktop as an Image overlay and export that as KML file.

But I get the following error, is the add_kml not designed for image overlays?

ERROR:fiona._env:Pointer 'hLayer' is NULL in 'OGR_L_GetName'.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-18-97f50d810440>](https://localhost:8080/#) in <module>
      5 m.add_gdf(gdf, layer_name="Ward")
      6 in_kml = '/content/drive/MyDrive/02_CONSULTING/GISMAP/Scugog_Image_Overlay.kml'
----> 7 m.add_kml(in_kml, layer_name="US States KML")
      8 m

6 frames
[/usr/local/lib/python3.7/dist-packages/fiona/collection.py](https://localhost:8080/#) in __init__(self, path, mode, driver, schema, crs, encoding, layer, vsi, archive, enabled_drivers, crs_wkt, ignore_fields, ignore_geometry, **kwargs)
    160             if self.mode == 'r':
    161                 self.session = Session()
--> 162                 self.session.start(self, **kwargs)
    163             elif self.mode in ('a', 'w'):
    164                 self.session = WritingSession()

fiona/ogrext.pyx in fiona.ogrext.Session.start()

ValueError: Null layer: ''

Perhaps a tutorial on how to do GeoTIFF export from QGIS to Leafmap would be future youtube content?

giswqs commented 1 year ago

Sorry for the delay. For adding local raster datasets, use m.add_raster() function. For remote raster datasets through HTTP URLs, use m.add_cog_layer()

image
Niko-La commented 1 year ago

ran both leafmap.update_package() and !pip install localtileserver osgeo-utils --quiet

Building wheel for osgeo (setup.py) ... error
  ERROR: Failed building wheel for osgeo
    Running setup.py install for osgeo ... error
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e77lpjbm/osgeo_108f7a531ffb40f584ead306c06956f1/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e77lpjbm/osgeo_108f7a531ffb40f584ead306c06956f1/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-9j6933r0/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.7/osgeo Check the logs for full command output.
[8]
0s
123456789

but cant proceed. 

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/leafmap/common.py](https://localhost:8080/#) in check_package(name, URL)
    270     try:
--> 271         __import__(name.lower())
    272     except Exception:

5 frames
ModuleNotFoundError: No module named 'osgeo_utils'

During handling of the above exception, another exception occurred:

ImportError                               Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/leafmap/common.py](https://localhost:8080/#) in check_package(name, URL)
    272     except Exception:
    273         raise ImportError(
--> 274             f"{name} is not installed. Please install it before proceeding. {URL}"
    275         )
    276 

ImportError: localtileserver is not installed. Please install it before proceeding. https://github.com/banesullivan/localtileserver

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------
giswqs commented 1 year ago

Try the following

pip install --find-links=https://girder.github.io/large_image_wheels --no-cache GDAL
pip install localtileserver
Niko-La commented 1 year ago

Still getting the same missing module error for osgeo_utils and localtileserver

pip install --find-links=https://girder.github.io/large_image_wheels --no-cache GDAL
pip install localtileserver
!pip install --find-links=https://girder.github.io/large_image_wheels --no-cache GDAL
!pip install localtileserver

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Looking in links: https://girder.github.io/large_image_wheels
Requirement already satisfied: GDAL in /usr/local/lib/python3.7/dist-packages (2.2.2)
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: localtileserver in /usr/local/lib/python3.7/dist-packages (0.6.1)
Requirement already satisfied: scooby in /usr/local/lib/python3.7/dist-packages (from localtileserver) (0.7.0)
Requirement already satisfied: flask-restx>=0.5.0 in /usr/local/lib/python3.7/dist-packages (from localtileserver) (1.0.3)
Requirement already satisfied: Flask-Caching in /usr/local/lib/python3.7/dist-packages (from localtileserver) (2.0.1)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from localtileserver) (2.28.1)
Requirement already satisfied: flask>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from localtileserver) (2.1.3)
Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from localtileserver) (8.1.3)
Requirement already satisfied: server-thread in /usr/local/lib/python3.7/dist-packages (from localtileserver) (0.2.0)
Requirement already satisfied: werkzeug<2.2 in /usr/local/lib/python3.7/dist-packages (from localtileserver) (2.1.2)
Requirement already satisfied: flask-cors in /usr/local/lib/python3.7/dist-packages (from localtileserver) (3.0.10)
Requirement already satisfied: GDAL in /usr/local/lib/python3.7/dist-packages (from localtileserver) (2.2.2)
Requirement already satisfied: large-image[gdal]>=1.14.1 in /usr/local/lib/python3.7/dist-packages (from localtileserver) (1.17.3)
Requirement already satisfied: importlib-metadata>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from flask>=2.0.0->localtileserver) (4.13.0)
Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.7/dist-packages (from flask>=2.0.0->localtileserver) (2.1.2)
Requirement already satisfied: Jinja2>=3.0 in /usr/local/lib/python3.7/dist-packages (from flask>=2.0.0->localtileserver) (3.1.2)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from flask-restx>=0.5.0->localtileserver) (2022.6)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from flask-restx>=0.5.0->localtileserver) (4.3.3)
Requirement already satisfied: aniso8601>=0.82 in /usr/local/lib/python3.7/dist-packages (from flask-restx>=0.5.0->localtileserver) (9.0.1)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=3.6.0->flask>=2.0.0->localtileserver) (3.10.0)
Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=3.6.0->flask>=2.0.0->localtileserver) (4.1.1)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.7/dist-packages (from Jinja2>=3.0->flask>=2.0.0->localtileserver) (2.0.1)
Requirement already satisfied: cachetools>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (5.2.0)
Requirement already satisfied: psutil>=4.2.0 in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (5.4.8)
Requirement already satisfied: numpy>=1.10.4 in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (1.21.6)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (7.1.2)
Requirement already satisfied: palettable in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (3.3.0)
Requirement already satisfied: large-image-source-gdal>=1.17.3 in /usr/local/lib/python3.7/dist-packages (from large-image[gdal]>=1.14.1->localtileserver) (1.17.3)
Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from large-image-source-gdal>=1.17.3->large-image[gdal]>=1.14.1->localtileserver) (21.3)
Requirement already satisfied: pyproj>=2.2.0 in /usr/local/lib/python3.7/dist-packages (from large-image-source-gdal>=1.17.3->large-image[gdal]>=1.14.1->localtileserver) (3.2.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from pyproj>=2.2.0->large-image-source-gdal>=1.17.3->large-image[gdal]>=1.14.1->localtileserver) (2022.9.24)
Requirement already satisfied: cachelib>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from Flask-Caching->localtileserver) (0.9.0)
Requirement already satisfied: Six in /usr/local/lib/python3.7/dist-packages (from flask-cors->localtileserver) (1.15.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->flask-restx>=0.5.0->localtileserver) (5.10.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->flask-restx>=0.5.0->localtileserver) (22.1.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->flask-restx>=0.5.0->localtileserver) (0.19.2)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->large-image-source-gdal>=1.17.3->large-image[gdal]>=1.14.1->localtileserver) (3.0.9)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->localtileserver) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->localtileserver) (1.24.3)
Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.7/dist-packages (from requests->localtileserver) (2.1.1)
Requirement already satisfied: uvicorn in /usr/local/lib/python3.7/dist-packages (from server-thread->localtileserver) (0.20.0)
Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.7/dist-packages (from uvicorn->server-thread->localtileserver) (0.14.0)
giswqs commented 1 year ago

This is a GDAL installation issue rather than a leafmap issue. You might need to uninstall GDAL and reinstall it. Or create a new fresh conda env to install it.

image

Niko-La commented 1 year ago

This is a GDAL installation issue rather than a leafmap issue. You might need to uninstall GDAL and reinstall it. Or create a new fresh conda env to install it.

image

ok got it, I was just doing colab google environment. will try local version.

giswqs commented 1 year ago

Localtileserver does not work on Colab. https://github.com/banesullivan/localtileserver/issues/70

Niko-La commented 1 year ago

Switched from Colab to local jupyterlab.

Try the following

pip install --find-links=https://girder.github.io/large_image_wheels --no-cache GDAL
pip install localtileserver
Looking in links: https://girder.github.io/large_image_wheels
Collecting GDAL
  Downloading GDAL-3.6.0.1.tar.gz (757 kB)
     ------------------------------------- 757.4/757.4 kB 11.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Installing collected packages: GDAL
  Running setup.py install for GDAL: started
  Running setup.py install for GDAL: finished with status 'error'
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data[ in a text editor](command:workbench.action.openLargeOutput?30550448-8c3b-480e-98b7-f5a6a7d74f61)
  DEPRECATION: GDAL is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  error: subprocess-exited-with-error

  × Running setup.py install for GDAL did not run successfully.
  │ exit code: 1
  ╰─> [140 lines of output]
      running install
      C:\Users\kava\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdal.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdalconst.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdalnumeric.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdal_array.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gnm.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\ogr.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\osr.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\__init__.py -> build\lib.win-amd64-cpython-311\osgeo
      creating build\lib.win-amd64-cpython-311\osgeo_utils
      copying gdal-utils\osgeo_utils\gdal2tiles.py -> build\lib.win-amd64-cpython-311\osgeo_utils
      copying gdal-utils\osgeo_utils\gdal2xyz.py -> build\lib.win-amd64-cpython-311\osgeo_utils
...
hint: See above for output from the failure.

[notice] A new release of pip available: 22.3 -> 22.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data[ in a text editor](command:workbench.action.openLargeOutput?ec12cc00-e974-48ba-97bf-c70febcb26b6)
Collecting localtileserver
  Using cached localtileserver-0.6.1-py3-none-any.whl (19.4 MB)
Requirement already satisfied: click in c:\users\kava\appdata\local\programs\python\python311\lib\site-packages (from localtileserver) (8.1.3)
Collecting flask>=2.0.0
  Using cached Flask-2.2.2-py3-none-any.whl (101 kB)
Collecting Flask-Caching
  Using cached Flask_Caching-2.0.1-py3-none-any.whl (28 kB)
Collecting flask-cors
  Using cached Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
Collecting flask-restx>=0.5.0
  Using cached flask_restx-1.0.3-py2.py3-none-any.whl (5.3 MB)
Collecting GDAL
  Using cached GDAL-3.6.0.1.tar.gz (757 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting large-image[gdal]>=1.14.1
  Using cached large_image-1.17.3-py3-none-any.whl (65 kB)
Requirement already satisfied: requests in c:\users\kava\appdata\local\programs\python\python311\lib\site-packages (from localtileserver) (2.28.1)
Collecting server-thread
  Using cached server_thread-0.2.0-py3-none-any.whl (8.5 kB)
Requirement already satisfied: scooby in c:\users\kava\appdata\local\programs\python\python311\lib\site-packages (from localtileserver) (0.7.0)
Requirement already satisfied: werkzeug<2.2 in c:\users\kava\appdata\local\programs\python\python311\lib\site-packages (from localtileserver) (2.1.2)
Collecting flask>=2.0.0
  Using cached Flask-2.2.1-py3-none-any.whl (101 kB)
  Using cached Flask-2.2.0-py3-none-any.whl (101 kB)
...
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\users\kava\appdata\local\programs\python\python311\lib\site-packages (from packaging->large-image-source-gdal>=1.17.3->large-image[gdal]>=1.14.1->localtileserver) (3.0.9)
Installing collected packages: GDAL, cachetools, cachelib, uvicorn, large-image, flask, server-thread, large-image-source-gdal, flask-restx, flask-cors, Flask-Caching, localtileserver
  Running setup.py install for GDAL: started
  Running setup.py install for GDAL: finished with status 'error'
Output exceeds the [size limit](command:workbench.action.openSettings?%5B%22notebook.output.textLineLimit%22%5D). Open the full output data[ in a text editor](command:workbench.action.openLargeOutput?c1863c7c-3c8e-43a7-ba24-c2fa660c71de)
  DEPRECATION: GDAL is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  error: subprocess-exited-with-error

  × Running setup.py install for GDAL did not run successfully.
  │ exit code: 1
  ╰─> [143 lines of output]
      running install
      C:\Users\kava\AppData\Local\Programs\Python\Python311\Lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-311
      creating build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdal.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdalconst.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdalnumeric.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gdal_array.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\gnm.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\ogr.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\osr.py -> build\lib.win-amd64-cpython-311\osgeo
      copying osgeo\__init__.py -> build\lib.win-amd64-cpython-311\osgeo
      creating build\lib.win-amd64-cpython-311\osgeo_utils
      copying gdal-utils\osgeo_utils\gdal2tiles.py -> build\lib.win-amd64-cpython-311\osgeo_utils
      copying gdal-utils\osgeo_utils\gdal2xyz.py -> build\lib.win-amd64-cpython-311\osgeo_utils
...
hint: See above for output from the failure.

[notice] A new release of pip available: 22.3 -> 22.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip

@giswqs switching to local from colab.

ps. also is there way to display leafmap in the vs-code notebook image

giswqs commented 1 year ago

Try creating a fresh conda env to install leafmap, GDAL, and localtileserver. Do not install packages into the base Python env. It will likely break things.