CartoDB / raster-loader

https://raster-loader.readthedocs.io
Other
15 stars 4 forks source link

Do not rely on gdalinfo subprocess calls nor gdal_bin #44

Closed francois-baptiste closed 1 year ago

francois-baptiste commented 1 year ago

It would be best if we can rely only on python wrapped gdal (like rasterio) instead of relying on subprocess calls

In the google clound shell I got the following error

fbaptiste@cloudshell:~/raster-loader (cartodb-on-gcp-datascience)$ carto bigquery upload --file_path ./raster_loader/tests/fixtures/mosaic.tif --project cartodb-on-gcp-datascience --dataset fbaptiste
Preparing to upload raster file to BigQuery...
File Path: ./raster_loader/tests/fixtures/mosaic.tif
Running gdalinfo...
Traceback (most recent call last):
  File "/home/fbaptiste/.local/bin/carto", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/dist-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/fbaptiste/.local/lib/python3.9/site-packages/raster_loader/cli/bigquery.py", line 56, in upload
    print_gdalinfo(file_path)
  File "/home/fbaptiste/.local/lib/python3.9/site-packages/raster_loader/io.py", line 386, in print_gdalinfo
    subprocess.run(["gdalinfo", file_path])
  File "/usr/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1823, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gdalinfo'

running sudo apt-get install gdal-bin fixed the problem but a full python managed solution would be best as you have to rerun this install at every start up on google cloud shell