gdal2tiles.py permission denied error when writing tiles to S3 via non-root user

[AnguIar]

Expected behavior and actual behavior.

1. I tried writing tiles to an AWS S3 bucket using gdal2tiles.py utility
2. I ran into this error: PermissionError: [Errno 13] Permission denied: '/vsis3'.
3. However, it did work with sudo
4. I later figured that GDAL is creating a vsis3 folder under root. A regular user is not able to create folder within / directory.
5. I thought it was related to the temporary folder GDAL is creating in order to upload to S3, so I configured the CPL_TMPDIR to be accessible to a regular user. That did not help.
6. After running with root privileges, only .png and .png.aux.xml files were transferred to the S3 bucket. .kml, .xml, openlayers.html files were left in the local /vsis3 folder.

Steps to reproduce the problem.

run gdal2tiles.py utility and try to upload to an S3 bucket as a non-root user. gdal2tiles.py -z 0-5 temp.vrt /vsis3/bucket/tiles

Operating system

Ubuntu 18.04.2 64bit

GDAL version and provenance

GDAL 3.2.0

[kannes]

This does not look like a valid vsis3 path, https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files says

Recognized filenames are of the form /vsis3/bucket/key, where bucket is the name of the S3 bucket and key is the S3 object “key”, i.e. a filename potentially containing subdirectories.

It makes sense that in your case gdal interprets the path as a local path and tries to write to a local "vsis3" directory in your local system root. Are you sure the PNG files did not end up there because of some other event? It would surprise me also if gdal2tiles.py would write .aux.xml files.

[AnguIar]

This does not look like a valid vsis3 path, https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files says

Recognized filenames are of the form /vsis3/bucket/key, where bucket is the name of the S3 bucket and key is the S3 object “key”, i.e. a filename potentially containing subdirectories.

You are correct, I've provided a wrong path in my example. I actually am using /vsis3/bucket/key convention - if I wouldn't the files weren't created in the S3 bucket.

It makes sense that in your case gdal interprets the path as a local path and tries to write to a local "vsis3" directory in your local system root. Are you sure the PNG files did not end up there because of some other event? It would surprise me also if gdal2tiles.py would write .aux.xml files.

I am not aware of any other way in which the PNGs could reach out the S3 bucket. I think it might be caused by some bad implementation of the libcurl library in this specific utility. I have worked against S3 with other utilities and everything seemed to work fine.