search

locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
240 stars 46 forks source link

Errors of "FAILURE(3) CPLE_AWSInvalidCredentials(15) 'AWSInvalidCredentials.'" has occurred while running "Supervised Machine Learning". #534

Closed JenniferYingyiWu2020 closed 3 years ago

JenniferYingyiWu2020 commented 3 years ago

Hi, Before I copied the codes and executed them, I have read the description about the "Supervised Machine Learning", which refers to "https://rasterframes.io/supervised-learning.html". However, I got the following errors: " FAILURE(3) CPLE_AWSInvalidCredentials(15) "AWSInvalidCredentials." AWS_SECRET_ACCESS_KEY and AWS_NO_SIGN_REQUEST configuration options not defined, and /root/.aws/credentials not filled

py4j.protocol.Py4JJavaError: An error occurred while calling o142.collectToPython. " After my searching on the Internet, I run the command to confirm whether the issue related about "AWS account". 1

If the running "Supervised Machine Learning" needs "AWS account", then how to create the "AWS account"? Shall I access the "AWS" website and create my own username? So, could you pls give me some suggestions?
vpipkt commented 3 years ago

@JenniferYingyiWu2020 can you try setting AWS_NO_SIGN_REQUEST=YES?

That is equivalent to the recommended options in the aws cli shown on the landsat-pds page

See the following GDAL doc for more info: https://gdal.org/user/virtual_file_systems.html#vsis3-aws-s3-files

JenniferYingyiWu2020 commented 3 years ago

@vpipkt , could you pls tell me how to set "AWS_NO_SIGN_REQUEST=YES"?

JenniferYingyiWu2020 commented 3 years ago

@vpipkt , I used the local ".tif" image dataset, and "https://github.com/locationtech/rasterframes/raw/develop/pyrasterframes/src/test/resources/luray-labels.geojson". However, when I run the "Supervised Machine Learning", I got the following errors: java.lang.IllegalArgumentException: Error fetching data for one of: GDALRasterSource(file:///home/jenniferwu/Raster_Data_Set/s22s-test-geotiffs/luray_snp/SCL.tif), GDALRasterSource(file:///home/jenniferwu/Raster_Data_Set/s22s-test-geotiffs/luray_snp/B01.tif), GDALRasterSource(file:///home/jenniferwu/Raster_Data_Set/s22s-test-geotiffs/luray_snp/B02.tif), ..., GDALRasterSource(file:///home/jenniferwu/Raster_Data_Set/s22s-test-geotiffs/luray_snp/B11.tif), GDALRasterSource(file:///home/jenniferwu/Raster_Data_Set/s22s-test-geotiffs/luray_snp/B12.tif) 6 Could you pls share the image dataset that are used under "Supervised machine learning" to me? The ".tif" image dataset of "s3://s22s-test-geotiffs/luray_snp". Thanks!

vpipkt commented 3 years ago

@JenniferYingyiWu2020 i think the second issue regarding java.lang.IllegalArgumentException: transpose requires all collections to have the same size may be due to different resolutions / dimensions in the set of rasters given.

JenniferYingyiWu2020 commented 3 years ago

OK, I got it, thanks!

vpipkt commented 3 years ago

It seems from this comment https://github.com/locationtech/rasterframes/issues/537#issuecomment-759884799 that the issue is resolved. If correct please close the issue, or let me know so I can close it.

JenniferYingyiWu2020 commented 3 years ago

Of course, the issue has been resolved, which can be closed after confirmation.