locationtech / rasterframes

Geospatial Raster support for Spark DataFrames
http://rasterframes.io
Apache License 2.0
244 stars 45 forks source link

Converting Cell type from not having 'NoData' defined to it being defined takes a lot of time. #612

Open arind123 opened 1 year ago

arind123 commented 1 year ago

I am trying to convert "int16raw" to "int16ud-99" takes about 7 minutes for a band of a tile of Sentinel 2 data. I saw the link 'https://github.com/locationtech/rasterframes/issues/343'. But was there any new functions made available?

I am currently using the following code as given in the documentation:

_ct = red_10mt_rf.select(rf_cell_type('red_10mt')).distinct().first()[0]

new_ct = CellType(_ct).with_no_data_value(-99)

red_10mt_rf_ct = red_10mt_rf.select('proj_raster_path', rf_convert_cell_type('red_10mt', new_ct).alias('red_10mt'), "spatial_index")