search

OSGeo / gdal

GDAL is an open source MIT licensed translator library for raster and vector geospatial data formats.
https://gdal.org
Other
4.92k stars 2.56k forks source link

Supporting float16 #10144

Open eschnett opened 5 months ago

eschnett commented 5 months ago

Feature description

I would like to be able to store and load float16 values in datasets. Many dataset formats support this, and many modern compilers support this as well. I do need to store float16 values in my project; I will either have to masquerade them as uint16, but I would obviously prefer if float16 was supported natively.

Is there a particular plan to support float16?

Additional context

C++23 will support float16, but I would not want to wait until C++23 is mature enough to be required to build gdal.

I assume that the fact that not all dataset formats support float16 is not a hindrance.

I assume that there would need to be some kind of fallback mechanism (soft-float?) to support compilers that do not support a native float16 format. At the very least this would need to involve converting from/to float32, so that callers can pass in float32 data that can then be converted.

If float16 support is desirable then I might be able to implement this.

jratike80 commented 5 months ago

Maybe this should be asked also on the gdal-dev list.

rouault commented 5 months ago

Is there a particular plan to support float16?

not by me at least :-), but no opposition that you tackle this

I assume that there would need to be some kind of fallback mechanism (soft-float?) to support compilers that do not support a native float16 format.

in port/cpl_float.h we have routines to convert from (integer representation of) Float16 <--> (integer representation of) Float32

If float16 support is desirable then I might be able to implement this.

There would be a fair amount to add a GDT_Float16 data type. 65bb32a37a3 / https://gdal.org/development/rfc/rfc87_signed_int8.html / #6633 (Int8 addition) or 930b2a06d88 / #5257 (Int64/UInt64 addition) can give you some clues of the places where changes might be be needed (obviously changes in drivers are dependent on their capability of handling the new data type). There are potentially some backward compatibility concerns as some formats (like GTiff and HDF5, cf recent https://github.com/OSGeo/gdal/pull/10052) do Float16 -> Float32 conversion. A RFC might be appropriate.