VioletGiraffe
commented
1 year ago We also need the latest libraw, so +1 for this request.
Additionally, could you add a feature to build just the libraw itself, without the examples and other unnecessary dependencies? This is how we build clean libraw:
cmake -DCMAKE_INSTALL_PREFIX=%CD%\%INSTALL_DIR% -DBUILD_SHARED_LIBS=OFF -DENABLE_OPENMP=OFF -DENABLE_LCMS=OFF -DENABLE_EXAMPLES=OFF -DCMAKE_DEBUG_POSTFIX=d ^
-DJPEG_INCLUDE_DIR=%CD%/jpeg/include -DJPEG_LIBRARY_DEBUG=%CD%/jpeg/lib/jpeg-staticd.lib -DJPEG_LIBRARY_RELEASE=%CD%/jpeg/lib/jpeg-static.lib ^ The dependencies on jasper, lcms are not needed for libraw functionality, or at the very least we don't understand what they are needed for and we never found a use case despite using "bare" version of libraw (as described above) quite heavily in our software.
It's not clear where the jasper dependency even came from.
Here's the full list of configurable libraw features, of which only OpenMP is exposed by vcpkg:
// Build library with Raspberry Pi RAW support (default=OFF)
ENABLE_6BY9RPI:BOOL=OFF
// Build library with debug message from dcraw (default=OFF)
ENABLE_DCRAW_DEBUG:BOOL=OFF
// Build library with extra Demosaic pack GPL2 (default=OFF)
ENABLE_DEMOSAIC_PACK_GPL2:BOOL=OFF
// Build library with extra Demosaic pack GPL3 (default=OFF)
ENABLE_DEMOSAIC_PACK_GPL3:BOOL=OFF
// Build library with sample command-line programs (default=ON)
ENABLE_EXAMPLES:BOOL=OFF
// Build library with LCMS support (default=ON)
ENABLE_LCMS:BOOL=OFF
// Build library with OpenMP support (default=ON)
ENABLE_OPENMP:BOOL=OFF
// Build library with extra RawSpeed codec support (default=OFF)
ENABLE_RAWSPEED:BOOL=OFF
// Build library with Foveon X3F support (default=OFF)
ENABLE_X3FTOOLS:BOOL=OFF
jedfrechette
aklemets
dg0yt
Library name: LibRaw
New version number: Snapshot 202101
Other information that may be useful (release notes, etc...)
== LibRaw snapshot 202101 ==
= Camera format support: = Lossy compressed CR3 files Lossy compressed RAF files Uncompressed floating point DNG files (16-24-32 bit) Deflate compressed striped DNG files XMP and HEIF-preview extraction from CR3 files
= Camera support = Apple iPhone 12 Max, iPhone 12 Max Pro Canon EOS R5, EOS R6, EOS 850D, EOS-1D X Mark III (lossy compressed files) FujiFilm X-S10 Hasselblad CFV II 50C", Leica M10-R, Q2 Monochrom, S3, SL2-S Nikon Z 5, Z 6 II, Z 7 II Olympus E-M10 Mark IV Panasonic DC-G100 / G110, DC-S5 Sony ILCE-7C (A7C), ILCE-7SM3 (A7S III) Zeiss ZX1 Plus multiple DNG-recording cameraphones/drones/etc.
= Almost dropped camera support: = Old/partially supported video/cinema cameras support is available only if LibRaw is compiled with USE_OLD_VIDEOCAMS defined (this #define is converted internally to LIBRAW_OLD_VIDEO_SUPPORT defined). This affects: Arri cameras Canon C500 RED Cine cameras We plan to keep the support in LibRaw 0.21 (if compiled with define mentioned above) and completely drop the support after 0.21. If you're using LibRaw to decode RED Cine files, see LibRaw*datastreams section below (in short: you'll need to use old fstream-based datastream under Windows). Cinema DNG files are not affected by this.
= API/ABI changes =
Decoding parameters imgdata.params is split into imgdata.rawparams: parameters used for metadata/raw data decoding imgdata.params: parameters used at postprocessing stage
imgdata.params.raw_processing options are split into two sub-options sets imgdata.rawparams.options: raw processing flags, useful for normal operations imgdata.rawparams.specials: special modes (e.g decoding only delta pixels from Sony ARW)
Old LIBRAWPROCESSING flags are renamed to LIBRAWRAWOPTIONS and LIBRAW_RAWSPECIAL* LIBRAW_PROCESSING_DP2Q_INTERPOLATE... flags are renamed to LIBRAW_RAWSPECIAL_NODP2Q_INTERPOLATE with inverted meaning (with the goal to have zero imgdata.rawparams.specials in normal use).
New processing flag LIBRAW_RAWOPTIONS_PROVIDE_NONSTANDARD_WB If set (default is not), and when applicable, color.cam_mul[] and color.WB_Coeffs/WBCT_Coeffs will contain WB settings for a non-standard workflow. Right now only Sony DSC-F828 is affected: camera-recorded white balance can't be directly applied to raw data because WB is for RGB, while raw data is RGBE.
New processing flag: LIBRAW_RAWOPTIONS_CAMERAWB_FALLBACK_TO_DAYLIGHT If set (default is not), LibRaw::dcraw_process() will fallback to daylight WB (excluding some very specific cases like Canon D30). This is how LibRaw 0.19 (and older) works. If not set: LibRaw::dcraw_process() will fallback to calculated auto WB if camera WB is requested, but appropriate white balance was not found in metadata.
Removed LIBRAW_PROCESSING_SKIP_MAKERNOTES processing flag: with new Windows(-handle) datastreams metadata parsing performance is enough to not skip makernotes parsing.
new output parameter: imgdata.params.output_flags and new enum LibRaw_output_flags Right now the only flag is implemented: LIBRAW_OUTPUT_FLAGS_PPMMETA setting it will result into metadata written to PPM/PGM output file
LibRaw_*datastreams change and corresponding open_file() parameters change LibRaw_file_datastream (based on iostreams) is moved under
ifdef LIBRAW_USE_DEPRECATED_IOSTREAMS_DATASTREAM
Standard datastreams are: LibRaw_bigfile_datastream (FILE based) for POSIX systems LibRaw_bigfile_buffered_datastream - for use with Win32 syscalls API changes: if LIBRAW_USE_DEPRECATED_IOSTREAMS_DATASTREAM is NOT defined, than LibRaw::open_file() and similar calls drops the last INT64 max_buffered_sz parameter which defines automatic selection between iostreams and FILE based datasteams.
Note: LibRaw_bigfile_buffered_datastream does NOT implements make_jas_stream() call, so RED Cine file decoding is not possible with this datastream. Use deprecaded iostreams implementation instead.
We plan to keep LIBRAW_USE_DEPRECATED_IOSTREAMS_DATASTREAM up to LibRaw 0.21 and drop in 0.22.
Fuji decoder: fuji_decode_loop/fuji_decode_strip parameters has changed; look into source code for details if you're implementing your own fuji_decode_loop() call in subclass to provide parallel fuji decoding.
Canon CR3 H265 (HEIF) preview limited support:
LibRaw::capabilities Two bits added: LIBRAW_CAPS_ZLIB if compiled with USE_ZLIB LIBRAW_CAPS_JPEG if compiled with USE_JPEG
samples/raw-identify.cpp sample: removed lot of extra output used only by development team
Windows datastreams: CreateFile2 is used if compiled for UWP.
API-datastruct: clarified behavior with use_camera_wb=1 setting.
identify(): restored the differentiation between fsize and flen.
New compile-time LIBRAW_FORCE_OPENMP to skip compiler version check (if LibRaw user is absolutely sure that OpenMP is supported by compiler)
2020-10-14 Alex Tutubalin lexa@lexa.ru
2020-10-14 Alex Tutubalin lexa@lexa.ru
LibRaw 0.20.1
Improvements:
Fixes for normal files processing:
Fixes for damaged/special crafted files processing:
Not fixes, but makes ASAN/compilers/etc happy:
2020-07-23 Alex Tutubalin lexa@lexa.ru
== Camera Format support == Canon CR3 GoPro (via GPR SDK) Panasonic 14-bit Fujifilm compressed/16bit Rapsberry Pi RAW+JPEG format (if USE_6BY9RPI defined) Foveon X3F support changed: it is supported only if USE_X3FTOOLS defined at build (see below for 'Imported code policy changed')
== Camera support (+59, 1131 total) == Canon: PowerShot G5 X Mark II, G7 X Mark III, SX70 HS, EOS R, EOS RP, EOS 90D, EOS 250D, EOS M6 Mark II, EOS M50, EOS M200 EOS 1DX Mark III (lossless files only) DJI Mavic Air, Osmo Action FujiFilm GFX 100, X-A7, X-Pro3, X100V, X-T4 (uncompressed/lossless compressed only), X-T200 GoPro Fusion, HERO5, HERO6, HERO7, HERO8 Hasselblad L1D-20c, X1D II 50C Leica D-LUX7, Q-P, Q2, V-LUX5, C-Lux / CAM-DC25, SL2, M10 Monochrom Nikon D780, Z50, P950 Olympus TG-6, E-M5 Mark III, E-PL10, E-M1 Mark III, Panasonic DC-FZ1000 II, DC-G90, DC-S1, DC-S1R, DC-S1H, DC-TZ95 PhaseOne IQ4 150MP Ricoh GR III Sony A7R IV, A9 II, ILCE-6100, ILCE-6600, RX0 II, RX100 VII Zenit M
also multiple smartphones (the tested ones are listed in LibRaw::cameraList)
== Source code re-arranged ==
== Normalized make/model ==
There is a huge number of identical cameras sold under different names, depending on the market (e.g. multiple Panasonic or Canon models) and even some identical cameras sold under different brands (Panasonic -> Leica, Sony -> Hasselblad).
To reduce clutter, a normalization mechanism has been implemented in LibRaw:
In imgdata.idata: char normalized_make[64]; - primary vendor name (e.g. Panasonic for Leica re-branded cameras) char normalized_model[64]; - primary camera model name unsigned maker_index; - primary vendor name in indexed form (enum LibRaw_cameramaker_index, LIBRAWCAMERAMAKER* constant). These fields are always filled upon LibRaw::open_file()/open_buffer() calls.
const char* LibRaw::cameramakeridx2maker(int index): converts maker_index to normalized_make.
We recommend that you use these normalized names in a variety of data tables (color profiles, etc.) to reduce the number of duplicate entries.
New vendor index values will be added strictly to the end of the LibRaw_cameramaker_index table, ensuring that the numbers assigned to vendors that are already known to LibRaw will not change.
== DNG frame selection ==
DNG frames selection code re-worked:
by default all frames w/ the NewSubfileType tag equal to 0 (high-res image) are added to the list of available images (selection performed via imgdata.params.shot_select field, as usual)
the special case for Fuju SuperCCD (SamplesPerPixel == 2) works as before: shot_select=1 will extract second sub-image.
Additional flags to imgdata.params.raw_processing_options: LIBRAW_PROCESSING_DNG_ADD_ENHANCED - will add Enhanced DNG frame (NewSubfileType == 16) to the list of available frames LIBRAW_PROCESSING_DNG_ADD_PREVIEWS - will add previews (NewSubfileType == 1) to the list.
By default, DNG frames are not reordered and are available in same order as in DNG (LibRaw traverses IFD/Sub-IFD trees in deep-first order). To prioritize the largest image, set LIBRAW_PROCESSING_DNG_PREFER_LARGEST_IMAGE bit in imgdata.params.raw_processing_options.
== Imported code policy disclaimer ==
We've changed the policy regarding 3rd party code imported into LibRaw.
We (like other authors of open-source RAW parsers) gladly import support code for various RAW formats from other projects (if the license allows it). This is done to expand camera support. Unfortunately, not all imported code can tolerate truncated or otherwise damaged raw files, as well as arbitrary conditions or arbitrary data; not all authors handle rejecting unexpected input well.
LibRaw is now widely used in various projects, including ImageMagick, which, in turn, is often used on web sites to process any input images, including arbitrary data from unknown users. This opens up wide possibilities for exploiting the various vulnerabilities present in the code borrowed from other projects into LibRaw. In order to avoid such security risks, - the borrowed code will no longer compile by default. We are not able to support it in general case, and the authors refuse to add code to reject unexpected input.
Thus, if you use some kind of camera for which the support is disabled by default, you need to recompile LibRaw for your specific case.
Formats currently affected: X3F (Foveon) file format. Code is imported from Kalpanika X3F tools: https://github.com/Kalpanika/x3f To turn the support on, define USE_X3FTOOLS
Rapsberry Pi RAW+JPEG format. Code is imported from https://github.com/6by9/dcraw/, To turn the support on, define USE_6BY9RPI Format support is indicated via LibRaw::capabilities() call with flags: LIBRAW_CAPS_X3FTOOLS - Foveon support LIBRAW_CAPS_RPI6BY9 - RPi RAW+JPEG support
== GoPro .gpr format support == GoPro format supported via open-source GPR SDK See README.GoPro.txt for details.
== Windows support/Windows unicode (wchar_t*) filenames support ==
(old) LibRaw's WIN32 external define split into 3 defines to fine tune compiler/api compatibility: LIBRAW_WIN32_DLLDEFS - use to compile DLLs (dllimport/dllexport attributes) LIBRAW_WIN32_UNICODEPATHS - indicates that runtime has calls/datatypes for wchar_t filenames LIBRAW_WIN32_CALLS - use Win32 calls where appropriative (binary mode for files, LibRaw_windows_datastream, _snprintf instead of snprintf, etc).
If the (old) WIN32 macro is defined at compile time, all three new defines are defined in libraw.h If not, these defines are defined based on compiler version/libc++ defines
LibRaw::open_file(wchar_t*) is always compiled in under Windows, but if LIBRAW_WIN32_UNICODEPATHS (see above) is not defined, this call will return LIBRAW_NOT_IMPLEMENTED. Use (LibRaw::capabilities() & LIBRAW_CAPS_UNICODEPATHS) on runtime to check that this call was really implemented (or check for #ifdef LIBRAW_WIN32_UNICODEPATHS after #include)
== LibRaw*datastream simplified ==
tempbuffer_open, subfile_open are not used, so removed from LibRaw_abstract_datastream and derived classes.
jpeg_src() call implemented using ->read() call and own buffering (16k buffer).
buffering_off() call added. It should be used in derived classes to switch from buffered reads to unbuffered.
== minor/unsorted changes ==
new flag LIBRAW_WARN_DNGSDK_PROCESSED to indicate decoder used
LibRaw::open() call, max_buf_size special meaning: == 1 => open using bigfile_datastream == 2 => open using file_datastream
Add support for zlib during configure
Fixed multiple problems found by OSS-Fuzz
Lots of changes in imgdata.makernotes (hope someone will document it)
DNG SDK could be used (if enabled) to unpack multi-image DNG files.
DNG whitelevel calculated via BitsPerSample if not set via tags.
DNG: support for LinearDNG w/ BlackLevelRepeat.. pattern
Generic Arri camera format replaced w/ list of specific camera models in supported cameras list.
new samples/rawtextdump sample: allows one to dump (small selection) of RAW data in text format.
samples/raw-identify:
samples/dcraw_emu: fixed +M handling
better support for Nikon Coolscan 16-bit NEF files.
Visual Studio project files: re-generated to .vcxproj (Visual Studio 2019), different intermediate folders for different sub-projects to allow 1-step rebuild.
imgdata.makernotes...cameraspecific: removed the vendor name prefix from variables.
Bayer images: ensure that even margins have the same COLOR() for both the active sensor area and the full sensor area.
raw processing flag bit LIBRAW_PROCESSING_CHECK_DNG_ILLUMINANT inverted and renamed to LIBRAW_PROCESSING_DONT_CHECK_DNG_ILLUMINANT. If not set, DNG illuminant will be checked.
New libraw_decoder_t flags: LIBRAW_DECODER_FLATDATA - in-file data could be used as is (if byte order matches), e.g. via mmap() LIBRAW_DECODER_FLAT_BG2_SWAPPED - special flag for Sony ARQ: indicates R-G-G2-B channel order in 4-color data
Camera-recorded image crop data is parsed into imgdata.sizes.raw_inset_crop structure: ctop,cleft,cwidth,cheight - crop size. aspect - LibRawImageAspects enum (3to2, 4to3, etc)
New define LIBRAW_NO_WINSOCK2 to not include winsock2.h on compile
New processing flag LIBRAW_PROCESSING_PROVIDE_NONSTANDARD_WB If set (default is not), and when applicable, color.cam_mul[] and color.WB_Coeffs/WBCT_Coeffs will contain WB settings for a non-standard workflow. Right now only Sony DSC-F828 is affected: camera-recorded white balance can't be directly applied to raw data because WB is for RGB, while raw data is RGBE.
New processing flag: LIBRAW_PROCESSING_CAMERAWB_FALLBACK_TO_DAYLIGHT If set (default is not), LibRaw::dcraw_process() will fallback to daylight WB (excluding some very specific cases like Canon D30). This is how LibRaw 0.19 (and older) works. If not set: LibRaw::dcraw_process() will fallback to calculated auto WB if camera WB is requested, but appropriate white balance was not found in metadata.
Google changes cherry-picked (thanks to Jamie Pinheiro)
speedup: ppg interpolate: const loop invariant
Bugs fixed -Fixed several UBs found by OSS Fuzz -Fixed several problems found by other fuzzers.
Thumbnail size range check (CVE-2020-15503) Thanks to Jennifer Gehrke of Recurity Labs GmbH for problem report.
fixed possible overflows in canon and sigma makernotes parsers
fixed possible buffer overrun in crx (cr3) decoder
fixed memory leak in crx decoder (if compiled with LIBRAW_NO_CR3_MEMPOOL)
fixed possible overrun in Sony SRF and SR2 metadata parsers