search

Beep6581 / RawTherapee

A powerful cross-platform raw photo processing program
https://rawtherapee.com
GNU General Public License v3.0
2.8k stars 316 forks source link

ARQ support not working for A7CR camera files #7197

Open AKADAP opened 4 weeks ago

AKADAP commented 4 weeks ago

Steps to reproduce Use the A7CR to create a pixel shift image of 16 frames Convert the 16 RAW files to a ARQ file using make_arq import arq file into Rawtherapee Result: color is wrong, only a portion of the image appears and is skewed. import arq file into ART which has the .config/ART/camconst.json file added results in a correct image.

See discussion here: https://discuss.pixls.us/t/trying-to-figure-out-how-to-use-pixel-shift-from-a7cr/43080/39 where you can find a link to example files.

Expected behavior I expect the loaded photo to be the correct color, full image, and not skewed. I expect the same results ART is achieving.

Additional information I delayed reporting this because I was unsure if the bug was in the make_arq file, or Rawtherapee (I do not have access to a windows or mac machine so I could not test creating the arq file using Sony's imaging edge software). At the time ART was giving different but still wrong results. Since then a bug was found and fixed in ART which proved that the make_arq script was working properly, and the bug had to be in Rawtherapee.

Rawtherapee version Version: 5.11 Branch: 5.11 Commit: db575c669 Commit date: 2024-08-24 Compiler: cc 9.4.0 Processor: generic x86 System: Linux Bit depth: 64 bits Gtkmm: 3.24.2 Lensfun: 0.3.2.0 libjxl: 0.10.3 Build type: release Build flags: -std=c++11 -mtune=generic -Werror=unused-label -Werror=delete-incomplete -fno-math-errno -Wno-attributes -Wall -Wuninitialized -Wcast-qual -Wno-deprecated-declarations -Wno-unused-result -Wunused-macros -fopenmp -Werror=unknown-pragmas -O3 -DNDEBUG -ftree-vectorize Link flags: -mtune=generic OpenMP support: ON MMAP support: ON Build OS: Linux 5.15.0-1070-azure x86_64 Build date: Sat, 24 Aug 2024 18:44:17 +0000 UTC Build epoch: 1724525057 Build UUID: 5e189ea9-cc93-4f4f-acc4-16df294715fb

System Info System: Kernel: 6.8.0-40-generic x86_64 bits: 64 compiler: N/A Desktop: Cinnamon 6.0.4 tk: GTK 3.24.33 wm: muffin vt: 7 dm: LightDM 1.30.0 Distro: Linux Mint 21.3 Virginia base: Ubuntu 22.04 jammy Machine: Type: Desktop Mobo: ASRock model: X570 Creator serial: UEFI: American Megatrends v: P5.61 date: 02/23/2024 Battery: ID-1: hidpp_battery_0 charge: 88% condition: N/A volts: 4.1 min: N/A model: Logitech G903 Wired/Wireless Gaming Mouse type: N/A serial: status: Discharging CPU: Info: 12-core model: AMD Ryzen 9 3900X bits: 64 type: MT MCP smt: enabled arch: Zen 2 rev: 0 cache: L1: 768 KiB L2: 6 MiB L3: 64 MiB Speed (MHz): avg: 2356 high: 4467 min/max: 2200/4672 boost: enabled cores: 1: 2200 2: 1985 3: 4115 4: 2196 5: 2196 6: 2200 7: 2200 8: 2190 9: 2200 10: 2200 11: 2194 12: 2196 13: 2200 14: 2049 15: 4467 16: 2196 17: 2196 18: 2195 19: 2200 20: 2200 21: 2195 22: 2195 23: 2196 24: 2200 bogomips: 182058 Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm Graphics: Device-1: NVIDIA GM200 [GeForce GTX 980 Ti] vendor: Micro-Star MSI driver: nvidia v: 535.183.01 pcie: speed: 2.5 GT/s lanes: 16 ports: active: none off: DP-3,HDMI-A-1 empty: DP-1,DP-2,DVI-I-1 bus-ID: 33:00.0 chip-ID: 10de:17c8 class-ID: 0300 Device-2: Logitech BRIO Ultra HD Webcam type: USB driver: hid-generic,snd-usb-audio,usbhid,uvcvideo bus-ID: 8-3:2 chip-ID: 046d:085e class-ID: 0300 serial: Display: x11 server: X.Org v: 1.21.1.4 driver: X: loaded: nvidia unloaded: fbdev,modesetting,nouveau,vesa gpu: nvidia display-ID: :0 screens: 1 Screen-1: 0 s-res: 6400x2160 s-dpi: 100 s-size: 1626x554mm (64.0x21.8") s-diag: 1718mm (67.6") Monitor-1: DP-4 pos: bottom-r res: 2560x1600 hz: 60 dpi: 101 size: 641x400mm (25.2x15.7") diag: 756mm (29.7") Monitor-2: HDMI-0 pos: primary,top-left res: 3840x2160 dpi: 68 size: 1440x810mm (56.7x31.9") diag: 1652mm (65") OpenGL: renderer: NVIDIA GeForce GTX 980 Ti/PCIe/SSE2 v: 4.6.0 NVIDIA 535.183.01 direct render: Yes Audio: Device-1: NVIDIA GM200 High Definition Audio vendor: Micro-Star MSI driver: snd_hda_intel v: kernel pcie: speed: 8 GT/s lanes: 16 bus-ID: 33:00.1 chip-ID: 10de:0fb0 class-ID: 0403 Device-2: AMD Starship/Matisse HD Audio vendor: ASRock driver: snd_hda_intel v: kernel pcie: speed: 16 GT/s lanes: 16 bus-ID: 35:00.4 chip-ID: 1022:1487 class-ID: 0403 Device-3: C-Media KT01 type: USB driver: hid-generic,snd-usb-audio,usbhid bus-ID: 7-4:4 chip-ID: 0d8c:0134 class-ID: 0300 Device-4: Logitech BRIO Ultra HD Webcam type: USB driver: hid-generic,snd-usb-audio,usbhid,uvcvideo bus-ID: 8-3:2 chip-ID: 046d:085e class-ID: 0300 serial: Sound Server-1: ALSA v: k6.8.0-40-generic running: yes Sound Server-2: PulseAudio v: 15.99.1 running: yes Sound Server-3: PipeWire v: 0.3.48 running: yes Network: Device-1: Intel I211 Gigabit Network vendor: ASRock driver: igb v: kernel pcie: speed: 2.5 GT/s lanes: 1 port: b000 bus-ID: 28:00.0 chip-ID: 8086:1539 class-ID: 0200 IF: enp40s0 state: up speed: 1000 Mbps duplex: full mac: Device-2: Intel Wi-Fi 6 AX200 driver: iwlwifi v: kernel pcie: speed: 5 GT/s lanes: 1 bus-ID: 29:00.0 chip-ID: 8086:2723 class-ID: 0280 IF: wlp41s0 state: down mac: Device-3: Aquantia AQC107 NBase-T/IEEE 802.3bz Ethernet [AQtion] vendor: ASRock driver: atlantic v: kernel pcie: speed: 8 GT/s lanes: 2 port: N/A bus-ID: 2e:00.0 chip-ID: 1d6a:07b1 class-ID: 0200 IF: enp46s0 state: down mac: Bluetooth: Device-1: Intel AX200 Bluetooth type: USB driver: btusb v: 0.8 bus-ID: 5-6:2 chip-ID: 8087:0029 class-ID: e001 Report: hciconfig ID: hci0 rfk-id: 0 state: down bt-service: enabled,running rfk-block: hardware: no software: yes address: RAID: Supported mdraid levels: raid1 raid0 raid6 raid5 raid4 raid10 Device-1: md1 type: mdraid level: mirror status: active size: 9.1 TiB Info: report: 2/2 UU blocks: 9766304768 chunk-size: N/A super-blocks: 1.2 Components: Online: 0: sdb 1: sda Device-2: bpool type: zfs status: ONLINE level: mirror-0 raw: size: 1.88 GiB free: 1.38 GiB allocated: 502 MiB zfs-fs: size: 1.75 GiB free: 1.26 GiB Components: Online: N/A Device-3: rpool type: zfs status: ONLINE level: mirror-0 raw: size: 1.81 TiB free: 1.32 TiB allocated: 503 GiB zfs-fs: size: 1.76 TiB free: 1.26 TiB Components: Online: N/A Drives: Local Storage: total: raw: 23.19 TiB usable: 15.86 TiB used: 4.68 TiB (29.5%) ID-1: /dev/nvme0n1 vendor: Gigabyte model: GP-ASM2NE6200TTTD size: 1.82 TiB speed: 63.2 Gb/s lanes: 4 type: SSD serial: rev: EGFM11.2 temp: 52.9 C scheme: GPT ID-2: /dev/nvme1n1 vendor: Gigabyte model: GP-ASM2NE6200TTTD size: 1.82 TiB speed: 63.2 Gb/s lanes: 4 type: SSD serial: rev: EGFM11.2 temp: 51.9 C scheme: GPT ID-3: /dev/sda vendor: Western Digital model: WD100EFAX-68LHPN0 size: 9.1 TiB speed: 6.0 Gb/s type: HDD rpm: 5400 serial: rev: 0A83 ID-4: /dev/sdb vendor: Western Digital model: WD100EFAX-68LHPN0 size: 9.1 TiB speed: 6.0 Gb/s type: HDD rpm: 5400 serial: rev: 0A83 ID-5: /dev/sdc vendor: Western Digital model: WD15EADS-00P8B0 size: 1.36 TiB speed: 3.0 Gb/s type: N/A serial: rev: 0A01 scheme: MBR Partition: ID-1: / size: 1.27 TiB used: 10.17 GiB (0.8%) fs: zfs logical: rpool/ROOT/ubuntu_f2mgxr ID-2: /boot size: 1.75 GiB used: 500.6 MiB (28.0%) fs: zfs logical: bpool/BOOT/ubuntu_f2mgxr ID-3: /boot/efi size: 511 MiB used: 14.3 MiB (2.8%) fs: vfat dev: /dev/nvme0n1p1 ID-4: /var/log size: 1.27 TiB used: 1.09 GiB (0.1%) fs: zfs logical: rpool/ROOT/ubuntu_f2mgxr/var/log Swap: ID-1: swap-1 type: partition size: 2 GiB used: 0 KiB (0.0%) priority: -2 dev: /dev/nvme1n1p2 USB: Hub-1: 1-0:1 info: Hi-speed hub with single TT ports: 2 rev: 2.0 speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900 Hub-2: 2-0:1 info: Super-speed hub ports: 2 rev: 3.1 speed: 10 Gb/s chip-ID: 1d6b:0003 class-ID: 0900 Hub-3: 2-1:2 info: Genesys Logic USB3.2 Hub ports: 4 rev: 3.2 speed: 10 Gb/s chip-ID: 05e3:0625 class-ID: 0900 Hub-4: 2-1.3:3 info: Genesys Logic USB3.2 Hub ports: 4 rev: 3.2 speed: 10 Gb/s chip-ID: 05e3:0625 class-ID: 0900 Device-1: 2-1.3.1:4 info: Genesys Logic USB Storage type: Mass Storage driver: usb-storage interfaces: 1 rev: 3.2 speed: 5 Gb/s power: 896mA chip-ID: 05e3:0764 class-ID: 0806 serial: Hub-5: 3-0:1 info: Hi-speed hub with single TT ports: 6 rev: 2.0 speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900 Hub-6: 3-1:2 info: Microchip (formerly SMSC) USB 2.0 Hub ports: 4 rev: 2.0 speed: 480 Mb/s power: 2mA chip-ID: 0424:2514 class-ID: 0900 Hub-7: 3-1.1:4 info: Microchip (formerly SMSC) USB 2.0 Hub ports: 4 rev: 2.0 speed: 480 Mb/s power: 2mA chip-ID: 0424:2602 class-ID: 0900 Device-1: 3-1.1.1:7 info: Microchip (formerly SMSC) 9-in-2 Card Reader type: Mass Storage driver: usb-storage interfaces: 1 rev: 2.0 speed: 480 Mb/s power: 100mA chip-ID: 0424:2228 class-ID: 0806 serial: Hub-8: 3-2:3 info: Hitachi ports: 4 rev: 2.1 speed: 480 Mb/s chip-ID: 045b:0209 class-ID: 0900 Hub-9: 3-2.1:6 info: Hitachi ports: 4 rev: 2.1 speed: 480 Mb/s chip-ID: 045b:0209 class-ID: 0900 Device-1: 3-2.4:8 info: U-Blox AG [u-blox 7] type: Abstract (modem),CDC-Data driver: cdc_acm interfaces: 2 rev: 1.1 speed: 12 Mb/s power: 100mA chip-ID: 1546:01a7 class-ID: 0a00 Hub-10: 3-5:5 info: Genesys Logic Hub ports: 4 rev: 2.1 speed: 480 Mb/s chip-ID: 05e3:0610 class-ID: 0900 Hub-11: 3-5.3:9 info: Genesys Logic Hub ports: 4 rev: 2.1 speed: 480 Mb/s chip-ID: 05e3:0610 class-ID: 0900 Hub-12: 4-0:1 info: Super-speed hub ports: 4 rev: 3.1 speed: 10 Gb/s chip-ID: 1d6b:0003 class-ID: 0900 Hub-13: 4-4:2 info: Hitachi ports: 4 rev: 3.0 speed: 5 Gb/s chip-ID: 045b:0210 class-ID: 0900 Hub-14: 4-4.1:3 info: Hitachi ports: 4 rev: 3.0 speed: 5 Gb/s chip-ID: 045b:0210 class-ID: 0900 Hub-15: 5-0:1 info: Hi-speed hub with single TT ports: 6 rev: 2.0 speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900 Device-1: 5-6:2 info: Intel AX200 Bluetooth type: Bluetooth driver: btusb interfaces: 2 rev: 2.0 speed: 12 Mb/s power: 100mA chip-ID: 8087:0029 class-ID: e001 Hub-16: 6-0:1 info: Super-speed hub ports: 4 rev: 3.1 speed: 10 Gb/s chip-ID: 1d6b:0003 class-ID: 0900 Hub-17: 7-0:1 info: Hi-speed hub with single TT ports: 4 rev: 2.0 speed: 480 Mb/s chip-ID: 1d6b:0002 class-ID: 0900 Device-1: 7-1:2 info: Logitech PowerPlay Wireless Charging System type: Keyboard,Mouse,HID driver: logitech-djreceiver,usbhid interfaces: 3 rev: 2.0 speed: 12 Mb/s power: 498mA chip-ID: 046d:c53a class-ID: 0300 Device-2: 7-2:3 info: Holtek USB Keyboard type: Keyboard,Mouse driver: hid-generic,usbhid interfaces: 2 rev: 1.1 speed: 1.5 Mb/s power: 100mA chip-ID: 04d9:0169 class-ID: 0301 Device-3: 7-4:4 info: C-Media KT01 type: Audio,HID driver: hid-generic,snd-usb-audio,usbhid interfaces: 3 rev: 1.1 speed: 12 Mb/s power: 100mA chip-ID: 0d8c:0134 class-ID: 0300 Hub-18: 8-0:1 info: Super-speed hub ports: 4 rev: 3.1 speed: 10 Gb/s chip-ID: 1d6b:0003 class-ID: 0900 Device-1: 8-3:2 info: Logitech BRIO Ultra HD Webcam type: Video,Audio,HID driver: hid-generic,snd-usb-audio,usbhid,uvcvideo interfaces: 6 rev: 3.1 speed: 5 Gb/s power: 896mA chip-ID: 046d:085e class-ID: 0300 serial: Sensors: System Temperatures: cpu: N/A mobo: N/A gpu: nvidia temp: 57 C Fan Speeds (RPM): N/A gpu: nvidia fan: 31% Repos: Packages: 3448 apt: 3415 flatpak: 33 No active apt repos in: /etc/apt/sources.list Active apt repos in: /etc/apt/sources.list.d/google.list 1: deb [arch=amd64] http: //dl.google.com/linux/earth/deb/ stable main Active apt repos in: /etc/apt/sources.list.d/kdenlive-kdenlive-stable-jammy.list 1: deb [arch=amd64 signed-by=/etc/apt/keyrings/kdenlive-kdenlive-stable-jammy.gpg] https: //ppa.launchpadcontent.net/kdenlive/kdenlive-stable/ubuntu jammy main Active apt repos in: /etc/apt/sources.list.d/mythbuntu-33-jammy.list 1: deb [arch=amd64 signed-by=/etc/apt/keyrings/mythbuntu-33-jammy.gpg] https: //ppa.launchpadcontent.net/mythbuntu/33/ubuntu jammy main Active apt repos in: /etc/apt/sources.list.d/official-package-repositories.list 1: deb https: //mirror.fcix.net/linuxmint-packages virginia main upstream import backport 2: deb http: //mirror.us-sc.kamatera.com/ubuntu jammy main restricted universe multiverse 3: deb http: //mirror.us-sc.kamatera.com/ubuntu jammy-updates main restricted universe multiverse 4: deb http: //mirror.us-sc.kamatera.com/ubuntu jammy-backports main restricted universe multiverse 5: deb http: //security.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse Active apt repos in: /etc/apt/sources.list.d/skype-stable.list 1: deb [arch=amd64] https: //repo.skype.com/deb stable main Active apt repos in: /etc/apt/sources.list.d/ubuntuhandbook1-audacity-jammy.list 1: deb [arch=amd64 signed-by=/etc/apt/keyrings/ubuntuhandbook1-audacity-jammy.gpg] https: //ppa.launchpadcontent.net/ubuntuhandbook1/audacity/ubuntu jammy main Info: Processes: 786 Uptime: 2h 18m wakeups: 6 Memory: 62.71 GiB used: 10.39 GiB (16.6%) Init: systemd v: 249 runlevel: 5 Compilers: gcc: 11.4.0 alt: 11/12 Client: Cinnamon v: 6.0.4 inxi: 3.3.13

Lawrence37 commented 3 days ago

This patch adds support for ILCE-7CR pixelshift, but I can't test the raw crop properly because I don't have enough RAM :frowning_face:.

diff --git a/rtengine/camconst.json b/rtengine/camconst.json
index 54bc65c23..15d36f64e 100644
--- a/rtengine/camconst.json
+++ b/rtengine/camconst.json
@@ -3140,7 +3140,10 @@ Camera constants:
     { // Quality B
         "make_model": [ "Sony ILCE-7CR", "Sony ILCE-7RM5" ], // 7RM5 is assumed to have the same sensor as the 7CR.
         "dcraw_matrix": [ 8200, -2976, -719, -4296, 12053, 2532, -429, 1282, 5774 ], // DNG v15.2 for ILCE-7CR and ILCE-7RM5.
-        "raw_crop": [ 0, 0, 9564, 6374 ], // From ART
+        "raw_crop": [
+            { "frame" : [19200, 12752], "crop" : [ 4, 3, 19128, 12749 ] },
+            { "frame" : [0, 0], "crop" : [ 0, 0, 9564, 6374 ] }
+        ], // Adapted from ART
         "pdaf_pattern": [ 0,12,18,36,42,60,66,72,78,96,108,120,126,138,156,168,180,186,192,198,210,222,228,240,246,252,270,276,282,288,306,312,318,330,336,348,360,366,372,378,390,396,408,420 ], // From issue #6938. Slightly different every repetition, maybe the real pattern is 3 or more multiples of 420 pixels. This is a composite.
         "pdaf_offset": 1
     },
diff --git a/rtengine/dcraw.cc b/rtengine/dcraw.cc
index e495a6105..0b167141b 100644
--- a/rtengine/dcraw.cc
+++ b/rtengine/dcraw.cc
@@ -7234,7 +7234,7 @@ void CLASS apply_tiff()
                     load_raw = &CLASS olympus_load_raw;
                    // ------- RT -------
                    if (!strncmp(make,"SONY",4) &&
-                       (!strncmp(model,"ILCE-7RM3",9) || !strncmp(model,"ILCE-7RM4",9)  || !strncmp(model,"ILCE-1",6)) &&
+                       (!strncmp(model,"ILCE-7RM3",9) || !strncmp(model,"ILCE-7RM4",9)  || !strncmp(model,"ILCE-1",6) || !strncmp(model,"ILCE-7CR",8)) &&
                        tiff_samples == 4 &&
                        tiff_ifd[raw].bytes == raw_width*raw_height*tiff_samples*2) {
                        load_raw = &CLASS sony_arq_load_raw;
Lawrence37 commented 1 day ago

I created pull request #7215 to add support for make_arq pixel shift files. I still cannot test the raw crop due to low memory. My swap settings need some tinkering...