get introspection in lua

[aster94]

hello,

I was willing to write a lua script that changes some values (like exposure) in a bunch of pictures, I didn't found any.

I thougth to start writing a "tool" to do this, it is far from ready but i think it is a good starting point if someone would like to collaborate

I never used lua before so you may find some weird syntax, you can find the code here https://repl.it/@VincenzoG/Darktable-xmp

example code:

require "tools/xmp_utility"

input = "0000000010c7babce0ce773e00004842000080c0"

exposure_params = {mode = 1, black = 2, exposure = 3, deflicker_percentile = 4, deflicker_target_level = 5}
exposure_table = dt_xmp_decode(input)

print(table.concat(exposure_table, " "))
print("Mode:", exposure_table[exposure_params.mode])
print("black:", exposure_table[exposure_params.black])
print("exposure:", exposure_table[exposure_params.exposure])
print("deflicker_percentile:", exposure_table[exposure_params.deflicker_percentile])
print("deflicker_target_level:", exposure_table[exposure_params.deflicker_target_level])

print(dt_xmp_encode(exposure_table))

exposure_table[exposure_params.exposure] = exposure_table[exposure_params.exposure] - 0.02
print("new exposure:", exposure_table[exposure_params.exposure])
print(dt_xmp_encode(exposure_table))

and it's output:

0 -0.022799998521805 0.24200010299683 50 -4
Mode:   0
black:  -0.022799998521805
exposure:   0.24200010299683
deflicker_percentile:   50
deflicker_target_level: -4
0000000010c7babce0ce773e00004842000080c0
new exposure:   0.22200010299683
0000000010c7babcfe53633e00004842000080c0

todo list:

• use compressed values: i found on google some part of code but i didn't integrated them, yet, you could find them in the base64.lua file
• automatically read/write the params: this can be done knowing the file path and using one of the already existing xml library for lua

If someone would like to collaborate tell me!

[wpferguson]

I think the idea of a library to read and write the xmp files is a good idea. Modifying the xmp files to change the development of images sounds interesting, but could have bad consequences.

Does this have anything to do with the discussions of handling timelapses in darktable by modifying the xmp files?

[aster94]

Yes, it is related, but I decided to keep the timelapse stuff apart. In my idea it would be better to have a dt_xmp_utility (or any other name) that can read and write xmp files and could be used by different files (like dtutils)

The timelapse Lua script would come in a second moment and would rely on this script. Other script (only imagination is the limit) could use this xmp writer/reader

[wpferguson]

I thought it might be related given the example.

I agree that an xmp file read/write script could/would be useful. It probably should be a library, such as lib/dutils/xmp.lua, then anyone who wanted to use it in their script could just local dx = require "dtutils/xmp"

[aster94]

okkk, before continuing i need some explanation about the xmp files, please correct me if you see any error

It looks like that the values we need to read are inside darktable:history and then into rdf:li. From any of these field we will need at least three values: operation, enabled and params. But i don't understand what happens when there are two operation of the same type, for example see this part of a xmp file:

      <rdf:li
      darktable:operation="exposure"
      darktable:enabled="1"
      darktable:modversion="5"
      darktable:params="0000000010c7babce0ce773e00004842000080c0"
      darktable:multi_name=""
      darktable:multi_priority="0"
      darktable:blendop_version="8"
      darktable:blendop_params="gz11eJxjYGBgkGAAgRNODGiAEV0AJ2iwh+CRxQcA5qIZBA=="/>
     <rdf:li
      darktable:operation="exposure"
      darktable:enabled="1"
      darktable:modversion="5"
      darktable:params="000000007a362bbde0ce773e00004842000080c0"
      darktable:multi_name=""
      darktable:multi_priority="0"
      darktable:blendop_version="8"
      darktable:blendop_params="gz11eJxjYGBgkGAAgRNODGiAEV0AJ2iwh+CRxQcA5qIZBA=="/>

here there are two exposure, one after the other, both are enabled. Which one is active? If only one of these is active we should write the library in a way that returns only the active one of all of them?

where i can find information about the compressed values?

[aster94]

done, I have a working code 🎉🎉🎉🎉🎉🎉🎉

I would love to receive some suggestion from @wpferguson or anyone who is interested in this. Here there is an example script and two images to show that it is working

local xmp_utility = require "tools.xmp_utility"
local inspect = require "tools.packages.inspect" -- only for debug purpose

-- the above three lines would be replaced by darktable functions to get the path of an image
cwd = io.popen("cd"):read "*all":gsub("\n", ""):gsub("\\", "/")
img_path = cwd .. "/images/IMG_1208.jpg"
out_path = cwd .. "/images/IMG_1208_out.jpg"

-- for now the code is very dirty, i would improve when everything would work as expected
start_time = os.clock()

-- dt_xmp_extract takes the image path and read the xml file associated
-- returns a table of all the edits done in darktable
params = dt_xmp_extract(img_path)
print(inspect(params))

-- dt_*_decode reads the result of dt_xmp_extract
-- and returns a table with all the settings in a human readable way
-- note: right now '*' can be only 'exposure', i need to do the others
exposure = dt_exposure_decode(params)
print(inspect(exposure))

-- we manually change the exposure
exposure.exposure = exposure.exposure - 0.9
print(exposure.exposure)

-- dt_*_encode write back the parameter
-- note: right now '*' can be only 'exposure', i need to do the others
params = dt_exposure_encode(exposure, params)
print(inspect(params))

-- dt_xmp_inject write back to a xmp file
dt_xmp_inject(out_path, params)
-- open darktable and see the result :)

print("Elapsed time:", os.clock() - start_time)

Here the original picture

and after the script run this is the result

There is a lot of room for improvements (I am new to lua), right now i can run the above script (without print) in 0.277 seconds

In the zip file there is the lua script that does everything xmp_utility.lua inside the folder packages there are the modules required by the script that actually should be installed with luarockos but i had some troubble with it so i just used the sources

xmp.zip

[houz]

I agree that an xmp file read/write script could/would be useful. It probably should be a library, such as lib/dutils/xmp.lua, then anyone who wanted to use it in their script could just local dx = require "dtutils/xmp"

Actually, the proper way to do this is to teach dt's Lua API to use iop introspection. Anything else is a hack.

[supertobi]

@houz You are absolutely right, but that is something I'm waiting for since 2014. So until someone implements an API to use iop introspection we have to use a hack.

[aster94]

I gave an example that this can be achieved in lua but honestly, if it won't be merged I don't feel the need to continue and waste others hours of coding