Can't do anything relate to calibration LG C1 OLED

[Zen3515]

2021-11-12 02:22:30 DEBUG (MainThread) [websockets.protocol] client > Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"id": 12, "type": "request", "uri": "ssap://externalpq/getExternalPqData", "payload": {"picMode": "hdr_game"}}', rsv1=False, rsv2=False, rsv3=False)
2021-11-12 02:22:30 DEBUG (MainThread) [websockets.protocol] client - event = data_received(<134 bytes>)
2021-11-12 02:22:30 DEBUG (MainThread) [websockets.protocol] client < Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"type":"error","id":12,"error":"500 Application error","payload":{"returnValue":false,"errorText":"no file path from tvservice"}}', rsv1=False, rsv2=False, rsv3=False)
2021-11-12 02:22:30 ERROR (MainThread) [homeassistant.components.webostv.media_player] Error calling async_command on entity media_player.my_webos_smart_tv_48c1: PyLGTVCmdError('500 Application error')

When using externalpq/getExternalPqData I got this error: "no file path from tvservice"

When using externalpq/setExternalPqData I got this error: "20 "Driver error while executing the command""

I just want to change OLED brightness. I try writing the function separately, but the error will occur as soon as I start calibration. I can't seem to figure it out.

BTW: for anyone who wants to use send command, the payload itself require a few more thing to complete for example

{"command": "BACKLIGHT_UI_DATA","picMode":"hdr_game"}

and perhaps

{"command": "BACKLIGHT_UI_DATA","picMode":"hdr_game", "data": "?WHATISIT", "dataCount": 30,"dataOpt": 1,"dataType":"float","profileNo": 0,"programID": 1}

But I still think that there's more to it.

[Zen3515]

The error originates from here:

rootfs.pak.unsquashfs/usr/palm/services/com.webos.service.apiadapter/adpters/externalpq/index.js>ExternalPqAdapter>onGetExternalPqData

            onGetExternalPqData: function(request) {
        var self = this;
        var payload = request.payload();

        return request.chainCall('luna://com.webos.service.pqcontroller/getExternalPqData', payload)
        .then(function(success) {
            var pqData;
            if (success.path) {
                try {
                    var buf = fs.readFileSync(success.path);
                } catch(e) {
                    logger.error('READ_EXTERNALPQ_FAILED', 'failed to read externalPQ', e);
                    return {returnValue: false, errorText: e};
                }
            } else {
                return {returnValue: false, errorText: 'no file path from tvservice'};
            }
            // Encode base64
            pqData = new Buffer(buf).toString('base64');

            // Remove temporary file made by tvservice
            try {
                if(fs.existsSync(success.path)) {
                    fs.unlinkSync(success.path);
                }
            } catch (e) {
                logger.error('REMOVE_EXTERNALPQ_DATA_FAILED', 'failed to remove externalPQ data from tvservice', e);
                return _.extend(success, {exception: e});
            }

            // RE-generate payload
            delete success.path;

            return _.extend(success, {data: pqData});
        }, function(error) {
            return _.extend(error);
        });
    },

[Zen3515]

I believe I've found the problem is that I don't know what is the data used in function start_calibration

And I have no idea how to know that numpy array of size 9 for my specific TV. I don't think the usage is in the firmware file, at least, not human readable.

I really wish somebody could help me figure out a way or where to look.

[bendavid]

For previous models as far as I can tell that data doesn't actually do anything (and you can provide different values with no effect.)

But it could indeed be that something has changed with the C1

[chros73]

When using externalpq/getExternalPqData I got this error: "no file path from tvservice"

This endpoint doesn't work on 2018 models either, only setExternalPqData.

I believe I've found the problem is that I don't know what is the data used in function start_calibration And I have no idea how to know that numpy array of size 9 for my specific TV. I don't think the usage is in the firmware file, at least, not human readable.

Yep, I haven't found anything in firmware either. The only way to know how to use these calibration commands with newer sets is to sniff the traffic of other working software solutions :)

I just want to change OLED brightness.

But if you only want to change picture settings/presets then you can do that already without calibration endpoint: with scripting (using set_current_picture_settings("{\"backlight\": 0, \"contrast\": 85}")). Or you can try my fork which allows you to do the same via command line (until Josh merges/migrate things back to aoipylgtv).

@bendavid , take a look at my fork (there are not so many commits), I implemented everything that I wanted. If you like the changes then just migrate them into yours and you can close the affected PRs (and I'll put a Depricated note into my readme :) )

[bendavid]

(I may have more time to look at this later in December/January )

[chros73]

No worries, take your time and thanks for all the work you've done so far!

[charuruflex]

I have a LG 48C1 firmware 03.20.80 and I'm trying to enter calibration mode but I'm getting this error:

Traceback (most recent call last):
  File "/mnt/c/Users/sharuru/test.py", line 14, in <module>
    asyncio.run(runloop())
  File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/mnt/c/Users/sharuru/test.py", line 8, in runloop
    await client.start_calibration(picMode="expert1")
  File "/home/sharuru/.local/lib/python3.9/site-packages/aiopylgtv/webos_client.py", line 1469, in start_calibration
    return await self.calibration_request(cal.CAL_START, picMode, data)
  File "/home/sharuru/.local/lib/python3.9/site-packages/aiopylgtv/webos_client.py", line 1465, in calibration_request
    return await self.request(ep.CALIBRATION, payload)
  File "/home/sharuru/.local/lib/python3.9/site-packages/aiopylgtv/webos_client.py", line 711, in request
    raise PyLGTVCmdError(response)
aiopylgtv.webos_client.PyLGTVCmdError: {'type': 'error', 'id': 12, 'error': '500 Application error', 'payload': {'returnValue': False, 'errorCode': 20, 'errorMessage': 'Driver error while executing the command'}}

My code:

import asyncio
from aiopylgtv import WebOsClient

async def runloop():
    client = await WebOsClient.create('10.42.0.109')
    await client.connect()

    await client.start_calibration(picMode="expert1")
    await client.set_oled_light(picMode="expert1", value=26)
    await client.end_calibration(picMode="expert1")

    await client.disconnect()

asyncio.run(runloop())

[Zen3515]

set_current_picture_settings

Sorry for late response, I just have time to test it thoroughly I try the fuction set_current_picture_settings but it doesn't do anything

async def _set_oled_light(ip_address: str, oled_backlight_value: int):
    try:
        client = await WebOsClient.create(ip_address)
        await client.disconnect()    
        await client.connect()
        test = await client.get_picture_settings()
        _LOGGER.debug('_set_oled_light -> before')
        _LOGGER.debug(test)
        # setting_value = f"""{{"contrast": 70, "backlight": {oled_backlight_value}, "brightness": 40, "color": "55"}}""" # , "contrast": 70
        setting_value = "{\"backlight\": 0, \"contrast\": 70}"
        _LOGGER.debug(f"setting_value\n{setting_value}")
        result = await client.set_current_picture_settings(setting_value)
        _LOGGER.debug('set_current_picture_settings -> result')
        _LOGGER.debug(result)
        test = await client.get_picture_settings()
        _LOGGER.debug('_set_oled_light -> after')
        _LOGGER.debug(test)
        await client.disconnect()
    except Exception as err:
        _LOGGER.error("An error occour during setting oled light\n")
        _LOGGER.error(err)
        raise err

And this is the log

2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client > Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"id": 12, "type": "request", "uri": "ssap://settings/getSystemSettings", "payload": {"category": "picture", "keys": ["contrast", "backlight", "brightness", "color"]}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client - event = data_received(<173 bytes>)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client < Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"type":"response","id":12,"payload":{"returnValue":true,"subscribed":false,"category":"picture","settings":{"contrast":70,"backlight":40,"brightness":40,"color":"55"}}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] _set_oled_light -> before
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] {'contrast': 70, 'backlight': 40, 'brightness': 40, 'color': '55'}
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] setting_value
{"backlight": 0, "contrast": 70}
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client > Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"id": 13, "type": "request", "uri": "ssap://system.notifications/createAlert", "payload": {"message": " ", "buttons": [{"label": "", "onClick": "luna://com.webos.settingsservice/setSystemSettings", "params": {"category": "picture", "settings": "{\\"backlight\\": 0, \\"contrast\\": 70}"}}], "onclose": {"uri": "luna://com.webos.settingsservice/setSystemSettings", "params": {"category": "picture", "settings": "{\\"backlight\\": 0, \\"contrast\\": 70}"}}, "onfail": {"uri": "luna://com.webos.settingsservice/setSystemSettings", "params": {"category": "picture", "settings": "{\\"backlight\\": 0, \\"contrast\\": 70}"}}}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client - event = data_received(<115 bytes>)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client < Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"type":"response","id":13,"payload":{"returnValue":true,"alertId":"com.webos.service.apiadapter-1638801047295"}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client > Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"id": 14, "type": "request", "uri": "ssap://system.notifications/closeAlert", "payload": {"alertId": "com.webos.service.apiadapter-1638801047295"}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client - event = data_received(<60 bytes>)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client < Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"type":"response","id":14,"payload":{"returnValue":true}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] set_current_picture_settings -> result
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] {'returnValue': True}
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client > Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"id": 15, "type": "request", "uri": "ssap://settings/getSystemSettings", "payload": {"category": "picture", "keys": ["contrast", "backlight", "brightness", "color"]}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client - event = data_received(<173 bytes>)
2021-12-06 21:30:48 DEBUG (MainThread) [websockets.protocol] client < Frame(fin=True, opcode=<Opcode.TEXT: 1>, data=b'{"type":"response","id":15,"payload":{"returnValue":true,"subscribed":false,"category":"picture","settings":{"contrast":70,"backlight":40,"brightness":40,"color":"55"}}}', rsv1=False, rsv2=False, rsv3=False)
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] _set_oled_light -> after
2021-12-06 21:30:48 DEBUG (MainThread) [custom_components.webostv_advance] {'contrast': 70, 'backlight': 40, 'brightness': 40, 'color': '55'}

Looks like C1 really has some change compared to the previous model. Maybe the hack at alert API doesn't work anymore ?

[chros73]

Maybe they removed it, but you set a string for setting_value that needs to be dictionary, try this:

client = await WebOsClient.create(ip_address)
await client.connect()
setting_value = {"backlight": "0"}
await client.set_current_picture_settings(setting_value)
await client.disconnect()

That command line tool auto converts JSON strings into dictionaries, hence the escaping there.

[Zen3515]

Maybe they removed it, but you set a string for setting_value that needs to be dictionary, try this:

client = await WebOsClient.create(ip_address)
await client.connect()
setting_value = {"backlight": "0"}
await client.set_current_picture_settings(setting_value)
await client.disconnect()

That command line tool auto converts JSON strings into dictionaries, hence the escaping there.

Thanks, just tested.

I can confirm that this hack still works. I thought I've already test it with dict, but, apparently not.

HERE IS THE FINAL CODE

async def _set_oled_light(ip_address: str, oled_backlight_value: int):
    try:
        client = await WebOsClient.create(ip_address)
        await client.disconnect()    
        await client.connect()
        setting_value = {"backlight": oled_backlight_value}
        await client.set_current_picture_settings(setting_value)
        await client.disconnect()
    except Exception as err:
        raise err
    finally:
        try:
            await client.disconnect()
        except Exception as err:
            pass

I've achieved my goal, but the issue with calibration still remains open.

[chros73]

Nice to hear that at least the hack works :)

[chros73]

{'returnValue': False, 'errorCode': 20, 'errorMessage': 'Driver error while executing the command'}

I only get this error when a preset name is misspelled, eg hdrCinema instead of hdr_cinema.

[24fpsDaVinci]

since it may not be clear to some LG C1 users, you have to manually merge the "fixed parsing model names" patch for calibration to work with C1 and higher(?) models, the patch is also missing on chros73's fork.

[chros73]

since it may not be clear to some LG C1 users, you have to manually merge the "fixed parsing model names" patch for calibration to work with C1 and higher(?) models, the patch is also missing on chros73's fork.

Yes, but it's in progress :)

[24fpsDaVinci]

@chros73 this is a bit off topic, i wasnt able to find a final conclusion to the c8 thread on sdr calibration. Is using factory 1d lut + 100 contrast still the recommended way to calibrate SDR? In particular this message by ben sounds like unity 1d lut should be used.

The (1024 element) 1D LUT maps an input of 0 onto the first element, and an input of 4092 onto the last element. The current definition of unity LUTs in aiopylgtv will therefore map a 0-4092 input to the 3D LUT perfectly onto the first and last entries of the 1D LUT, and I think this part is ok.

and when using factory 1d lut, what is the correct option in terms of these:

graphics card: full range, TV black level = high, Contrast = 85 -> 255 patch measures 104 cd
graphics card: full range, TV black level = high, Contrast = 100 -> 255 patch measures 131 cd

graphics card: full range, TV black level = low, Contrast = 85 -> 255 patch measures 131 cd
graphics card: full range, TV black level = low, Contrast = 100 -> 255 patch measures 131 cd (same as above)

[chros73]

I'm not sure either :) Everyone out there (maybe apart from Calman users) use the factory 1dlut and create a 3dlut on top of it outside of calibration mode (after ddc_reset (I'll call it bypass_modes) is set without unity 1dlut). Whether the 100 or 85 contrast is the right one, again a different topic that I don't know the answer to, I just go for 85. :) But does it really matter with SDR anyway? :)

[chros73]

New bscpylgtv v0.4.0 is out, it's mostly about calibration changes, lot of new stuff has been added. Enjoy!

[24fpsDaVinci]

very nice! is the ITPG functions useful for calibration at this time?

[chros73]

Which model do you have? Eg it's almost completely unusable with 2022 models. The other issue is that until someone integrates it into HCFR or something like that, it can only be used for manual measurements, take a look at the linked script in the readme. The gradients part is fun though :)

[24fpsDaVinci]

Which model do you have?

very interesting. i have the 2021 model. I have SDR down, trying to do HDR and DV correctly.

[chros73]

Good question :) If it works fine there then it can be used for even manual measurements for those 2 modes, since only couple of measurements need to be taken. Take a look at the iTPG part of our 2022 buglist (2 links): https://www.avsforum.com/threads/2022-lg-c2-g2-owners-thread-no-price-talk.3250717/page-69#post-61675612

[24fpsDaVinci]

@chros73 do you have any insights on the correct input/output encoding setting when generating 3d lut for SDR? I've been using 8bit full(0-255) hdmi output for calibration and full(0-255) input/output encoding for 3dlut. Recently i switched to limited (16-235) encoding for 3dlut and ycbcr 4:4:4 for calibration, the two result looks very different, the limited looks more "correct" to me.

[chros73]

Which TPG do you use? I only use madvr and madtpg for sdr profiling, and use the created 3dlut in madvr itself, let alone I use PC mode and full range 12 bit output from pc, so not an expert in internal luts of LG.

Other guys stated, who use their set in normal mode (non-PC) and limited range, that internal luts operate on: 2019 models in full range newer (>=2020) models in limited range

If you found out something worthy, you can do a PR to the readme, e.g. Tips and tricks? :)

[24fpsDaVinci]

Currently i use displaycal/argylltpg, i will try madtpg next.

Other guys stated, who use their set in normal mode (non-PC) and limited range, that internal luts operate on: 2019 models in full range newer (>=2020) models in limited range

very interesting. I will do more testing. I am trying to achieve this result from BenDavid in the LG C8 lut thread https://hub.displaycal.net/forums/topic/lg-c8-lut/page/10/#post-21726

[24fpsDaVinci]

@chros73 over the holidays i had some time to attempt HDR calibration with madTPG and displaycal. I got poor results using 1d factory lut + custom 3D lut following some of the tips provided in the LG C8 thread. Some of the things i tried: -use madtpg to put windows into hdr mode to skip OS hdr mode -calibrate to 2.2, generate 3D lut 2.2 + bt2020 -set target white to 700nits in displaycal gui

i have not tested synthetic profiles.

So at least from first principles, one should try to do an XYZLUT profile in calibration mode, and generate a 3d lut for BT.2020 gamma 2.2 with e.g. “absolute colorimetric with whitepoint scaling” for the rendering intent.

this is about where bendavid left off in the c8 thread. was there any new progress with HDR calibration? There was also the issue of nvidia driver outputting tone mapping that couldnt be turned off that bendavid mentioned, I wasnt able to find any new info on that. I also looked in the DWM lut thread and it looks like people gave up on HDR calibration there.

another issue i came across, C1 hdmi diagnostics shows 8bit when set to ycbcr422 10 bit in NVCP. Oddly enough, diagnostics shows it will accept ycbcr444 12bit signals. I cant seem to put C1 into 10bit no matter what encoding/output combination i choose in NVCP.

from the nvidia documentation i see this, dont know if it affects accuracy

10bpc over HDMI can be selected on NVIDIA Ampere GPUs. Older NVIDIA GPUs do not support 10bpc over HDMI however you can use 12 bpc to enable 30-bit color.

something interesting i came across in the C1 edid, the last block has dolby primaries(native gamut primaries(?)) and target max PQ both appear to be factory measurements unique to the panel. im assuming these are the values in the factory dolby config.

Vendor-Specific Video Data Block (Dolby), OUI 00-D0-46: Version: 2 (12 bytes) DM Version: 4.x Backlt Min Luma: 100 cd/m^2 Interface: Standard + Low-Latency Supports 10b 12b 444: Not supported Target Min PQ v2: 0 (0.00000000 cd/m^2) Target Max PQ v2: 2965 (807 cd/m^2) Unique Rx, Ry: 0.67578125, 0.32031250 Unique Gx, Gy: 0.25390625, 0.68359375 Unique Bx, By: 0.14453125, 0.05078125

[chros73]

About DV config in Edid: that's for LLDV external devices, the internal one may or may not be the same (they can update it via firmware update).

About nvidia driver: I use Full range, RGB, 12 bit output, and that what appears on the Diag screen as well.

About HDR10 calibration: I haven't heard anyone who successfully profiled a Woled and created an acceptable 3dlut from it in HDR. Even Calman only measures WRGB and generates a matrix 3dlut from.it, hencr the talk about synthetic luts in that thread. But other folks don't even touch the 3dlut: they only create a 1dlut or adjust the 22pt WB RGB controls in calibration mode, but even the latter is highly questionable due to how instable Woled is in HDR. With the G2, I only adjusted the 22pt WB RGB values to slightly correct WB (so only chroma adjustments, not even trying to correct gamma) and I haven't even upload any tonemapping params (or I uploaded the default 990 one with hardclip params in 1 preset), but this result is barely visible when switching between thia and an untouched preset). Read through the avs 2022 calibration thread if you are in the mood for more info: https://www.avsforum.com/threads/2022-lg-oled-calibration-and-user-settings-no-price-talk.3242257/page-45

[24fpsDaVinci]

@chros73 do you think its theoretically possible for argyllcms to use bscpylgtv iTPG in the same way as madTPG?

[bendavid]

argyllcms supports using an external command line tool to set the patches IIRC, so this might already be possible with the right command line options

[chros73]

woow, welcome back @bendavid ! :)

[24fpsDaVinci]

my steps for hdr:
1. use madTPG put into hdr mode
2. calibrated to g2.2 absolute D65 
3. upload gamma .cal file with bscpylgtv
4. manually measure xy coordinates of WRGB patches using spotread and dispwin 
5. using step 4 xy coordinates and displaycal gui generate a synthetic .icc (is brightness tag needed?)
6. using argyll collink,  generate a 33x33 .cube using the included reference rec2020.icc and synthetic .icc from step 5
7. upload .cube and set tonemapping parameters using bscpylgtv

the xy coordinates in step 5 can be used for dolbyvision config

I think this is close to what calman is doing for hdr