UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 0: invalid continuation byte

[Daemon1992]

This is a bug report. to get help go to https://github.com/FreeOpcUa/python-opcua/discussions

Describe the bug The server program is developed using Open62541.The client program is developed using python-opcua. Sometimes the client can retrieve the data correctly, but it will report an error later, or it will report an error as soon as the client fetches the data.

Here is an error message: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 0: invalid continuation byte

To Reproduce Steps to reproduce the behavior incl code:

Expected behavior the client can fetch data.

Screenshots If applicable, add screenshots to help explain your problem.

[AndreasHeine]

did you found out what request causes it? is it a reponse to a read/write/subscribe/methodcall/... request? what whas requested maybe a custom type or structure or something else like a simple variable?

maybe use wireshark to figgure that out!

[Daemon1992]

did you found out what request causes it? is it a reponse to a read/write/subscribe/methodcall/... request? what whas requested maybe a custom type or structure or something else like a simple variable?

maybe use wireshark to figgure that out! I use other clients, such as UAExpert, that get the data normally and without errors.

[Daemon1992]

I create an array of strings on the server side, and then the client gets the array data, subscribes to it, and then I get an error and can't get the data.But UAExpert can get array data and report no errors.

[Daemon1992]

I just used the official client tool opcua-client-gui to connect to my server and found that it reported the same error.Is it because my server was developed in C ++ and Open62541...Why does UAExpert get data normally and report no errors，，s

[AndreasHeine]

could you post a screenshot from UaExpert which shows the attributes of your variable? aswell as the code where you define the variable aswell as where you write it?

[Daemon1992]

[Daemon1992]

could you post a screenshot from UaExpert which shows the attributes of your variable? aswell as the code where you define the variable aswell as where you write it?

can u see the screenshot?

[Daemon1992]

this is FreeOpcUa

[AndreasHeine]

in the UaExpert screenshot the most relevant part is not seen the attributes below value like datatype and stuff 😉

like:

then i know exacly what the variable is and how its defined -> then i can try to reproduce!

[Daemon1992]

in the UaExpert screenshot the most relevant part is not seen the attributes below value like datatype and stuff wink

like:

then i know exacly what the variable is and how its defined -> then i can try to reproduce!

I've been a little busy lately, sorry, I changed the array of the server to a variable, and then subscribed to the variable with the Python client, and found that the value of the callback variable was not fully returned.I set the subscription publishing period, from 500ms to 10ms, and did not fully get the full variation of the variable.When server-side variables change very quickly, there are always missing messages that change values.

[Daemon1992]

this is client

[Daemon1992]

this is server

[Daemon1992]

As for the error of the array, I will test it again after I finish the project these days and give you a feedback screenshot.thank you

[AndreasHeine]

if you have missing missing Messages you need to bump up the queuesize!

10ms publisch interval is unrealistic with python... it only means that the client asks every 10ms for changes ... you can use a higher interval like 200 -1000 and an bigger queuesize and you will not miss messages

http://documentation.unified-automation.com/uasdkhp/1.0.0/html/subscription_model.png

https://github.com/FreeOpcUa/python-opcua/blob/1cca33dd5fb70858b5e811b08356ca97520bcbaf/opcua/common/subscription.py#L190

[Daemon1992]

if you have missing missing Messages you need to bump up the queuesize!

10ms publisch interval is unrealistic with python... it only means that the client asks every 10ms for changes ... you can use a higher interval like 200 -1000 and an bigger queuesize and you will not miss messages

http://documentation.unified-automation.com/uasdkhp/1.0.0/html/subscription_model.png

https://github.com/FreeOpcUa/python-opcua/blob/1cca33dd5fb70858b5e811b08356ca97520bcbaf/opcua/common/subscription.py#L190

Maybe I'm not being clear enough, but when I say I'm losing a lot of information, I mean when I update a variable very quickly, I lose some of the values in the middle.Let's say I update the value of A variable, and it goes from 1 to 2,3,4,5,6,7,8.I received incomplete values in my Python client subscription, maybe 2,4,6,8, and nothing else.Hope is clearly expressed. Thank you.

[AndreasHeine]

again! if you update the variable for example with 10Hz (so 10 changes per second) and you have a publish intervall of 1000 you need to use a queuesize of at least 10 because every 1000ms you make a publishrequest and the server queues the changes from publishrequest to publishrequest. on publishrequest the server makes a publishresponse with all queued datachangenotifications (so 10Hz -> ~ 10 Messages per Second sometimes 9 or 11 depends on the processingtime)

your code above shows that the queuesize argument is not used in create_subscription() so it will stay 0 (default) that means only the last change will be published on request so in the 1Hz example i will only recv 1 of 10 changes (9 missing) thats what i would bet happens to you!

i hope i could explain it understandable?

[Daemon1992]

First of all, thanks for your reply.I have tried your way, but still can't solve the problem, still can't receive all the changes... No data has changed more than 10 times during the publication cycle of a request, so quesize = 10 should be fine.....
feel bad................ from 500 to 1000 ...

subscribing to a variable node

self.mSub = self.mClient.create_subscription(1000, self.handler)

self.mSub = self.mClient.create_subscription(500, self.handler) self.handle = self.mSub.subscribe_data_change(msgArr,ua.AttributeIds.Value,10)

[Daemon1992]

May I ask if you have used it before? Now I have added waiting on my server, so that the data does not change so fast, and the problem can be solved for the time being...

[AndreasHeine]

May I ask if you have used it before?

python-opcua, lets say quite often ^^

[AndreasHeine]

Server:

import time, copy
from opcua import Server, ua

myarray = []
for item in range(64):
    myarray.append(0)

if __name__ == "__main__":
    server = Server()
    server.set_endpoint("opc.tcp://127.0.0.1:4840/")
    idx = server.register_namespace("test")
    var = server.get_objects_node().add_variable(idx, "MyArray", ua.Variant(value=myarray, varianttype=ua.VariantType.Int64))
    server.start()

    while 1:
        for idx, item in enumerate(myarray):
            time.sleep(0.1)
            myarray[idx]+=1
            val = copy.copy(myarray)
            dv = ua.DataValue(ua.Variant(value=val, varianttype=ua.VariantType.Int64))
            var.set_value(dv)
            print(var.get_value())

Client:

from opcua import Client
from opcua import ua

class SubHandler(object):

    """
    Subscription Handler. To receive events from server for a subscription
    data_change and event methods are called directly from receiving thread.
    Do not do expensive, slow or network operation there. Create another 
    thread if you need to do such a thing
    """

    def datachange_notification(self, node, val, data):
        print(val)

if __name__ == "__main__":
    client = Client("opc.tcp://localhost:4840")
    client.connect()
    client.load_type_definitions()
    handler = SubHandler()
    sub = client.create_subscription(1000, handler)
    handle = sub.subscribe_data_change(client.get_node("ns=2;i=1"),queuesize=15)

Result:

as you can see there is no dataloss!

long things short! -turn on the logger in debug mode and post the output here so i can see when the error occurs and maybe find out what happens -another thing is checking with wireshark whats inside the publishresponse from the server maybe its possible to see what might cause the error while decoding on clientside...

[Daemon1992]

Thank you very much. I tried your demo by myself and found that there is no problem. However, the server I developed with Open6254 C ++ still has this problem, which may be caused by my server writing.Recently put down this piece, temporarily did not deal with.