Why is nullable filtered out?

[Gunni]

I am talking with a varlink service, one of its fields is type ?int when the api returns a payload where this value is null then the python code filters it out, I tracked it down to this block of code: https://github.com/varlink/python/blob/master/varlink/scanner.py#L428

I'm just trying to understand.

1. Why is a null field in a struct not propagated to a client
2. How can a client discover all fields and create a valid instance of the object if it doesn't get to know all the fields.
3. If it's being sent to the client then it can't be about saving traffic, so why filter it out?

f.ex:

import varlink
with varlink.Client('unix:/tmp/test.sock') as client, client.open('com.example.network') as c:
    n = c.Get()
    print(type(n))
    print(n)

Where the interface is:

interface com.example.network

type NetworkSettings (
    ...
    ETH1VLAN_VLAN: ?int,
    ETH1VLAN_CIDR_V4: []string,
    ETH1VLAN_CIDR_V6: []string,
    ...
)

method Get() -> (settings: NetworkSettings)
method Set(settings: NetworkSettings) -> ()
method Validate(settings: NetworkSettings) -> ()

I get:

<class 'dict'>
{'settings': {... 'ETH1VLAN_CIDR_V4': [], 'ETH1VLAN_CIDR_V6': [], ...}}

If the value for ETH1VLAN_VLAN is null then the client gets it but filters it out, meaning I cannot discover the key ETH1VLAN_VLAN. If I want to print out the dict I got back, it's not even type NetworkSettings, it's just a plain dict...

Am I using this wrong?

[Gunni]

So I managed to get using this hack i cobbled together:

import json
import varlink

def parse_signature(signature):
    _, return_type = signature.split('->')
    return_type = return_type.strip('() ')
    args = return_type.split(', ')
    return {arg.split(': ')[0]: arg.split(': ')[1] for arg in args}

def merge_dicts(dict1, dict2):
    for key, value in dict2.items():
        if key in dict1 and isinstance(dict1[key], dict) and isinstance(value, dict):
            dict1[key] = merge_dicts(dict1[key], value)
        else:
            dict1[key] = value
    return dict1

def varlink_func_to_objs(client: varlink.Client, intf: str, func: str, svc):
    iface = client.get_interface(intf)
    sig = iface.get_method(func).signature
    key_types = parse_signature(sig)
    ret = {}

    for key, value in key_types.items():
        obj_fields = client.get_interface(intf).members[value].type.fields
        obj = {}

        for field_key, field_value in obj_fields.items():
            if isinstance(field_value, varlink.scanner._Array):
                obj[field_key] = []
            elif isinstance(field_value, varlink.scanner._Maybe):
                obj[field_key] = None

            ret[key] = obj

    result = getattr(svc, func)()
    return merge_dicts(dict(ret), result)

def main():
    interface = 'com.mareldigital.os.network'
    with varlink.Client('unix:/tmp/test.sock') as client, client.open(interface) as service:
        objs = varlink_func_to_objs(client, interface, 'Get', service)
        print(json.dumps(objs, indent='\t'))

if __name__ == '__main__':
    main()

It just seems really not the right way to do it, even going as far as using protected members of the scanner to figure out defaults for keys.

I just want to understand, so please do tell.