Map

[SchrodingersGat]

This PR adds the ability to encode (and decode) a protogen structure to a QMap<QString, QString>, essentially providing a key:value pair encoding for the structure.

(This is functionality I have been using but previously it was written in Python and an extra script ran through the same .xml file)

This functionality is similar to the textprint functions but it provides some advantages:

• Greater flexibility in how the data are used (textprint just returns a single string, cannot easily be easily rearranged, appended, pruned, etc)
• Ability to partially load settings files (e.g. settings files can be hand-adjusted to remove any settings that the user does not want to be overwritten
• QMap provides quicker lookup than parsing (repeatedly) across a very long string object
• Provides protection against the (rare) case where you may have a string literal that matches the search part of extractText function

Example

Here is an example xml descriptor:

    <Structure name="SubStruct" file="packets" map="true">
        <Data name="x" inMemoryType="float"/>
        <Data name="y" inMemoryType="float"/>
        <Data name="z" inMemoryType="float"/>
    </Structure>

    <Packet name="ExampleStruct" file="packets" map="true">
        <Data name="uchar" inMemoryType="unsigned8" comment="Unsigned byte"/>
        <Data name="ichar" inMemoryType="signed8" comment="Signed byte"/>
        <Data name="b" inMemoryType="bitfield4" comment="Bitfield (4 bits)"/>
        <Data name="n" inMemoryType="unsigned32" comment="Array size"/>
        <Data name="f" inMemoryType="float" array="10" variableArray="n" comment="Float array (up to 10 items)"/>
        <Data name="s" inMemoryType="string" array="32"/>
        <Data name="a2d" inMemoryType="signed16" array="10" array2d="5" comment="Two dimensional array"/>
        <Data name="sub" struct="SubStruct" comment="Sub structs work too!"/>
    </Packet>

And the resulting code:

/*!
 * Encode the contents of a Settings_SubStruct_t structure to a Key:Value string map
 * \param _pg_prename is prepended to the key fields in the map
 * \param _pg_map is a reference to the map
 * \param _pg_user is the structure to encode
 */
void mapEncodeSettings_SubStruct_t(const QString& _pg_prename, QMap<QString, QString>& _pg_map, const Settings_SubStruct_t* _pg_user)
{

    _pg_map[_pg_prename + ":x"] = QString::number(_pg_user->x, 'g', 16);

    _pg_map[_pg_prename + ":y"] = QString::number(_pg_user->y, 'g', 16);

    _pg_map[_pg_prename + ":z"] = QString::number(_pg_user->z, 'g', 16);

}// mapEncodeSettings_SubStruct_t

/*!
 * Decode the contents of a Settings_SubStruct_t structure to a Key:Value string map
 * \param _pg_prename is prepended to the key fields in the map
 * \param _pg_map is a reference to the map
 * \param _pg_user is the structure to encode
 */
void mapDecodeSettings_SubStruct_t(const QString& _pg_prename, QMap<QString, QString>& _pg_map, Settings_SubStruct_t* _pg_user)
{
    QString key;

    key = _pg_prename + ":x";
    if(_pg_map.contains(key))
        _pg_user->x = _pg_map[key].toDouble();

    key = _pg_prename + ":y";
    if(_pg_map.contains(key))
        _pg_user->y = _pg_map[key].toDouble();

    key = _pg_prename + ":z";
    if(_pg_map.contains(key))
        _pg_user->z = _pg_map[key].toDouble();

}// mapEncodeSettings_SubStruct_t

/*!
 * Encode the contents of a Settings_ExampleStruct_t structure to a Key:Value string map
 * \param _pg_prename is prepended to the key fields in the map
 * \param _pg_map is a reference to the map
 * \param _pg_user is the structure to encode
 */
void mapEncodeSettings_ExampleStruct_t(const QString& _pg_prename, QMap<QString, QString>& _pg_map, const Settings_ExampleStruct_t* _pg_user)
{
    unsigned _pg_i = 0;
    unsigned _pg_j = 0;

    _pg_map[_pg_prename + ":uchar"] = QString::number(_pg_user->uchar);

    _pg_map[_pg_prename + ":ichar"] = QString::number(_pg_user->ichar);

    _pg_map[_pg_prename + ":b"] = QString::number(_pg_user->b);

    _pg_map[_pg_prename + ":n"] = QString::number(_pg_user->n);

    for(_pg_i = 0; (_pg_i < 10) && (_pg_i < (unsigned)_pg_user->n); _pg_i++)
    {
        _pg_map[_pg_prename + ":f" + "[" + QString::number(_pg_i) + "]"] = QString::number(_pg_user->f[_pg_i], 'g', 16);
    }

    _pg_map[_pg_prename + ":s"] = QString(_pg_user->s);

    for(_pg_i = 0; _pg_i < 10; _pg_i++)
    {
        for(_pg_j = 0; _pg_j < 5; _pg_j++)
        {
            _pg_map[_pg_prename + ":a2d" + "[" + QString::number(_pg_i) + "]" + "[" + QString::number(_pg_j) + "]"] = QString::number(_pg_user->a2d[_pg_i][_pg_j]);
        }
    }

    mapEncodeSettings_SubStruct_t(_pg_prename + ":sub", _pg_map, &_pg_user->sub);

}// mapEncodeSettings_ExampleStruct_t

/*!
 * Decode the contents of a Settings_ExampleStruct_t structure to a Key:Value string map
 * \param _pg_prename is prepended to the key fields in the map
 * \param _pg_map is a reference to the map
 * \param _pg_user is the structure to encode
 */
void mapDecodeSettings_ExampleStruct_t(const QString& _pg_prename, QMap<QString, QString>& _pg_map, Settings_ExampleStruct_t* _pg_user)
{
    unsigned _pg_i = 0;
    unsigned _pg_j = 0;
    QString key;

    key = _pg_prename + ":uchar";
    if(_pg_map.contains(key))
        _pg_user->uchar = _pg_map[key].toUInt();

    key = _pg_prename + ":ichar";
    if(_pg_map.contains(key))
        _pg_user->ichar = _pg_map[key].toInt();

    key = _pg_prename + ":b";
    if(_pg_map.contains(key))
        _pg_user->b = _pg_map[key].toUInt();

    key = _pg_prename + ":n";
    if(_pg_map.contains(key))
        _pg_user->n = _pg_map[key].toUInt();

    for(_pg_i = 0; (_pg_i < 10) && (_pg_i < (unsigned)_pg_user->n); _pg_i++)
    {
        key = _pg_prename + ":f" + "[" + QString::number(_pg_i) + "]";
        if(_pg_map.contains(key))
            _pg_user->f[_pg_i] = _pg_map[key].toDouble();
    }

    key = _pg_prename + ":s";

    if (_pg_map.contains(key))
        strncpy(_pg_user->s, _pg_map[key].toLatin1().constData(), 32);

    for(_pg_i = 0; _pg_i < 10; _pg_i++)
    {
        for(_pg_j = 0; _pg_j < 5; _pg_j++)
        {
            key = _pg_prename + ":a2d" + "[" + QString::number(_pg_i) + "]" + "[" + QString::number(_pg_j) + "]";
            if(_pg_map.contains(key))
                _pg_user->a2d[_pg_i][_pg_j] = _pg_map[key].toInt();
        }
    }

    mapDecodeSettings_SubStruct_t(_pg_prename + ":sub", _pg_map, &_pg_user->sub);

}// mapEncodeSettings_ExampleStruct_t

[SchrodingersGat]

The generated map decode routines are now significantly improved. The process for decoding a primitive from the map is as follows:

1. Check that the provided key is actually in the map
2. Check that the value in the map can be successfully decoded to the primitive data type
3. If 1. and 2. pass, then extract the value, convert it, and store it in the struct

This way, missing values are ignored, as are mal-formed values (e.g. "1234xx" cannot be converted to an integer)

[SchrodingersGat]

Some example code for the improved mapDecode_ routine (generated code):

    QString key;  // Temporary map key variable
    bool ok = false;  // Temporary data validation variable
    unsigned _pg_i = 0;  // Array iterator

    // Number of map elements
    key = _pg_prename + ":size";
    if(_pg_map.contains(key))
    {
        _pg_map[key].toUInt(&ok);
        if(ok)
            _pg_user->size = _pg_map[key].toUInt();
    }

    // Manual RPM setpoint timeout
    key = _pg_prename + ":manualRpmTimeout";
    if(_pg_map.contains(key))
    {
        _pg_map[key].toUInt(&ok);
        if(ok)
            _pg_user->manualRpmTimeout = _pg_map[key].toUInt();
    }

[jefffisher]

Just out of pure curiosity, why are the values stored as QString rather than QVariant?

[SchrodingersGat]

No particular reason, apart from providing consistent comparison between different variables. Enforcing QString means that there's no ambiguity (and the data types are defined in protogen anyway). Is there a specific advantage you could see from using QVariant?

[jefffisher]

Well we've been using QJsonDocument, et. al., lately, which basically gives you a big map of quasi-QVariants and it's pretty slick. So really it was more that I was wondering whether you'd seen any specific advantages to using QString

[SchrodingersGat]

I'll have a look at the way I am using the QMap and see if I can shoehorn a QVariantMap. You'll owe me a drink :)

[SchrodingersGat]

Instead of QMap<QString, QString> it now uses QVariantMap.

Everything else works the same. The encode routines are a bit cleaner now as they rely on the QVariant object to correctly "encode" the primitive types.

[billvaglienti]

It looks like the map encode / decode functionality is incomplete. In ProtocolStructure.cpp, at the end of the function getMapEncodeString() and getMapDecodeString() the code is not yet implemented.

Also I noticed that if you set a global mapfile="somefile" you don't get any output. This is contrary to the text output functions; which, if you set a global textprint file you get outputs for every structure.

[SchrodingersGat]

Bill, when does ProtocolStructure::GetMapEncodeString actually get called? I have put some debug messages in there that never get displayed.

As far as I can tell, all the necessary mapEncode / mapDecode functions are in:

ProtocolStructre::getMapEncodeFunctionString

ProtocolField::getMapEncodeString

If you can provide the case where ProtocolStrucutre::getMapEncodeString is actually called, I can test and fix the missing code.

[billvaglienti]

Run it against the exampleprotocol.xml, that will show the problem.

On Wed, Apr 18, 2018, 6:44 PM Oliver notifications@github.com wrote:

Bill, when does ProtocolStructure::GetMapEncodeString actually get called? I have put some debug messages in there that never get displayed.

As far as I can tell, all the necessary mapEncode / mapDecode functions are in:

ProtocolStructre::getMapEncodeFunctionString

ProtocolField::getMapEncodeString

If you can provide the case where ProtocolStrucutre::getMapEncodeString is actually called, I can test and fix the missing code.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/billvaglienti/ProtoGen/pull/76#issuecomment-382582460, or mute the thread https://github.com/notifications/unsubscribe-auth/AKWWHoD3t-4XlZuYwJ9lz3HEKkOzKmYnks5tp-v6gaJpZM4TEWtX .

[SchrodingersGat]

Thanks, I have added "mapfile" to the test protocol so this feature is tested there too.

I think I have implemented all the different encode/decode cases now...

I've also made the global "mapfile" declaration force encode/decode functions, as per your other request.