include group parsing

[rspadim]

hi, maybe we could implement a group parsing object

it receive the group tags that exists in a group at construction when init it receive the tag group name (nomdentries) and the group that will be read it find the first tag and the last tag, with that we can recursive find group inside group the point is find the start byte and final byte of a group

do you think it's a good idea? i could help implementing

[da4089]

The current mechanism for accessing parsed group members, and especially nested group members, is admittedly not great.

But the parser currently avoids almost all semantic knowledge. To do something more with groups, it would need to know which tag numbers were NumInGroup type, and I think it'd also need to know which tags were legal group-element tags too, so that it could find the first non-group-element field after the final element?

Thus far, I've tried to avoid this kind of thing in simplefix: with its emphasis on being simple, I am reluctant to require a schema definition in order to parse a message.

All that said, I have wondered whether optional layers on top of the basic simplefix functionality are a good idea. Networking has been the main possibility here, but a schema-checking layer has been another option, and perhaps a schema-aware parser as part of that would be useful?

[rspadim]

i like the idea of a simplefix as a parser with python only language the network idea is ok, but the today world is asyncio, cython, numba

about groups:

we can copy fix group definitions from quickfix, for example: https://github.com/quickfix/quickfix/blob/master/src/python/quickfix44.py#L3228

the idea is something like:

1) create a "group object" -> this is simplefix

2) create many "group object" that already know what fix tags it accept (hardcoded) like quickfix -> that's the part non "simplefix"

3) pass a bytearray/memoryview group object and it will create some memoryview to read this bytearray without double copy -> this is simplefix

i think it works and is optimized with memoryview instead of double copy strings/bytearray

[da4089]

So, the reason you'd rather not use QuickFIX is that it isn't pure Python?

[rspadim]

Well i just like the simplefix, quickfix have more features, but here with python world we don’t need a low level high speed lib, but a lib with python language is very nice, or cython/numba if we need speed

I use simplefix mainly to parse strings

[da4089]

Right -- that's the sweet spot for simplefix: it's not intended to be super fast, just quick and easy.

So I am interested in making using repeating groups easier, but given that it requires using a schema definition, that's a big loss of simplicity to overcome to make it easier overall, I think.

So, if it's ok with you, I'd like to leave this issue open while I think some more about it, and perhaps get some feedback from other users. I'd like to do something here, I'm just not certain how it should be structured yet.

I'll post an update when I've thought some more.

[rspadim]

Yeap

I’m think something like

G= Simplefix.create_group(message, tag_id, simplefix.groups.marketdatasnapshot.nomdentries)

The simplefix.groups is a module just to “save standard groups definition”

And use the g to anything

Like g.tags (all tags found in this group), g.groups (something to [for i in g.groups] or g.groups[0]), g.get_string(), g.startbyte, g.endbyte, g.mapsbytes (array of start/end)