feat: Initial Support for FlatBuffer context - Githubissues

VowpalWabbit / reinforcement_learning

Interaction-side integration library for Reinforcement Learning loops: Predict, Log, [Learn,] Update

MIT License

75 stars 40 forks source link

feat: Initial Support for FlatBuffer context #608

Open lokitoth opened 8 months ago

lokitoth commented 8 months ago

Add support for using flatbuffer spans as input to RLClientLib when using a VW model.

Note: Currently only working in CB models when using choose_rank().

[x] Fix memory leak when providing bad buffer (e.g. JSON) to a Flatbuffer-configured LiveModel
[x] Figure out why the build breaking on fb_parser_test
- reinforcement_learning builds on ubuntu1804; vowpal_wabbit builds on ubuntu2004
[ ] Verify whether CA models work out of the box with Flatbuffer contexts
[ ] When no more changes to VW need to be made, return submodule back to pointing to a trunk commit of VW

Stretch:

[ ] Support RequestDecision/MultislotDecision (need to implement context introspection for FB contexts)
[ ] (unlikely) Support Episodic (need to implement history generation / merge; to do this right requires a fair bit of engineering)