search

scanner-research / esper-tv

Esper instance for TV news analysis
Apache License 2.0
39 stars 10 forks source link

add proposed rekall_vgrid glue #97

Closed njlxyaoxinwei closed 5 years ago

njlxyaoxinwei commented 5 years ago

Here is a proposed module acting as bridge code from rekall to the redesigned VGrid.

It attempts to take out some of the complexities VGrid currently handles internally to transform interval sets and do it in a more principled way. The goal is to produce a JSON structure that maps to VGrid concepts directly. Specifically, the output of build() is exactly intervals: {[key: string]: IntervalSet}[] in VGridProps.

In particular, this would supersede the hacky to_json method in rekall IntervalSet3D and become the standard way to convert rekall intervals to JSON for VGrid.

This PR is purely to propose such a bridge module on esper side and does not work with current VGrid. If accepted, VGrid would need additional work to correctly handle outputs produced by this module, but I suspect it would clean up the code there a bit.

Take a look at the examples in module doc string and let me know what you guys think.

njlxyaoxinwei commented 5 years ago

As per today's discussion, I moved most of the logic to a subpackage in rekall.vgrid_utils in the rekall repo, leaving only things coupled with esper database schema.