This PR solves one main problem (1) but includes lots of clean up and refactoring of old V1 code and concepts as well (2 & 3). It's a very large PR, so I've annotated it heavily to help with reviewing it
1: In IO-1163, we introduced logic to exports to compress the resulting JSON if we determine that the export size was sufficiently large that it would cause it to OOM. This compression took the form of only exporting keyframes, and adding an only_keyframes field in the JSON to reflect this. Unfortunately, darwin-py import logic was built on the assumption that every frame has a main annotation type, which this change violates, breaking these imports
2: Removal of the concept of a complex_polygon annotation type. In V1, we stored polygons as complex polygons if they had >1 path. We removed this distinction in V2, instead referring to all polygons as type polygon. Therefore we should update darwin-py to reflect this
3: Removal of old functions & erroneous deprecation messages. Darwin-py contains a lot of unused functions from the V1 version of the platform. These have already been re-written for V2 and will thus never be used again. There were also many deprecation messages that were not needed, out of date or wrong (functions were actually in use)
Solution
1: For each annotation with only_keyframes set to True, get the first frame with a main type. Store this type & annotation data. Then, iterate through each frame of the annotation. If the main type is missing, use the stored value to populate it for import. Otherwise if the main type is present, update the stored annotation data. Repeat this for the entire annotation
2: Removed the concept of complex polygons from darwin-py, instead referring to all polygons simply as type polygon to reflect how we store polygons in our DB
3: Removed the unused functions, and removed incorrect deprecation messages
Changelog
Updated import logic to support importing of annotations that were compressed during export to avoid memory cap errors
Removed the distinction between complex polygons & polygons
Problem
This PR solves one main problem (1) but includes lots of clean up and refactoring of old V1 code and concepts as well (2 & 3). It's a very large PR, so I've annotated it heavily to help with reviewing it
only_keyframes
field in the JSON to reflect this. Unfortunately, darwin-py import logic was built on the assumption that every frame has a main annotation type, which this change violates, breaking these importscomplex_polygon
annotation type. In V1, we stored polygons as complex polygons if they had >1 path. We removed this distinction in V2, instead referring to all polygons as typepolygon
. Therefore we should update darwin-py to reflect thisSolution
only_keyframes
set to True, get the first frame with a main type. Store this type & annotation data. Then, iterate through each frame of the annotation. If the main type is missing, use the stored value to populate it for import. Otherwise if the main type is present, update the stored annotation data. Repeat this for the entire annotationpolygon
to reflect how we store polygons in our DBChangelog