Closed cehrett closed 1 month ago
If input data is sufficiently small, chunk size is adjusted to half of the data size, such that an intermediate file should always be formed. Will fail for data with 0 or 1 unique posts, but these cases are trivial.
The script produces an intermediate csv file to temporarily store results, while it extracts frames. The purpose of this intermediate file is to avoid losing all prior extracted frames in cases where the frame extraction process runs into an error. However, when the input csv is sufficiently short, no such temporary intermediate file is created. When this happens, the frame extraction script fails, because it tries to delete the intermediate temp file.