search

cehrett / social_media_frame_analysis

Tools for extracting and analyzing frames/theories/narratives from social media posts.
2 stars 0 forks source link

Frame extraction script fails when the input csv is too short #18

Closed cehrett closed 1 month ago

cehrett commented 1 month ago

The script produces an intermediate csv file to temporarily store results, while it extracts frames. The purpose of this intermediate file is to avoid losing all prior extracted frames in cases where the frame extraction process runs into an error. However, when the input csv is sufficiently short, no such temporary intermediate file is created. When this happens, the frame extraction script fails, because it tries to delete the intermediate temp file.

Cooper-Taylor commented 1 month ago

If input data is sufficiently small, chunk size is adjusted to half of the data size, such that an intermediate file should always be formed. Will fail for data with 0 or 1 unique posts, but these cases are trivial.