struct.error: unpack requires a buffer of 32 bytes

[msw17002]

PyShp Version

2.3.1

Python Version

3.9.12

Your code

#-create file
    w  = shapefile.Writer(outbin+outdir+"/"+outproc)
    #-define 'w' file's fields
    ro = shapefile.Reader(infileo)
    for field in ro.fields[1:]:
        if field[0] in keep_fields:
            w.field(*field)
    if dtofile==dtefile:
        #-append temporal records to a file
        for shapeRec in ro.iterShapeRecords():
            #-retrieve record
            rec = shapeRec.record
            #-determine if record occured on day
            #-issued and expired
            issto = datetime.datetime(int(rec[1][0:4]),int(rec[1][4:6]),int(rec[1][6:8]),int(rec[1][8:10]),int(rec[1][10:12]))
            isste = datetime.datetime(int(rec[2][0:4]),int(rec[2][4:6]),int(rec[2][6:8]),int(rec[2][8:10]),int(rec[2][10:12]))
            #-init_iss and init_exp
            inito = datetime.datetime(int(rec[1][0:4]),int(rec[1][4:6]),int(rec[1][6:8]),int(rec[1][8:10]),int(rec[1][10:12]))
            inite = datetime.datetime(int(rec[2][0:4]),int(rec[2][4:6]),int(rec[2][6:8]),int(rec[2][8:10]),int(rec[2][10:12]))
            #-take the max of the ends and the min of the starts
            pgdto = min([issto,inito])
            pgdte = max([isste,inite])
            #-determine if record is a tornado or severe thunderstorm warning...
            # If both conditions are met, save the record
            #if (has_overlap(dto, dte, pgdto, pgdte) and ((rec[5]+rec[7])=="SVW" or (rec[5]+rec[7])=="TOW")):
            if has_overlap(dto, dte, pgdto, pgdte):
                w.record(*shapeRec.record)
                w.shape(shapeRec.shape)

Full stacktrace

Traceback (most recent call last):
  File "/simulations/NWS/./gen-nws.py", line 83, in <module>
    for shapeRec in ro.iterShapeRecords():
  File "/home/walters.michael/miniconda3/lib/python3.9/site-packages/shapefile.py", line 1771, in iterShapeRecords
    for shape, record in izip(self.iterShapes(), self.iterRecords(fields=fields)):
  File "/home/walters.michael/miniconda3/lib/python3.9/site-packages/shapefile.py", line 1483, in iterShapes
    shape = self.__shape(oid=i, bbox=bbox)
  File "/home/walters.michael/miniconda3/lib/python3.9/site-packages/shapefile.py", line 1319, in __shape
    record.bbox = _Array('d', unpack("<4d", f.read(32)))
struct.error: unpack requires a buffer of 32 bytes

Other notes

Hello all,

I'm trying to subset a shapefile by a time range. This script works for other files (older/smaller sized shapefiles), but not this one along with several others. The script does run for a few iterations of ro.iterShapeRecords(), but it usually fails at some point.

Thanks!

[msw17002]

To add more details;

If I run the script for different dates/files, I receive the same error, but with varying amount of 'buffer' bytes.

So I also got these errors, as well: struct.error: unpack requires a buffer of 2800 bytes struct.error: unpack requires a buffer of 9680 bytes etc...

[msw17002]

I wasn't able to script a solution to this, but the problem isn't duplicated for smaller shapefiles. The files I was working with were too larger.