2.0.0-dev ? .records

[apiszcz]

It appears this attribute is no longer available.

create__outputs
    if len(ws.records)>0:
AttributeError: 'Writer' object has no attribute 'records'

[karimbahgat]

Correct, that is one of the new incompatible changes for the upcoming major version v2.x.

Previously all records and shapes were stored in memory as attributes on the Writer class, which limits how large shapefiles it is possible to write. This limitation is removed in the upcoming major version, so that the writer now writes each record/shape immediately to a temporary file. It's a similar streaming approach as the csv module. The "Working with Large Shapefiles" section of the Readme explains the new behavior.

This will mean that you can't inspect which records or shapes you have written after the fact, but to check how many entries you have written you can instead call len(ws). I will likely add a large note at the top of the v2 Readme to explain the main changes such as this.

[apiszcz]

Hmm, ok, is there a work around for this?

                    w.records.extend(r.records())
                    w._shapes.extend(r.shapes())

I see the iterShapeRecords for the read method, is there a new corresponding write ShapeRecord to write one feature at a time to the file? Maybe this...?

                    r = shapefile.Reader(f)
                    for rsfi in r.iterShapeRecords():
                        w.record(rsfi.record)
                        w.shape(rsfi.shape)

[karimbahgat]

That's exactly how you would do it. Except the record input has to be a list arg as before, meaning w.record(*rsfi.record).

Writing records and shapes now has a 1-to-1 mapping in terms of how one would read records and shapes.

Type	Reading	Writing
Records	r.record(...)	w.record(...)
Shapes	r.shape(...)	w.shape(...)

What's more, the new .shape() method can also take a GeoJSON dictionary if you prefer that.

The new way of writing shapes is explained in the "Adding from an existing Shape object" section of the Readme:

Finally, geometry can be added by passing an existing "Shape" object to the "shape" method. This can be particularly useful for copying from one file to another:

>>> r = shapefile.Reader('shapefiles/test/polygon')

>>> w = shapefile.Writer()
>>> w.fields = r.fields[1:] # skip first deletion field

>>> for shaperec in r.iterShapeRecords():
...     w.record(*shaperec.record)
...     w.shape(shaperec.shape)

>>> w.save('shapefiles/test/copy')

[apiszcz]

Thank you for the quick response with an improved example, I'll give this a try on with my uses of shapefile.py

[apiszcz]

It appears the bbox may not be set correctly, or in my case the bbox is not correct. Reviewing that now.

(Pdb) print(w.bbox()) [0, 0, 0, 0]

So for POINT shapefiles a bbox is not considered/created? Is that the right? Lines 997-1001

        # Write a single point
        if s.shapeType in (1, 11, 21):
            try:
                f.write(pack("<2d", s.points[0][0], s.points[0][1]))
            except error:
                raise ShapefileException("Failed to write point for record %s. Expected floats." % recNum)

[apiszcz]

Is there a better way so we can work against more than the 'POINT' case?

( inside record iteration )
                        if shaperec.shape.points[0][0] < x0: x0=shaperec.shape.points[0][0]
                        if shaperec.shape.points[0][0] > x1: x1=shaperec.shape.points[0][0]
                        if shaperec.shape.points[0][1] < y0: y0=shaperec.shape.points[0][1]
                        if shaperec.shape.points[0][1] > y1: y1=shaperec.shape.points[0][1]
.
.
w._bbox=[x0,y0,x1,y1]

[pstatix]

I've posted a new issue on this since ArcGIS requires shapefiles to have an extent regardless of shape type. If creating a POINT shapefile with a single point, say (3, 4), the bounding box must be updated to include the new point. If it is not, ArcGIS loads the file but does not display the shapes because no spatial index exists for referencing the points.

Further, regardless of shape type, the bounding box (self._bbox) is always in the form 0 0 Max-X Max-Y. This produces unnecessary extent; say if points are only between x: [10 < x < 50], y: [3 < y < 15], you end up with all the extent from 0,0 that does not contain any shape data.

The heart of this issue lies within the self.__bbox() method and how the global bounding box is being updated. Since it is initialized to [0, 0, 0, 0], the min() calls will always be 0 unless a point is negative.

[apiszcz]

Understand, thank you, using FIONA/GEOPANDAS/GDAL the correct behavior for ArcMap is satisfied. I look forward to the same for phsyp. Thank you.

On Fri, Dec 15, 2017 at 11:09 AM, pstatix notifications@github.com wrote:

I've posted a new issue on this since ArcGIS requires shapefiles to have an extent regardless of shape type. If creating a POINT shapefile with a single point, say (3, 4), the bounding box must be updated to include the new point. If it is not, ArcGIS loads the file but does not display the shapes because no spatial index exists for referencing the points.

Further, regardless of shape type, the bounding box (self._bbox) is always in the form 0 0 Max-X Max-Y. This produces unnecessary extent; say if points are only between x: [10 < x < 50], y: [3 < y < 15], you end up with all the extent from 0,0 that does not contain any shape data.

The heart of this issue lies within the self.__bbox() method and how the global bounding box is being updated. Since it is initialized to [0, 0, 0, 0], the min() calls will always be 0 unless a point is negative.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/GeospatialPython/pyshp/issues/113#issuecomment-352044264, or mute the thread https://github.com/notifications/unsubscribe-auth/ABXVTciUdR-BaTTl6wcUBVprfvMpDuprks5tApmqgaJpZM4PBPSr .

[pstatix]

If you are using pyshp now, other than the incorrect extent of a POINT file, you can correct it in ArcMap by running the layer through the "Add Spatial Index" tool. External solution but fixes it none-the-less.

[karimbahgat]

Closing this since it has been raised as a new separate issue in #130.