spatial-go / geoos

A library provides spatial data and geometric algorithms
http://www.spatial-go.com
GNU Lesser General Public License v2.1
617 stars 130 forks source link

ReadBytes buffer is 1 byte, Ultra slow #102

Closed TheFuzzyGiggler closed 1 year ago

TheFuzzyGiggler commented 1 year ago

In the GeoJson package, under the base encoder file your ReadBytes buffer is 1 byte. For any decently sized file this makes it take FOREVER.

Any reason this isn't bumped up to like 4096?

// ReadBytes Returns geometry from reader.
func (e *BaseEncoder) ReadBytes(r io.Reader) ([]byte, error) {
    buf := []byte{}
    b := make([]byte, 1)
    for {
        if _, err := r.Read(b); err == io.EOF {
            //buf = append(buf, b[0:n]...)
            break
        } else if err != nil {
            return nil, err
        } else {
            buf = append(buf, b...)
        }
    }
    return buf, nil
}
coolwxb commented 1 year ago

Thank you for your interest in and usage of geoos. Actually, there is no specific reason to set the buffer size as 1. We have also noticed this issue, and it does affect the parsing efficiency. In the upcoming release, we will optimize and fix this problem.

In the GeoJson package, under the base encoder file your ReadBytes buffer is 1 byte. For any decently sized file this makes it take FOREVER.

Any reason this isn't bumped up to like 4096?

// ReadBytes Returns geometry from reader.
func (e *BaseEncoder) ReadBytes(r io.Reader) ([]byte, error) {
  buf := []byte{}
  b := make([]byte, 1)
  for {
      if _, err := r.Read(b); err == io.EOF {
          //buf = append(buf, b[0:n]...)
          break
      } else if err != nil {
          return nil, err
      } else {
          buf = append(buf, b...)
      }
  }
  return buf, nil
}