tmontaigu / shapefile-rs

Rust library to read & write shapefiles
MIT License
59 stars 19 forks source link

why the shapefile read .shp Slower than shapefile(pyshp) of python? It doesn't make #35

Open kl402401 opened 6 months ago

kl402401 commented 6 months ago

it don't make any sense

kl402401 commented 6 months ago

@tmontaigu

tmontaigu commented 6 months ago

What about sharing the code you use ? and how you compile, and the file you try to read

Your issue doesn't make sense

tmontaigu commented 6 months ago

@kl402401

tmontaigu commented 6 months ago

no info = no bug = no fix

kl402401 commented 6 months ago

no info = no bug = no fix

sorry ,it maybe not a bug, but i use the pyshp of python to read and iterate through the .shp that it has 96w shp, it just cost 30secs,but the same .shp, use shapefile of rust ,it cost 180secs

tmontaigu commented 6 months ago

I can't do anything about that unless you give some actual info

kl402401 commented 6 months ago

sorry ,I can't share the file ,it's a secret file.

.it's a road.shp of the beijing that one of the city in china. it may vary big, it has 9.6 million shape in one .shp。

.the rust code just like that, i build it with --release flag: use std::time::{Instant}; fn main() { let start = Instant::now(); let mut reader = shapefile::Reader::from_path("D:\road0714out\result1\beijing\road.shp").unwrap();

for result in reader.iter_shapes_and_records() {
    let (shape, record) = result.unwrap();
    println!("Shape: {}, records: ", shape);
    for (name, value) in record {
        println!("\t{}: {:?}, ", name, value);
    }
    println!();
}
let duration = start.elapsed();
let seconds = duration.as_secs();

println!("Execution time in seconds: {}", seconds);

}

kl402401 commented 6 months ago

Execution time in seconds: 1114 this time it cost 1114secs, is that the code problem? that maybe a little slow

tmontaigu commented 6 months ago

Does the file have an index file road.shx that goes along with it ? If so I think I have an idea for the where slowness comes from and know how to fix

tmontaigu commented 6 months ago

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

kl402401 commented 6 months ago

Does the file have an index file road.shx that goes along with it ? If so I think I have an idea for the where slowness comes from and know how to fix

it has a 'road.shx'.

Ok, I have a local fix that makes me read a shape file with 2246324 shapes in ~17s intead of 27s

Also, When timing the reading code don't include the prints as its what actually takes most of the time

That's really great,can you update the project?

tmontaigu commented 6 months ago

Fix merged in #36