mbloch / mapshaper

Tools for editing Shapefile, GeoJSON, TopoJSON and CSV files
http://mapshaper.org
Other
3.76k stars 532 forks source link

Enable verbosity in runCommands #233

Open simonepri opened 6 years ago

simonepri commented 6 years ago

Problem

-verbose flag does not show anything while using runCommands.

Ideas

It would be nice to add a callback to runCommands that is called each time a new message is generated by the verbosity instead of print it to the console. In that way we programmatically can show or not show certain messages to the user!

Use case

Show the progress of the command to the user.

Is that possible?

mbloch commented 6 years ago

If you're using mapshaper as a node module, console logging is not enabled by default. You have to enable it like this:

var mapshaper = require("mapshaper");
mapshaper.enableLogging();

With logging enabled, you should see the default set of console messages, even without the -verbose flag. The -verbose command doesn't do much more than display timing info for each command.

Does this satisfy your use case?

I'm not perfectly happy with the way the -verbose and -quiet commands currently operate. Maybe I should deprecate them and add a single command to control logging, used like this: -logging on -logging off -logging verbose. This way, you could enable console logging in the command string instead of having to call enableLogging().

simonepri commented 6 years ago

Thank you for the answer @mbloch!

Does this satisfy your use case?

The problem is that mapshaper it seems slow with "big" files. It would be really nice to be able to have something like a "progress-bar".

BTW thank you for this amazing tool, the project where i'm using it (geo-maps) is almost entirely based on it.

What I mean with slow?

For instance I've run this command 60 minutes ago:

mapshaper \
-i combine-files ./world-rivers.geo.json ./world-lakes.geo.json ./world-seas.geo.json \
-merge-layers
-dissolve \
-o precision=0.000001 format=geojson ./world-waterboadies.geo.json \
-verbose

And I'm still waiting for an output.

EDIT: Removing 'dissolve' speeds up the process to 11 mins.

mapshaper \
-i combine-files ./world-rivers.geo.json ./world-lakes.geo.json ./world-seas.geo.json \
-merge-layers
-o precision=0.000001 format=geojson ./world-waterboadies.geo.json \
-verbose

image

That's strange because the following message is printed almost immediately (with the -dissolve option): image

Can this slowness be related with the length of the array? Should I open a separate issue for this thing?

What I mean with big?

File Size
world-rivers.geo.json (GeometryCollection with a single MultiPolygon) 48.8 MB
world-seas.geo.json (GeometryCollection with a single MultiPolygon) 39.3 MB
world-lakes.geo.json (GeometryCollection with a single MultiPolygon) 66.6 MB