search

refactor / geoimg_shredder

geo-image shredder, cut out tiles along space-cell, like gdal2tiles
1 stars 0 forks source link

wrong syntax in readme.md? #1

Open lamuertepeluda opened 12 years ago

lamuertepeluda commented 12 years ago

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?). Thanks :)

refactor commented 12 years ago

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?). Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

lamuertepeluda commented 12 years ago

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you know how to query riak for generated tiles key? My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

Do you think I could reconfigure your program to use these parameters?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?). Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825.

refactor commented 12 years ago

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub < https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

lamuertepeluda commented 12 years ago

Hi,

the biggest problem with gdal2tiles.py is... it's sloooooowwww! :)

I have a machine (a powerful Mac Pro) running it with its GUI wrapper (from http://www.maptiler.org/) on this same map since June 27th (zoom levels 8-16), still there computing. The map is a very high detailed map coming from a GIS software and it represents just a region, not the whole globe, and that's why the source reference system is EPSG:32632 and not EPSG:900913. I need to make this map compatible with Google, in TMS format, and gdal2tiles would do the job.

I know there is an high parallelized version (Maptiler cluster) sold on the same site, but its price is about 2300€, which is way too much for the tests I have to do with 0€ budget (no profit). This long computation time means also that, if some parameters was wrong, I have to wait a long time (not mentioning the Mac is almost unuseful in this while). A demo of Maptiler cluster did the task on my i5 Linux laptop in less than two hours (with watermarks). A couple of days would be fine too.

I was searching the internet for an open source clone of the parallelized gdal2tiles, if existed, in order to save time.

Googling around I found your imgs2tiles project which seems very interesting, then "you redirected" me to geoimg_shredder. I didn't know anything about Erlang before (therefore I'm annoying you :P) and this seems a promising solution to parallelize the task, but Erlang is not as popular as other languages and (as other exotic languages) it takes time to be understood, even with guides.

Anyway the keys vector of the database after 2 hours of elaboration is empty (JSON output):

{
     "keys": [""]
}

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

refactor commented 12 years ago

发自我的 iPhone

在 2012-9-3，21:14，lamuertepeluda notifications@github.com 写道：

Hi,

the biggest problem with gdal2tiles.py is... it's sloooooowwww! :)

I have a machine (a powerful Mac Pro) running it with its GUI wrapper (from http://www.maptiler.org/) on this same map since June 27th (zoom levels 8-16), still there computing. The map is a very high detailed map coming from a GIS software and it represents just a region, not the whole globe, and that's why the source reference system is EPSG:32632 and not EPSG:900913. I need to make this map compatible with Google, in TMS format, and gdal2tiles would do the job.

I know there is an high parallelized version (Maptiler cluster) sold on the same site, but its price is about 2300�, which is way too much for the tests I have to do with 0� budget (no profit). This long computation time means also that, if some parameters was wrong, I have to wait a long time (not mentioning the Mac is almost unuseful in this while). A demo of Maptiler cluster did the task on my i5 Linux laptop in less than two hours (with watermarks). A couple of days would be fine too.

I was searching the internet for an open source clone of the parallelized gdal2tiles, if existed, in order to save time.

Googling around I found your imgs2tiles project which seems very interesting, then "you redirected" me to geoimg_shredder. I didn't know anything about Erlang before (therefore I'm annoying you :P) and this seems a promising solution to parallelize the task, but Erlang is not as popular as other languages and (as other exotic languages) it takes time to be understood, even with guides.

Anyway the keys vector of the database after 2 hours of elaboration is empty (JSON output):

{ "keys": [""] }

NO, the key SHOULD be increasing while the compute is processing.

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

— Reply to this email directly or view it on GitHub.

lamuertepeluda commented 12 years ago

发自我的 iPhone

???

NO, the key SHOULD be increasing while the compute is processing.

I guessed so... I'm not even able to post with curl, i.e.

curl -v -X PUT -d '{"bar":"baz"}' -H "Content-Type: application/json" -H "X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA==" http://127.0.0.1:8087/riak/test/doc?returnbody=true

also results in a empty key vector.

I think something's should be wrong in ryaks configuration. I'll dig into and let you know.

Thanks

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on

GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8239494.

refactor commented 12 years ago

eh... you want super fast gdal2tiles

I just make a test in my old 2006MacBook (2GHz intel Core Duo), 2G ddr2 memory, the tif img is 9983x10028 size, file size is 300MB, gdal2tiles take 325 sec, and geoimg_shredder 203 sec, to generate base tiles.

To generate overtime take another 67 sec ,by gdal2tile.

Another problem is i just finished generating base tiles, but generating overview tiles is the next work i havenot done.

So actually, I need more time to finished it.

发自我的 iPhone

在 2012-9-3，22:50，lamuertepeluda notifications@github.com 写道：

发自我的 iPhone

???

NO, the key SHOULD be increasing while the compute is processing.

I guessed so... I'm not even able to post with curl, i.e.

curl -v -X PUT -d '{"bar":"baz"}' -H "Content-Type: application/json" -H "X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA==" http://127.0.0.1:8087/riak/test/doc?returnbody=true

  • About to connect() to 127.0.0.1 port 8087 (#0)
  • Trying 127.0.0.1... connected

    PUT /riak/test/doc?returnbody=true HTTP/1.1 User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: 127.0.0.1:8087 Accept: / Content-Type: application/json X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA== Content-Length: 13

  • upload completely sent off: 13out of 13 bytes

also results in a empty key vector.

I think something's should be wrong in ryaks configuration. I'll dig into and let you know.

Thanks

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on

GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8239494.

— Reply to this email directly or view it on GitHub.

lamuertepeluda commented 12 years ago

I see... well may be my input file is just very big and the processing takes all that time. May be with your program I could reduce this time by 30% or so, not bad actually, but it's still a lot of time in my awful case! :) Anyway thank you for your great work and assistance. I'll continue monitoring geoimg_shredder to stay up to date! I wish I could help you but my Erlang knowledge by now is zero.

Vito

Il 03/09/2012 17:16, refactor ha scritto:

eh... you want super fast gdal2tiles

I just make a test in my old 2006MacBook (2GHz intel Core Duo), 2G ddr2 memory, the tif img is 9983x10028 size, file size is 300MB, gdal2tiles take 325 sec, and geoimg_shredder 203 sec, to generate base tiles.

To generate overtime take another 67 sec ,by gdal2tile.

Another problem is i just finished generating base tiles, but generating overview tiles is the next work i havenot done.

So actually, I need more time to finished it.

发自我的 iPhone

在 2012-9-3，22:50，lamuertepeluda notifications@github.com 写道：

发自我的 iPhone

???

NO, the key SHOULD be increasing while the compute is processing.

I guessed so... I'm not even able to post with curl, i.e.

curl -v -X PUT -d '{"bar":"baz"}' -H "Content-Type: application/json" -H "X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA==" http://127.0.0.1:8087/riak/test/doc?returnbody=true

  • About to connect() to 127.0.0.1 port 8087 (#0)
  • Trying 127.0.0.1... connected

    PUT /riak/test/doc?returnbody=true HTTP/1.1 User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: 127.0.0.1:8087 Accept: / Content-Type: application/json X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA== Content-Length: 13

  • upload completely sent off: 13out of 13 bytes

also results in a empty key vector.

I think something's should be wrong in ryaks configuration. I'll dig into and let you know.

Thanks

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on

GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8239494.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8240918.

refactor commented 12 years ago

In fact, there is nothing to config for riak, but don't run too many riak on one computer, if your PC as old and slow as mine.

The problem is likely u use port 8087, which seems to be riak's protocol buffer interface, the HTTP port is 8098

发自我的 iPhone

在 2012-9-3，22:50，lamuertepeluda notifications@github.com 写道：

发自我的 iPhone

???

NO, the key SHOULD be increasing while the compute is processing.

I guessed so... I'm not even able to post with curl, i.e.

curl -v -X PUT -d '{"bar":"baz"}' -H "Content-Type: application/json" -H "X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA==" http://127.0.0.1:8087/riak/test/doc?returnbody=true

  • About to connect() to 127.0.0.1 port 8087 (#0)
  • Trying 127.0.0.1... connected

    PUT /riak/test/doc?returnbody=true HTTP/1.1 User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3 Host: 127.0.0.1:8087 Accept: / Content-Type: application/json X-Riak-Vclock: a85hYGBgzGDKBVIszMk55zKYEhnzWBlKIniO8mUBAA== Content-Length: 13

  • upload completely sent off: 13out of 13 bytes

also results in a empty key vector.

I think something's should be wrong in ryaks configuration. I'll dig into and let you know.

Thanks

Perhaps it's just because the geoimg_shredder is still running and I have to wait until it finishes and commits everything to the DB?

Thanks

Vito

Il 03/09/2012 14:06, refactor ha scritto:

congratulations.

On Mon, Sep 3, 2012 at 6:56 PM, lamuertepeluda notifications@github.comwrote:

Hi, thanks for the instructions. After setting the image directory and installing riak it works (at first I thought it was just a dependency which would be downloaded automatically by rebar, but in fact it is a DB somehow similar to CouchDB, right?). I mean, I get a lot of output like this:

r.xoff: 5462, r.yoff: 43109, r.xsz: 136, r.ysz: 132 w.xoff: 0, w.yoff: 0, w.xsz: 1024, w.ysz: 993 bandscount: 3, CPLCalloc size=3050496

The output means it works now.

Do you think I could I write some javascript script to query all the pics in the DB and save them to file, ie. Node.js?

Do you mean you want to save them as files?

Do you know how to query riak for generated tiles key?

You can browse with this: localhost:8098/buckets/gis/keys?keys=true All keys will be display, as json text?

and using: localhost:8098/riak/gis/ to see the tile image.

My map image is a huge png (67355 x 43241) and its SRS is EPSG:32632, but I'd like to generate a EPSG:900913 system.

EPSG:900913 is global mercator projection. so there's nothing to do with projection.

Do you think I could reconfigure your program to use these parameters?

the webmercator(EPSG:900913) and global geodetic(EPSG:4326) is enough to me?

geoimg_shredder is just my part of work to build a GIS world in a cluster distributed computers. I just wonder what are you plan, sometimes gdal2tiles.py is enough for most of our work, isn't it?

Thank you

Vito

Il 03/09/2012 10:51, refactor ha scritto:

in geoimg_shredder.app ,there is a "imgdir" indicate the dir of img files, which include all gdal support format file.

global_geodetic means the img will be cut in geodetic(EPSG:4326) projection. there are only two projection supported: geodeotic and webmercator

the cut img will send and be stored in riak, which is NoSQL store, so we had to install riak first.

erl -pz ebin/ deps/*/ebin Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false] [systemtap]

Eshell V5.9.1 (abort with ^G) application:start(compiler). application:start(syntax_tools). application:start(lager). application:start(geo_utils). application:start(ppool). application:start(geoimg_shredder).

you had to indicate "imdir" in geoimg_shredder.app to the img directory first.

On Mon, Sep 3, 2012 at 4:01 PM, lamuertepeluda notifications@github.comwrote:

Line: img_picker:start_link({{"127.0.0.1", 8087}, global_geodetic).

of Readme seems to have one { too much, I think it should be:

img_picker:start_link({"127.0.0.1", 8087}, global_geodetic).

instead.

Btw: if you could give some more details, such as image types accepted (tiff, png), configuration options (what's global_geodetic, for instance) and directories (i.e. is ../tz the input and the output directory?).

Thanks :)

— Reply to this email directly or view it on GitHubhttps://github.com/refactor/geoimg_shredder/issues/1.

— Reply to this email directly or view it on GitHub <

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8232825>.

— Reply to this email directly or view it on

GitHubhttps://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8235468.

— Reply to this email directly or view it on GitHub

https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8236739.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/refactor/geoimg_shredder/issues/1#issuecomment-8239494.

— Reply to this email directly or view it on GitHub.