LEEClab / LS_METRICS

A tool for calculating landscape connectivity and other ecologically scaled landscape metrics
GNU General Public License v2.0
22 stars 9 forks source link

LS Metrics statistic text files outputs do not exported (big rasters). #5

Open rafaguimaraess opened 5 years ago

rafaguimaraess commented 5 years ago

Hello,

I have been working with Cerrado biome and the enhanced LS Metrics (new version) for some landscape metrics (patch size, functional connectivity). The problem is: the rasters outputs were exported to the folder, but the statistic text files not. I would like to know what it is probably happening because the LS Metrics keeps going on processing even after the rasters maps were exported.

Obs.: Raster input size (columns and rows): 66977, 85490

Thanks for the package. It has been useful mainly with big landscapes!

Best wishes,

Rafaela

bniebuhr commented 5 years ago

Hi @rafaguimaraess Thanks for your report. This behavior is strange, since it seems that this functionality runs for small maps (or even for other large maps we have used as input). But I'll test that with the dataset you sent me and try to find out what is happening.

Best Bernardo

JohnWRRC commented 5 years ago

Hello, Bernardo, the site size is very large 66k lines by 80k columns. This takes time to process. Rafaela, what is the solution you are working on? Have you tried to process a smaller sample with your own test data?

"I'd like to know what's probably happening because LS Metrics continue to process even after the rasters maps have been exported" Can you specify this better? It may not have reached the end of the process and so TXT is not in the folder.

I would like to receive the data too

John

Em seg, 12 de nov de 2018 às 12:40, rafaguimaraess notifications@github.com escreveu:

Hello,

I have been working with Cerrado biome and the enhanced LS Metrics (new version) for some landscape metrics (patch size, functional connectivity). The problem is: the rasters outputs were exported to the folder, but the statistic text files not. I would like to know what it is probably happening because the LS Metrics keeps going on processing even after the rasters maps were exported.

Obs.: Raster input size (columns and rows): 66977, 85490

Thanks for the package. It has been useful mainly with big landscapes!

Best wishes,

Rafaela

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/LEEClab/LS_METRICS/issues/5, or mute the thread https://github.com/notifications/unsubscribe-auth/ALAkp5Dniml6pqnaoo_6T1_Gz9e2Egnqks5uuYhTgaJpZM4YZx_9 .

--

John Wesley Ribeiro *Desenvolvedor e *Analista em Geotecnologias Laboratório de Ecologia Espacial e Conservação-LEEC : link http://www.leec.eco.br/ Fones: 30242284 LATTES:Lattes: http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4889487Y8 http://buscatextual.cnpq.br/buscatextual/visualizacv.do?id=K4889487Y8

bniebuhr commented 5 years ago

Hello, Bernardo, the site size is very large 66k lines by 80k columns. This takes time to process. Rafaela, what is the solution you are working on? Have you tried to process a smaller sample with your own test data?

John, I talked to Rafaela and, in principle, there is no limitation in GRASS or LSMetrics for calculating metrics for such a map (although it takes time). Therefore, I think the issue is not exactly the size.

"I'd like to know what's probably happening because LS Metrics continue to process even after the rasters maps have been exported" Can you specify this better? It may not have reached the end of the process and so TXT is not in the folder.

That's a good hypothesis. Rafaela, are you sure the process got to an end (did the user interface tell you that the "calculations have finished")?

I would like to receive the data too Rafela sent me that in private, I'll forward it to you, John.

The strange thing is that the process works for smaller regions or maps, even in her computer. But it would be good to test that here and there.

As a temporary solution to her problem, she ran r.stats after processing the maps; however, our problem is not solved yet.

B

rafaguimaraess commented 5 years ago

Hello John and Bernardo,

After the LS Metrics download (corrected version), I started to test the package choosing a metric once a time (patch size). The maps outputs had been exported in two hours while text files hadn't exported in 7 days. So, I stopped the process (before the message "calculations have finished") because I just decided to test the package with a small landscape and got it well (the maps and text files exported). It seems as the LS Metrics are processing more slowly to export text files than rasters maps. This "behavior" is so strange for me because I expect the opposite. The function "r.stats" which I tested with the maps outputs (patch size) worked well and fastly (15minutes).

Which email can I share the input?

Thanks for help and recommendations.

Best

Rafaela

PC features: Intel Core I7 2.4GHz, 8 GB RAM, 1TB HD, GeForce 2GB (dedicated).

bniebuhr commented 5 years ago

Hi @rafaguimaraess

How have you been? I guess you already solved this problem, but I think I found out the reason why the text file export was not working. We created a function to calculate areas in hectares, based on r.stats, but I just found out the way it saves the information in the text file is completely inefficient. For maps with small number of patches this is not an issue, but for large extent maps such as yours, this is a problem.

r.stats, in turn, is much more efficient in writing the output file.

I'll find out some way of dealing with that and I keep you (and other users) informed here.

Best, Bernardo

rafaguimaraess commented 5 years ago

Hi @bniebuhr,

I'm glad to know about the solution. I solved the problems with your suggestion of using r.stats. Thanks for your help.

Best,

Rafaela