ipeaGIT / geobr

Easy access to official spatial data sets of Brazil in R and Python
https://ipeagit.github.io/geobr/
790 stars 118 forks source link

single progress bar for read functions? #42

Closed pedro-andrade-inpe closed 5 years ago

pedro-andrade-inpe commented 5 years ago

Function calls such as:

states <- read_state(year=2010, code_state = "all")

create one progress bar each time a state will be donloaded, summing up 27 progress bars in this case. Possibly creating one single progress bar that grows as each state is downloaded would be more interesting for the user.

rafapereirabr commented 5 years ago

Thanks for the suggestion, Pedro. As of today, the function has to separately download 27 files (one for each state) and then pile them up. For the moment, I don't know how to put a single progress bar to work in this case of downloading separate files.

Perhaps a compromise would be to add a 'counter' to all read_ functions. This is already implemented in read_municipality. Try for example read_municipality(code_muni = "all", year=2018). What do you think?

rafapereirabr commented 5 years ago

The simplest way I can think of how to implement this would be to incorporate the pbapply package as a dependency to geobr. The would only need to replace two lines of codes in some functions. For example, in the function read_meso_region(), we would have to replace lines 70 and 71 with this:

pbapply::pblapply(X=filesD, function(x){httr::GET(url=x, # httr::progress(),
                                       httr::write_disk(paste0(tempdir(),"/", unlist(lapply(strsplit(x,"/"),tail,n=1L))), overwrite = T))
                                       }
                                      )
rafapereirabr commented 5 years ago

Oh, and another simple alternative using the utils package. I think this solution is probably preferred because geobr already has dependency on utils, so this would be a minor change to the code. What do you think @pedro-andrade-inpe ?

total <- length(filesD)
pb <- utils::txtProgressBar(min = 0, max = total, style = 3)

lapply(X=filesD, function(x){ i <- match(c(x),filesD)

                              httr::GET(url=x, #httr::progress(),
                                        httr::write_disk(paste0(tempdir(),"/", unlist(lapply(strsplit(x,"/"),tail,n=1L))), overwrite = T))

                              utils::setTxtProgressBar(pb, i)
                              }
                            )
pedro-andrade-inpe commented 5 years ago

I like the solution with utils. I always close() the progress bar in the end of the loop (although I do not know if it is really necessary, the documentation says to do so).

close(pb)