Open maelle opened 2 years ago
Happy to help if needed!
@oggioniale
Dear Maëlle, apologise for my late replay! We working on re-structuring old functions and with a lot new functions.
In the next days we release the new version, also with the resolution of the error.
Thanks!!
Thank you!
Error I see in the vignette
siteGeneral <- ReLTER::get_site_info(
deimsid = "https://deims.org/f30007c4-8a6e-4f11-ab87-569db54638fe",
category = "General"
)
#> Warning: Unknown or uninitialised column: `generalInfo.keywords`.
#> Error in `colnames<-`(`*tmp*`, value = c("keywordsLabel", "keywordsURI")): attempt to set 'colnames' on an object with less than two dimensions
Created on 2022-10-11 with reprex v2.0.2
I have updated to the last version all packages and I can't reproduce the error.
> ReLTER::get_site_info(
+ deimsid = "https://deims.org/f30007c4-8a6e-4f11-ab87-569db54638fe",
+ category = "General"
+ )
# A tibble: 1 × 21
title uri geoCo…¹ country geoEl…² geoEl…³ geoEl…⁴ geoEl…⁵ gener…⁶ gener…⁷ gener…⁸ gener…⁹ gener…˟ gener…˟ gener…˟ gener…˟ gener…˟ gener…˟ gener…˟ gener…˟ gener…˟
<chr> <chr> <chr> <list> <dbl> <int> <int> <chr> <chr> <chr> <chr> <int> <lgl> <list> <lgl> <chr> <chr> <chr> <lgl> <lgl> <list>
1 Lago Mag… http… POINT … <chr> 194. 186 4633 msl "Lake … Operat… https:… 1960 NA <df> NA Lago M… IT08-0… Statio… NA NA <df>
# … with abbreviated variable names ¹geoCoord, ²geoElev.avg, ³geoElev.min, ⁴geoElev.max, ⁵geoElev.unit, ⁶generalInfo.abstract, ⁷generalInfo.status.label,
# ⁸generalInfo.status.uri, ⁹generalInfo.yearEstablished, ˟generalInfo.yearClosed, ˟generalInfo.hierarchy.parent, ˟generalInfo.hierarchy.children,
# ˟generalInfo.siteName, ˟generalInfo.shortName, ˟generalInfo.siteType, ˟generalInfo.protectionLevel, ˟generalInfo.landUse, ˟generalInfo.images
I still get the error, I added session info in case it is relevant. Does this function need authentication?
siteGeneral <- ReLTER::get_site_info(
deimsid = "https://deims.org/f30007c4-8a6e-4f11-ab87-569db54638fe",
category = "General"
)
#> Warning: Unknown or uninitialised column: `generalInfo.keywords`.
#> Error in `colnames<-`(`*tmp*`, value = c("keywordsLabel", "keywordsURI")): attempt to set 'colnames' on an object with less than two dimensions
Created on 2022-10-20 with reprex v2.0.2
You could pre-build the vignette https://ropensci.org/blog/2019/12/08/precompute-vignettes/ (and set a reminder for yourself in a release_bullets()
function cf https://usethis.r-lib.org/reference/use_release_issue.html#ref-usage, example https://github.com/lucarraro/traudem/blob/2038f221d3197674b62dbb20da22fdb16991e15c/R/utils.R#L15)
But we should still find out why I can't run the code I suppose :sweat_smile:
thanks!
Please let me know what is the content of the get_site_general()
function?
Because the past version of this function contains the "keywordsLabel", "keywordsURI" as some column names of the output. Currently is:
get_site_general <- function(deimsid) {
qo <- queries_jq[[get_deims_API_version()]]$site_general
jj <- get_id(deimsid, qo$path)
if (is.na(attr(jj, "status"))) {
invisible(
utils::capture.output(
general <- dplyr::as_tibble(do_Q(qo$query, jj))
)
)
} else {
message("\n----\nThe requested page could not be found.
Please check again the DEIMS ID\n----\n")
general <- NULL
}
general
}
And please can execute also ReLTER::get_deims_API_version()
and ReLTER::get_deims_base_url()
. The output should be:
> ReLTER::get_deims_API_version()
[1] "1.1"
> ReLTER::get_deims_base_url()
[1] "https://deims.org/"
>
Did you forget to push updates? These functions aren't exported (I'm using a clone of this repository)
I push all currently in the dev branch. May be is this the problem? Do you have clone main branch?
yes I was using the main branch, and that is what the pkgdown building infrastructure uses.
Ahhh ok the error is about the function because I don't already done the PR from dev to main.
I think to solve all the issue (e.g. ERROR: dependency ‘SPARQL’ is not available for package ReLTER) in the dev branch and after merge this in the main.
Isn't that right?
Now I also get the error about SPARQL. It has been archived on CRAN https://cran.r-project.org/web/packages/SPARQL/
However see https://discuss.ropensci.org/t/optional-suggests-dependency-that-has-been-archived-on-cran/3071
I think to solve all the issue (e.g. ERROR: dependency ‘SPARQL’ is not available for package ReLTER) in the dev branch and after merge this in the main.
yes it makes sense!
Regarding SPARQL you could also stop using it by making the API calls with httr2 or httr, example in https://github.com/lvaudor/glitter/blob/3fe7c9f90258a4ac9892f832a05da9e5b1d6e8e1/R/send_sparql.R#L34 (please tell me if you'd prefer me to make a PR, or have any question!)
do you have some suggestion about this error?
We're both too fast answering :joy:
So I think the best would be to remove the dependency on SPARQL by using httr2 or httr see https://github.com/ropensci/ReLTER/issues/62#issuecomment-1285715579
In that package I'm giving as an example we had removed the SPARQL dependency, reasoning in https://github.com/lvaudor/glitter/issues/19
(httr might be a better choice since ReLTER already imports it, unless you switch everything to httr2 :zany_face: )
Yes httr is already imported.
But before to remove SPARQL and use httr, can you tell me how to use httr in this case (get_sensor_observed_properties()
function):
library(SPARQL)
ireaEndpoint <- "http://fuseki1.get-it.it/directory/query"
ireaQuery <- paste0(
"PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX qudt: <http://qudt.org/schema/qudt/>
SELECT ?c ?l ?code ?s
WHERE {
SERVICE <http://vocab.nerc.ac.uk/sparql/sparql> {
?c rdf:type skos:Concept .
<http://vocab.nerc.ac.uk/collection/P06/current/> skos:member ?c .
OPTIONAL {
?c skos:altLabel ?l .
?c owl:sameAs ?s .
}
FILTER(?l = 'degC')
FILTER(REGEX(STR(?s), 'qudt'))
}
SERVICE <https://www.qudt.org/fuseki/qudt/query> {
?s qudt:udunitsCode ?code
}
}
ORDER BY ASC(?l)
LIMIT 1"
)
qudtUOM <- SPARQL::SPARQL(
url = ireaEndpoint,
query = ireaQuery,
curl_args = list(.encoding = "UTF-8")
)
qudtUOM
I want understand because I am a fan of sparql queries. 🤪
Here's a reprex with both httr2 and httr solutions (I haven't checked the encoding, but it might help as a starting point)
ireaEndpoint <- "http://fuseki1.get-it.it/directory/query"
ireaQuery <- paste0(
"PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX qudt: <http://qudt.org/schema/qudt/>
SELECT ?c ?l ?code ?s
WHERE {
SERVICE <http://vocab.nerc.ac.uk/sparql/sparql> {
?c rdf:type skos:Concept .
<http://vocab.nerc.ac.uk/collection/P06/current/> skos:member ?c .
OPTIONAL {
?c skos:altLabel ?l .
?c owl:sameAs ?s .
}
FILTER(?l = 'degC')
FILTER(REGEX(STR(?s), 'qudt'))
}
SERVICE <https://www.qudt.org/fuseki/qudt/query> {
?s qudt:udunitsCode ?code
}
}
ORDER BY ASC(?l)
LIMIT 1"
)
library("magrittr")
resp <- httr2::request(ireaEndpoint) %>%
httr2::req_url_query(query = ireaQuery) %>%
httr2::req_method("POST") %>%
httr2::req_headers(Accept = "application/sparql-results+json") %>%
httr2::req_user_agent("ReLTER dev") %>%
httr2::req_retry(max_tries = 3, max_seconds = 120) %>%
httr2::req_perform()
httr2::resp_check_status(resp)
httr2::resp_body_json(resp)
#> $head
#> $head$vars
#> $head$vars[[1]]
#> [1] "c"
#>
#> $head$vars[[2]]
#> [1] "l"
#>
#> $head$vars[[3]]
#> [1] "code"
#>
#> $head$vars[[4]]
#> [1] "s"
#>
#>
#>
#> $results
#> $results$bindings
#> $results$bindings[[1]]
#> $results$bindings[[1]]$c
#> $results$bindings[[1]]$c$type
#> [1] "uri"
#>
#> $results$bindings[[1]]$c$value
#> [1] "http://vocab.nerc.ac.uk/collection/P06/current/UPAA/"
#>
#>
#> $results$bindings[[1]]$l
#> $results$bindings[[1]]$l$type
#> [1] "literal"
#>
#> $results$bindings[[1]]$l$value
#> [1] "degC"
#>
#>
#> $results$bindings[[1]]$code
#> $results$bindings[[1]]$code$type
#> [1] "literal"
#>
#> $results$bindings[[1]]$code$value
#> [1] "°C"
#>
#>
#> $results$bindings[[1]]$s
#> $results$bindings[[1]]$s$type
#> [1] "uri"
#>
#> $results$bindings[[1]]$s$value
#> [1] "http://qudt.org/vocab/unit/DEG_C"
qudtUOM <- httr::GET(
url = ireaEndpoint,
query = list(query = ireaQuery)
)
httr::parsed_content(qudtUOM)
#> text_content() deprecated. Use content(x, as = 'parsed')
#> $head
#> $head$vars
#> $head$vars[[1]]
#> [1] "c"
#>
#> $head$vars[[2]]
#> [1] "l"
#>
#> $head$vars[[3]]
#> [1] "code"
#>
#> $head$vars[[4]]
#> [1] "s"
#>
#>
#>
#> $results
#> $results$bindings
#> $results$bindings[[1]]
#> $results$bindings[[1]]$c
#> $results$bindings[[1]]$c$type
#> [1] "uri"
#>
#> $results$bindings[[1]]$c$value
#> [1] "http://vocab.nerc.ac.uk/collection/P06/current/UPAA/"
#>
#>
#> $results$bindings[[1]]$l
#> $results$bindings[[1]]$l$type
#> [1] "literal"
#>
#> $results$bindings[[1]]$l$value
#> [1] "degC"
#>
#>
#> $results$bindings[[1]]$code
#> $results$bindings[[1]]$code$type
#> [1] "literal"
#>
#> $results$bindings[[1]]$code$value
#> [1] "°C"
#>
#>
#> $results$bindings[[1]]$s
#> $results$bindings[[1]]$s$type
#> [1] "uri"
#>
#> $results$bindings[[1]]$s$value
#> [1] "http://qudt.org/vocab/unit/DEG_C"
Created on 2022-10-20 with reprex v2.0.2
(as a side note you might be interested in https://github.com/lvaudor/glitter since you like SPARQL queries :wink:)
(as a side note you might be interested in https://github.com/lvaudor/glitter since you like SPARQL queries 😉)
(I had seen this package, but didn't have time to look at it. Better than httr or httr2?)
Here's a reprex with both httr2 and httr solutions (I haven't checked the encoding, but it might help as a starting point)
ireaEndpoint <- "http://fuseki1.get-it.it/directory/query" ireaQuery <- paste0( "PREFIX owl: <http://www.w3.org/2002/07/owl#> PREFIX skos: <http://www.w3.org/2004/02/skos/core#> PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX qudt: <http://qudt.org/schema/qudt/> SELECT ?c ?l ?code ?s WHERE { SERVICE <http://vocab.nerc.ac.uk/sparql/sparql> { ?c rdf:type skos:Concept . <http://vocab.nerc.ac.uk/collection/P06/current/> skos:member ?c . OPTIONAL { ?c skos:altLabel ?l . ?c owl:sameAs ?s . } FILTER(?l = 'degC') FILTER(REGEX(STR(?s), 'qudt')) } SERVICE <https://www.qudt.org/fuseki/qudt/query> { ?s qudt:udunitsCode ?code } } ORDER BY ASC(?l) LIMIT 1" ) library("magrittr") resp <- httr2::request(ireaEndpoint) %>% httr2::req_url_query(query = ireaQuery) %>% httr2::req_method("POST") %>% httr2::req_headers(Accept = "application/sparql-results+json") %>% httr2::req_user_agent("ReLTER dev") %>% httr2::req_retry(max_tries = 3, max_seconds = 120) %>% httr2::req_perform() httr2::resp_check_status(resp) httr2::resp_body_json(resp) #> $head #> $head$vars #> $head$vars[[1]] #> [1] "c" #> #> $head$vars[[2]] #> [1] "l" #> #> $head$vars[[3]] #> [1] "code" #> #> $head$vars[[4]] #> [1] "s" #> #> #> #> $results #> $results$bindings #> $results$bindings[[1]] #> $results$bindings[[1]]$c #> $results$bindings[[1]]$c$type #> [1] "uri" #> #> $results$bindings[[1]]$c$value #> [1] "http://vocab.nerc.ac.uk/collection/P06/current/UPAA/" #> #> #> $results$bindings[[1]]$l #> $results$bindings[[1]]$l$type #> [1] "literal" #> #> $results$bindings[[1]]$l$value #> [1] "degC" #> #> #> $results$bindings[[1]]$code #> $results$bindings[[1]]$code$type #> [1] "literal" #> #> $results$bindings[[1]]$code$value #> [1] "°C" #> #> #> $results$bindings[[1]]$s #> $results$bindings[[1]]$s$type #> [1] "uri" #> #> $results$bindings[[1]]$s$value #> [1] "http://qudt.org/vocab/unit/DEG_C" qudtUOM <- httr::GET( url = ireaEndpoint, query = list(query = ireaQuery) ) httr::parsed_content(qudtUOM) #> text_content() deprecated. Use content(x, as = 'parsed') #> $head #> $head$vars #> $head$vars[[1]] #> [1] "c" #> #> $head$vars[[2]] #> [1] "l" #> #> $head$vars[[3]] #> [1] "code" #> #> $head$vars[[4]] #> [1] "s" #> #> #> #> $results #> $results$bindings #> $results$bindings[[1]] #> $results$bindings[[1]]$c #> $results$bindings[[1]]$c$type #> [1] "uri" #> #> $results$bindings[[1]]$c$value #> [1] "http://vocab.nerc.ac.uk/collection/P06/current/UPAA/" #> #> #> $results$bindings[[1]]$l #> $results$bindings[[1]]$l$type #> [1] "literal" #> #> $results$bindings[[1]]$l$value #> [1] "degC" #> #> #> $results$bindings[[1]]$code #> $results$bindings[[1]]$code$type #> [1] "literal" #> #> $results$bindings[[1]]$code$value #> [1] "°C" #> #> #> $results$bindings[[1]]$s #> $results$bindings[[1]]$s$type #> [1] "uri" #> #> $results$bindings[[1]]$s$value #> [1] "http://qudt.org/vocab/unit/DEG_C"
Created on 2022-10-20 with reprex v2.0.2
sounds very good!!
... and could be possible also obtain a tibble as qudtUOM
?
(I had seen this package, but didn't have time to look at it. Better than httr or httr2?)
It is not a replacement for them, it's a package (not stable yet) that should help write SPARQL queries (well in your case you already know, but that can also help people who read the code)
and could be possible also obtain a tibble as qudtUOM?
not via httr/httr2, at least I don't think so? which means, you'd need to add parsing code :thinking:
yes for sure I can parse the code ... just to know
I have changed the code using httr2 with this commit a72e1ee004749bfcb8730da51fd5670927f9be24
Cool!
Is the number of Imports becoming too high? CRAN has a limit, and you'd get a NOTE. Maybe the other httr code can be switched to httr2?
currently the imported packages are 45. Is it more the limit of CRAN?
The httr code has been wrote by my colleague and I would like to wait for changes or suggestions from him.
currently the imported packages are 45. Is it more the limit of CRAN?
oh yeah definitely, it's about 20 if I remember correctly.
the limit is for packages listed in Imports
, not Suggests
the limit is for packages listed in
Imports
, notSuggests
45 are only in the Imports
. Suggests
packages are only 7.
How can I actually know whether a package is imported or suggested? Just from the fact that it is used in a function? In that case, do I have to import it and not suggests it?
imported = listed in Imports.
Now, a package has to be listed in Imports when you call it from your code.
I'd suggest reading https://r-pkgs.org/dependencies.html
To limit the number of dependencies, you might need to
purrr::map()
but which to remove the purrr dependency you could be inspired by https://github.com/r-lib/rlang/blob/main/R/compat-purrr.R (rlang licence is MIT)rlang::check_installed("blop")
before using the blop package.Hi @oggioniale
Here are some ideas of packages that we might be able to remove:
leaflet (if we switch all map plots to tmap
)
rgeos (probably not necessary)
lubridate (nice to have, but we might get by with base R)
tidyr and dplyr and dtplyr and magrittr ? - these could be consolidated
rworldmap and rnaturealearth. One should be enough
qrcode - How important is the produce_site_qrcode
function? What is the purpose?
ggforce - Really required?
Best regards, Micha
Hi @micha-silver I think it is a very accurate analysis. Thank you!!!
We have to work in terms of consolidation. I would now like to arrive at a new and final version with the new functions and then think about consolidation and harmonisation in the use of packages.
Someone have idea about the rarer in the actions (https://github.com/ropensci/ReLTER/actions/runs/3378798361)? Because if we don't get past this step, I wouldn't dare make any more changes.
Hi @oggioniale
I see the errors in the Ubuntu builds have to do with openssl. I think there have been some updates to that software lately. It might be that we are temporarily between versions. Maybe after a day or two, when the repositories get synchronized, those errors will go away.??
If I understand, we are working on the dev
branch? Should I wait till the above errors are fixed before pushing any new changes?
:wave: @oggioniale @micha-silver is any help needed here?
Dear @micha-silver,
yes we working on the dev
branch and we have a lot of changes in the package.
We still working to solve some issues at the moment.
wave @oggioniale @micha-silver Happy New Year! Is any help needed?
Hi @maelle, Happy New Year to you too.
Nothing at the moment. We will need your support when we will reduce the number of packages imported.
I thinking … soon
Hi @oggioniale One idea came to my mind this week: We have a minimal R version 3.5. And we need to import magrittr just for the '%>%' pipe operator. If we up the minimum R version to 4.1, then we can change to the new built in '|>' operator, and drop the magrittr dependency. But let's wait till after the workshop ;-)
@maelle Happy holiday to you as well
Hello @oggioniale @micha-silver, any update? :smile_cat:
:wave: @oggioniale @micha-silver
wave @oggioniale @micha-silver
:wave: @oggioniale @micha-silver -- what's the current status? Any help needed?