Suggestion: Add a section on spatial metadata handling

[eblondel]

Hi @Robinlovelace @Nowosad , we've been liaising regarding ows4R and how to grab data using OGC web standards. I'm wondering if you would be interested to add a section for spatial metadata handling. Metadata handling is still an obscure side for many people (and quite new in the R community), but its interest is growing a lot, being a huge task of scientific data managers, especially in the context of FAIR data principles (Findable / Accessible / Interoperable / Reusable).

If it's of interest, i'd happy to contribute on that. Spatial metadata handling is covered is different packages right now.

• In ncdf4 package (dealing with NetCDF-CF format), in which we can extract metadata information (NetCDF-CF format embedds metadata). In raster package as well we can extract metadata embedded in raster files.
• EML/emld packages (dealing with Ecological Metadata Language - EML), indeed specific to biodiversity community, but with a strong spatial footprint, as inspired from ISO/OGC geographic information metadata.
• geometa (that I authored) which deals with writing/reading/validating ISO/OGC geographic metadata, and since soon spatial metadata converters from the above other geographic metadata standards. ows4R, that you already use in your book (for WFS data retrieval), supports management of geographic metadata handled with geometa (including publishing metadata on the web).

Let me know if it could be interesting,

Cheers,

PS: I'm at user!2019, we may have a talk on that

[Robinlovelace]

Hi @eblondel, many thanks for this suggestion. I think that metadata is certainly an under-appreciated aspect of geographic data processing. I think the first issue is that many people (myself included!) do not have a clear idea of whate geographic metadata is / is not. Would it include, for example, the output from st_layers(), or is it more about standards? I see some reproducible code in the wiki of the geometa package: https://github.com/eblondel/geometa/wiki

But must confess, I'm not 100% sure what it does. In summary, I'd very much welcome a section on geographic metadata and think it would strengthen the book. Will await comments from @Nowosad and @jannes-m before confirming, however.

All the best, and sorry our paths didn't cross at UseR, was hectic, as I documented in this write-up from the event (great to see lots of R-spatial there): https://www.robinlovelace.net/2019/07/13/user2019/

[Nowosad]

Hi all. @Robinlovelace - I met @eblondel during the last day of UseR!2019, and we have a nice discussion about spatial metadata.

I agree with everything @Robinlovelace wrote above. @eblondel would it be possible for you to write a short paragraph or two about spatial metadata? E.g. (1) what's that, (2) why should we use them, (3) how to set them, (4) how to read/edit them. We could next decide if it would be a good fit to the book (and where should we put it).

[eblondel]

Hi @Nowosad i'm fine with your proposal. I will work on 1/2 paragraphs to introduce about spatial metadata and the R perspective. Thanks both for your answers

[Robinlovelace]

Hi @eblondel do you have any updates on this? Thanks!

[eblondel]

Hi @Robinlovelace I've been busy and now even more, but I will get it ASAP. Let me know if there is some deadline for you to get this. Cheers

[Robinlovelace]

No deadline, just checking you still wanted to add something on this. Thanks.

[Robinlovelace]

Hi @eblondel, just looking through issues, now is a great time to think about this because we've started work on the 2nd edition. How does end of the year sound as a provisional deadline? Thanks.

[eblondel]

Sounds good! Thanks for recalling this

Le mar. 7 sept. 2021 22:41, Robin @.***> a écrit :

Hi @eblondel https://github.com/eblondel, just looking through issues, now is a great time to think about this because we've started work on the 2nd edition. How does end of the year sound as a provisional deadline? Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Robinlovelace/geocompr/issues/419#issuecomment-914615979, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKDK3BDOVMPU7XHR7FJKNTUAZ2H5ANCNFSM4ICGKJZA .

[Robinlovelace]

Hi @eblondel apologies for another nudge but any chance of working on this in the next month or so? Will be good to get your feedback before we finalise the updated version of Part 1 (chapters 1-8).

[eblondel]

@Robinlovelace do you have specific requirements ? maximum number of words? Do you plan an extended section on Geospatial data services? ows4R has lot more that would deserve mentioning in your book.

[Robinlovelace]

Hi @eblondel thanks for the response. No max. number of words but I would say no more than ~3 paragraphs. I think a section on data services, with references to ows4 could be good in this section: https://geocompr.robinlovelace.net/read-write.html#geographic-web-services

Let us know if you want a pointer to the source code on that, and any suggested changes to that section, that was written a while back, welcome.

A broader question: to me it looks like C8 has become pretty big. I wonder if there would be benefit of splitting out some of the stuff, e.g. sections below, into another chapter as we did with vec-raster interactions. Thoughts @Nowosad and @jannes-m ?

[eblondel]

@Robinlovelace I will do my best to do the summary on metadata by early next week. If you can give me pointers to the source code, for that, and also for the geographic web services, that needs some changes, including the examples provided.

On this part of OGC services, the WPS is now supported in ows4R. I'm also now adding the support to WCS (OGC service to get raster/coverage data, as parallel with the WFS for vector data). This will be ready in the coming months. If you can tell me your timeline to have this section ready, I could push further on getting the WCS ready, to make a section in the book. This would be great asset for the users.

[Robinlovelace]

If you can give me pointers to the source code, for that, and also for the geographic web services, that needs some changes, including the examples provided.

Sure, here is the source code: https://github.com/Robinlovelace/geocompr/blob/main/08-read-write-plot.Rmd#L218 If you can edit that file and put in a PR that would be most appreciated :pray:

On this part of OGC services, the WPS is now supported in ows4R. I'm also now adding the support to WCS (OGC service to get raster/coverage data, as parallel with the WFS for vector data). This will be ready in the coming months. If you can tell me your timeline to have this section ready, I could push further on getting the WCS ready, to make a section in the book. This would be great asset for the users.

That is great to hear. WPS is pretty powerful, allowing you to send remote operations to run on remote servers, right? Impressive if we can do that from R! Timeline for second edition is complete manuscript by summer so good if ready by then. However, please focus on a simple 'minimum viable product' section in the first PR. We can add more detail + advanced topics like WPS later is my thinking.

[eblondel]

@Robinlovelace sorry to come late, I can dedicate time this week to write a section for spatial metadata. I would like to extend the ows4R section a bit to cover the new standard services supported now. I will do 2 distinct PRs PS: the dependency on rgdal in ows4R is going to removed

[Robinlovelace]

Fantastic, many thanks Emmanuel. Good timing to pick up on this, with plan to revert #825 (due to wanting to get the book building again) we can 'kill two birds with one stone', hopefully this provides a good motivation to remove rgdal dep.

[Robinlovelace]

Hi @eblondel please can you try to get this in by end of the year as we're looking to wrap up the 2nd edition.

[eblondel]

Hi @Robinlovelace How/where do you want me to contribute this? Another thing, as explained here https://github.com/geocompx/geocompr/issues/419#issuecomment-1042734918, there is much more supported in ows4R, this includes the WFS, but also the WCS (for raster/coverage) and the WPS (for processing). Would you mind if I create a pull request for reviewing the section on OGC web-services?

[Robinlovelace]

That would be great thanks @eblondel, I'll point you towards the relevant bit.

[Robinlovelace]

Anything in this section v. welcome, as you'll see there's plenty of comments from us that you can ignore/remove at your leisure: https://github.com/geocompx/geocompr/blob/ba9111a51c2693a16d0a5497d2c82053dc145069/08-read-write-plot.Rmd#L210

[eblondel]

@Robinlovelace i'm working on it. I will proceed with a review/update of the web-services part, and suggest a new section on geographic metadata handling. In case you wish, we can pin a call to discuss it.

[eblondel]

@Robinlovelace for the geographic web services, see a review/update done with this PR https://github.com/geocompx/geocompr/pull/1028

For the metadata handling (initial topic of this issue), i'm writing a text, however by reading the full chapter, I would suggest to refactor a bit the sections, if possible. I would suggest the following:

• handle the "File Formats" before the "Geographic Web-Services". Indeed there you introduced already the OGC. We may have so redundancy with what is writen with the services part.
• split the File Formats between Data (with subsection on Input / Output) and Metadata, where I could introduce the geographic metadata handling with R.
• (maybe) handle the geographic data packages, at the end, as it deals more with some utilities to access common datasets. If you wish we can discuss a bit, and based on that I will create a 2d PR for the metadata part.

[Robinlovelace]

Thanks for the comments @eblondel I've taken a look at your PR. Regarding your wider comments will aim to have a think and act on this later this week.

[eblondel]

Great, thanks for your prompt review. As soon as I have some text for the metadata I will share it with you (sure this week).

[eblondel]

I've created a 2d PR (from 'main') for the metadata handling proposal at #1029 . I've just grabed another section, but IMHO it may deserve some restructuring, e.g. -Intro (that might need to be broaden to cover data/metadata)

• Data handling
  • Vector data
  • Raster data
  • dataset packages
  • data visualization
  • Metadata handling (cf. #1029)
  • Geographic web-services (cf. #1028) - that cover both data/metadata In any case, let me know if you want to talk on this, i'll be available Cheers

[Robinlovelace]

Great work, looking at this now...