Closed nichtich closed 8 years ago
As a start I created importer and exporter that can handle the Breaker format: https://github.com/LibreCat/Catmandu-Breaker
Then the examples are:
# Harvest a dataset in breaker format
$ catmandu convert OAI --url http://biblio.ugent.be/oai to Breaker > data.breaker
# Select all creators out of this data
$ catmandu convert Breaker to TSV --fix 'select all_match(tag,'creator'); retain(data)' < data.breaker
# Select all id and creators
$ catmandu convert Breaker to TSV --fix 'select all_match(tag,'creator'); retain(_id,data)' < data.breaker
I just stumbled upon the Code4Lib journal article Metadata Analysis at the Command-Line from around 3 years ago. It illustrates metadata processing with standard Unix tools (
grep
,sort
,uniq
...) and a custom scriptdc_breaker.py
availbale at https://github.com/vphill/metadata_breakers. We could go through the article and show how Catmandu can be used to achieve the same. If something does not work well, we spotted a gap in Catmandu based on this use-case.One feature of
dc_breaker.py
which is not fully available in Catmandu is a nice statistic of field usage. Catmandu::Stat does similar but not as nice visualized (could be done with another Exporter), less clear and with a limitation (https://github.com/LibreCat/Catmandu-Stat/issues/6).