Jefferson49 / RepositoryHierarchy

A weebtrees 2.1 custom module to present the structure of a repository and its sources in a hierarchical manner.
GNU General Public License v3.0
6 stars 3 forks source link
archive-management custom-module ead php webtrees webtrees-module webtrees2-1

Latest Release webtrees major version

Repository Hierarchy

A weebtrees custom module to present the structure of a repository and its sources in a hierarchical manner. Based on the hierarchical structure, a finding aid document (HTML or PDF export) can be generated. The module also provides an EAD XML export, which enables data exchange and linking with an external archive management system. The module also includes additional features to improve the handling of sources, source citations, and media objects.

The module uses delimiters to cut call numbers (of sources) into sub-strings and extracts call number categories. Based on the extracted categories, a hierarchical tree of call number categories with the related sources is constructed and shown.

Example call numbers:

Delimiter: " / "

Resulting repository hierarchy:

To specify delimiters, the module can handle single characters as well as complex regular expressions:

Screenshot

Screenshot

Table of contents

This README file contains the following main sections:What](

What are the benefits of using this module?

Installation

Webtrees version

The latest release of the module was developed and tested with webtrees 2.1.18, but should also run with any other webtrees 2.1 version.

Concepts of the Repository Hierarchy module

In the following, the concepts of the Repository Hierarchy module are described.

Call number categories

In the module, a new concept "Call numbers category" is introduced. Call number categories are defined as hierarchical elements, which constitute the structure of an archival arrangement.

Relationship between call number categories, call numbers, and delimiters

Call number categories are extracted form call numbers. The module identifies sub-strings in call numbers as call number categories by using delimiters. A chosen delimiter (or a set of delimiters) cuts the full call number into sub-strings of call number categories.

Example call number structure: "Fonds/Record group/Series/Folder/Source"

In this case, the module identifies the following strings as call number categories:

Based on the identified call number categories, the module creates the following hierarchical structure for the archive:

Delimiter expressions for call numbers

A delimiter is a sequence of one or more characters for specifying the boundary between separate, independent regions in a text. In the Repository Hierarchy module, delimiters are used to cut call numbers into sub-strings of call number categories. The call number categories will be used to construct a hierarchy of call numbers.

How to use the module?

Usage of a single delimiter

A single delimiter is used by providing a single character or a sequence of characters in the related input form ("delimiter expression").

Example:

Usage of a set of delimiters

A set of delimiters is used by providing the delimiters in the input form ("delimiter expression") separated by ";".

Example:

In a set of delimiters, the delimiters are evaluated from left to right, i.e. the most left delimiter is evaluated first. Delimiters will also be applied recursively for as many matches as possible. Only if no further matches of a delimiter are found, the next delimter is evaluated.

Example:

I.e. the "-" delimiter in "Record-group" is not evaluated, because the "/" delimiter is evaluated first. After all matches of the "/" delimiter have been evaluated, the "-" delimiter is found in "Series A-Nr. 7".

Usage of a regular expression for the delimiter

A regular expression is used by providing it in the input form ("delimiter expression"). The regular expression needs to contain the delimiter in brackets. This provides a much more powerful way to specify delimiters.

Please note, that the "full" regular expression will be used to find a certain pattern in the call numbers. However, only the characters in the brackets ("the match" of the regular expression) will be used as the delimiter.

Example:

In this example, the delimiter is the space character in the brackets, i.e. "( )". However, the full pattern "Film( )Number" is used to find corresponding strings. Therefore, only space characters, which match the pattern, are identified as delimiter. Other space characters, are NOT identified as delimiter.

Usage of a set of regular expressions for the delimiter

A set of regular expressions can be used by providing several regular expressions in the input form ("delimiter expression") separated by ";". It is also possible to mix simple delimiters (i.e. a single character or sequence of characters) with regular expressions.

Example:

Like for a set of simple delimiters, the delimiter expressions are evaluated from left to right. Please refer to the description and example for a set of simple delimiters.

Save and load options

By pressing certain radio buttons in the front end, certain load and save operations can be executed while pressing the "view" button.

Save and load a repository

If the "save repository" radio button is activated while the "view" button is pressed, the currently selected repository will be stored for the active user.

If the "load repository" radio button is activated while the "view" button is pressed, the module will load a stored repository of the user if already stored.

Save and load a delimiter expression

If the "save delimiter expression" radio button is activated while the "view" button is pressed, the current delimiter expression will be stored for the active user. If the user is administrator, the expression will also (parallely) be stored as adminstrators' delimiter expression.

If the "load delimiter expression" radio button is activated while the "view" button is pressed, the module will load a stored delimiter expression of the user if already stored.

If the "load delimiter expression from administrator" radio button is activated while the "view" button is pressed, the module will load a stored delimiter expression of the administrator(s) if available.

Rename a call number category

By opening the "Rename" link close to a call number category, a data fix page with a search/replace form is opened, where the name of the chosen call number can be modified.

Add a new source to a call number category

By opening the "Add new source" link close to a call number category, a form is opened, which allows to add a new source to the chosen call number. When opening the form, a "{new}" placeholder is inserted, which should be modified by the user.

While the "{new}" placeholder should be modified, the rest of the call number, which consists of the call number category hierarchy should only be modified if the "route" or "path" of the call number category shall also be changed. If the intention is to simple add a new source to an existing call number category, only the "{new}" placeholder should be changed.

Provide and show titles for call number categories

While the Repository Hierarchy module can generate call number categories from call numbers, the webtrees and GEDCOM data model does not provide features to describe the call number categories. To fill this gap, the Repository Hierarchy module offers a possibility to describe and view titles for the call number categories.

The following two screenshots show examples for the titles in the module browser front end and in an exported finding aid.

Screenshot for call number category titles

Screenshot for call number category titles in finding aids

In order to provide titles for call number categories, the Repository Hierarchy module uses a mechanism based on gettext and .po files, which is frequently used for translation of software projects. In the context of the Repository Hierarchy module, call number categories are "translated" into titles.

Everytime a repository hierarchy for a repository is shown in the browser front end, a gettext file is generated and stored in the /resources/caln/ folder of the Repository Hierarchy module. The gettext file with the name "\<tree name>_\<repository XREF>.php" contains all call number categories of the related repository.

In order to provide category titles, a .po editor like Poedit can be used to "translate" the call number categories into titles. It is also possible to use a simple text editor; however, a .po editor offers more comfortable features for .po updates and plausibility checks. The translation files need to have the following name: "\<tree name>\<repository XREF>\<language tag>.po". If a .po file is provided without language tag (i.e. "\<tree name>_\<repository XREF>.po"), it is used as a default for all languages without specifically provided .po files.

An example with the related files in the /resources/caln/ folder is shown in the following screenshot.

Screenshot for call number category .po files

Using a meta repository

A meta repository provides the possibility to view sources of a set of repositories in a shared hierarchy. The basic idea is to share a common namespace for call number categories between sources from different repositories. The common namespace can also be seen as common archival arrangement for the related sources.

A meta repository can be added in webtrees as a "regular" repository without specific extensions. While no restrictions apply, it is proposed to use the prefix "Meta:" in the repository name.

All the sources, which are planned to be integrated in the meta respository, need to be assigned to the meta repository and also need to be assigned call numbers within the context of the meta repository. The call numbers within the meta respository should follow the approach of a common namespace or a common archival arrangement.

Screenshot for call numbers of sources in the meta repository

Given that the mentioned definiton and assignment of the meta repository is available, it can be used and viewed with the Repository Hierarchy module. The handling is the same like for a single repository, but it provides the means to view and organize a much broader range of sources.

Using a combined meta repository

The Repository Hierarchy module offers a feature to combine a meta repository with a base repository. Like in the last chapter, the intention is again to share a common namespace between the meta repository and the base repository. By combining the base repository, it is not necessary to add the meta repository to each of the sources of the base repository. This is especially useful, if the base repository is the "home" of the user's genealogy and contains a lot of sources.

The Repository Hierarchy module will provide means to view a combined view of all the sources of the base repository and all the sources of the meta repository. In order to activate these features, the following setting needs to be selected in the control panel: "Use Meta Repositories".

To make the relationship between the repositories more visible, ít is proposed to use the same repository name and a prefix "Meta:" for the meta repository. An example is shown in the following screenshot.

Screenshot for repository and meta repository

In order to link the meta repository to the base repository, the XREF of the meta repository needs to be assinged as a user defined "Reference number" (i.e. Gedcom tag REPO:REFN) to the base repository. Additionally, the "Reference number" needs to be assigned the type "META_REPOSITORY" (i.e. Gedcom tag REPO:REFN:TYPE).

Screenshot assign meta repository

Given that the mentioned definiton and assignment of the meta repository is available, the Repository Hierarchy module will include all the sources of the meta repository when viewing the structure of the base repository. The handling is the same like for a single repository, but it provides the means to view and organize a much broader range of sources. It is even possible to view all sources of a webtrees tree in one Repository Hierarchy view.

More details can be found in the chapter Demo website and examples.

Generate and download a finding aid document

A finding aid document contains detailed, indexed, and processed metadata and other information about a specific collection of source records within an archive. More simple, it is a (hiearchical) list of sources in an archive with additional metadata.

The benefit of a finding aid document is to provide a fast overview of the available sources for a user/visitor of an archive. It also provides insights about the structure of the archive and the kind of sources, which can be found in the archive.

With the Repository Hierarchy module, webtrees can generate a finding aid document for a chosen repository. After selecting the Repository Hierarchy module from the list menu, a repository can be chosen and a delimiter expression needs to be provided. The chosen delimiter expression will be used to generate and view a hierarchical structure for the repository.

Based on the generated repository and call number structure, a finding aid document can be generated and downloaded by clicking the "Download" button and selecting one of the options "Finding aid as HTML" or "Finding aid as PDF".

The generated finding aid document contains the following metadata for each of the sources:

Screenshot

Demo download for a finding aid document

An example for a finding aid document can be downloaded with the following link.

In page 1-6 of this example, a preface was added with a word processing software. The pages starting from page 7 were generated by the Repository Hierarchy module.

Generate and download an EAD XML export

With the Repository Hierarchy module, webtrees can generate an EAD XML export for a chosen repository. The export is provided in apeEAD XML.

The EAD XML export contains:

After selecting the Repository Hierarchy module from the list menu, a repository can be chosen and a delimiter expression needs to be provided. The chosen delimiter expression will be used to generate and view a hierarchical structure for the repository.

Based on the generated repository and call number structure, an EAD XML export can be generated and downloaded by clicking the "Download" button and selecting the option "EAD XML".

Settings for EAD XML exports

In order to generate EAD XML exports, some settings need to be provided.

The EAD XML settings can be provided by clicking the "EAD XML settings" button. A specific window will open to enter the values.

Within the EAD XML settings window, a button is available to load the settings from an administrator. Hence, if an administrator provided values for the EAD XML settings, they can be loaded and used.

The apeEAD standard, which is used for the export, requires to provide at least the following values:

Additionally, the following values can be provided:

The Repository Hierarchy module will provide a default proposal for the values.

The main agency code is an unique code identifying the archival institution maintaining the described collection; encoded according to ISO 15511 (ISIL). The main agency code is officially assigned to archives by a national authority. As a substitute value (e.g. for a private or inofficial archive), the country code and "-XXXXX" might be chosen, e.g. FR-XXXXX.

Export data to an external archive management system

The EAD XML exports described in the section "[Generate and download an EAD XML export](#generate-and download-an-ead-xml-export)" can be used to transfer data from webtrees to an external archive management system.

Some examples for archive management systems:

The EAD export of the Repository Hierarchy module was tested with AtoM. Further details about AtoM import can be found in the AtoM import documentation.

For re-importing EAD XML from the same webtrees repository, the AtoM setting "Ignore matches and import as new" should be used. The re-import will generate a new (parallel) archival institution in AtoM. To add (newly imported) sources to the exiting archival institution, they can be selected from the imported structure and moved within AtoM to the available structure. Please note that AtoM does not offer features to sync and update existing records. For more details, please refer to the AtoM documentation for XML imports.

Create links between webtrees and an external archive management system

When exporting webtrees data to an external archive system, linking between webtrees and the external system helps to connect the two systems and to keep redundancy at a mimimum.

The following concept was tested with the AtoM archive management system. If you are interested to interact with other archive management systems, please contact the module author for further clarifcation and discussion.

While exporting data from webtrees with an EAD XML export, a XML structure with URLs to each of the sources in webtrees is included. After importing the EAD XML into AtoM, the links are shown in the user interface and can be used to navigate to the related webtrees source.

Screenshot

Within webtrees, the Repository Hierarchy module can provide links to the related AtoM records. In order to use this feature, the following steps need to be taken:

Screenshot

Show facts of sources within source citations

The Repository Hierarchy module provides a feature to show the facts of sources (e.g. publication, notes, reference numbers, media objects, ...) within source citations. This helps to obtain the information about related sources more quickly. In the standard webtrees installation, only the title of a source with a link is shown.

In the module settings, there is a selection field (see screenshot) to choose the source facts to be shown in source citations. For some facts, which have the character of a group, a second selection field is available to choose whether the group is automatically expanded or otherwise collapsed as a default.

Screenshot: Settings for source facts within source citations Screenshot

In the following screenshot, it can be seen how the information is provided in two section. One section for the source information and another section for the source citation.

Screenshot: Show further source facts within source citations Screenshot

Copy-Paste mechanism for source citations in the individual and family view

The Repository Hierarchy module provides a feature to copy/paste source citations. If this feature is activated in the module settings, a copy icon is shown on the right hand side of each source citation. By clicking on the icon, the source citation can be copied to an internal clipboard. If a source citation is available in the clipboard, a source icon is shown in the edit area of all facts/events, which allow to use source citations. By clicking on the source icon, the source citation can be copied from the clipboard to the fact/event.

Screenshot: Copy/Paste of source citations Screenshot

In order to activate this feature, the setting "Enable copy/paste mechanism for source citations in the individual and family view" needs to be selected in the control panel.

Delete mechanism for source citations in the individual and family view

The Repository Hierarchy module provides a feature to delete source citations. If this feature is activated in the module settings, a delete icon is shown on the right hand side of each source citation. By clicking on the icon, the source citation can be deleted.

Screenshot: Delete source citations Screenshot

In order to activate this feature, the setting "Enable delete mechanism for source citations in the individual and family view" needs to be selected in the control panel.

Sort mechanism for source citations in the individual and family view

The Repository Hierarchy module provides a feature to sort source citations. If this feature is activated in the module settings, arrow icons are shown on the right hand side of each source citation. By clicking on the move-up/move-down arrows, the source citations can be sorted.

Screenshot: Sort source citations Screenshot

In order to activate this feature, the setting "Enable sort mechanism for source citations in the individual and family view" needs to be selected in the control panel.

Directly show media objects of source citations in individual and family facts

The Repository Hierarchy module provides a feature to directly show media objects of source citations in individual and family facts (and not subordinated in the source citation).

Screenshot of individual fact with directly shown media objects: Screenshot

Screenshot of individual fact with media objects subordinated to source citation: Screenshot

In order to activate this feature, the setting "Directly show media objects of sources in facts and not subordinated within source citations" needs to be selected in the control panel.

Show all images of media objects

The functionality of the custom module webtrees-simple-media-display developed by Carmen Just was fully integrated into Repository Hierarchy. The objective of this module is to show all images of media objects. Webtrees allows to add several images to media objects. However, a lot of views in the frontend only show the first image. With the extension of Carmen Just, all media objects can be shown.

Screenshot: Show multiple images of media objects Screenshot

Since both - Repository Hierarchy and webtrees-simple-media-display - do change the same webtrees views to show media objects, it is strongly recommended to deactivate the webtrees-simple-media-display (control panel: "Media Display as in webtrees 2.0") if Repository Hierarchy is used, because parallel activation can lead to unintended behavior. The identical functionality of webtrees-simple-media-display - even the same code - is also integrated in the Repository Hierarchy module.

Screenshot: Warning: webtrees-simple-media-display is activated in parallel Screenshot

Preferences

The following preferences can be activated/deactivated by administrators in the control panel.

Preferences for the main Repository Hierarchy list

Preferences for source citations

Preferences for finding aid export

Preferences for EAD XML exports

Preferences for Meta Repositories

Preferences for linking to external archive management tools

Demo website and examples

The following links provide access to a demo website, which demonstrates some of the module features.

Demo website and examples for a meta repository

Additionally to the above demo website links, the following links, examples, and GEDCOM snippets demonstrate the features of the Repository Hierarchy module, which make use of a meta repository.

Please note in this webtrees demo site, how the Base Repository is linked to the meta repository:

Reference number: R8773
Type: META_REPOSITORY

The following GEDCOM snippets show the usage of a meta repository in the INDI, SOUR, and REPO structures of webtrees/Gedcom. The snippets are related to the demo website above; the XREFs from the snippets can be directly found on the demo website.

0 @I6684@ INDI
1 NAME Leonhard /Kastler/
1 BIRT
2 DATE 03 NOV 1735
2 PLAC Deubach, Landkreis Guenzburg, Bayern, DEU
2 SOUR @S8685@
3 PAGE Seite 62, https://data.matricula-online.eu/de/deutschland/augsburg/deubach/1-T/?pg=33

0 @S8685@ SOUR
1 TITL Kirchenbuch Deubach, Taufen 1670-1804
1 DATA
2 EVEN BAPM
3 DATE FROM 1670 TO 1804
3 PLAC Deubach, Landkreis Guenzburg, Bayern, DEU
1 REPO @R7328@
2 CALN Deubach, 1-T, https://data.matricula-online.eu/de/deutschland/augsburg/deubach/1-T
3 MEDI ELECTRONIC
1 REPO @R8773@
2 CALN BiHu/Kop/KB/Deubach, Landkreis Guenzburg, Bayern, DEU/Taufen 1670-1804

0 @R1@ REPO
1 NAME My Family Archive
1 REFN R8773
2 TYPE META_REPOSITORY

0 @R8773@ REPO
1 NAME Meta: My Family Archive

0 @R7328@ REPO
1 NAME Matricula Online

In this example, R1 is the base repository, R8773 the meta repository, and R7328 is just some other repository. With the Gedcom lines "1 REFN R8773 2 TYPE META_REPOSITORY", the meta repository is combined with the base repository. If the Repository Hierarchy module identifies this relationship, the two repositories (e.g. R1 and R8773) are combined and shown in the same hierarchical view. In order to create a benefit from this concept, the two repositories should use the same call number format (i.e. same delimiters, and the same call number category structure).

Please note that combining the base repository (R1) to the meta repository (R8773) mainly reduces effort and avoids redundancy for source repository citations of sources in the base repository. This is illustrated by the following example for a source in the base repository R1:

Example 1
Source with "double" source repository citations (one for the base repository, and a second one for the meta repository)

0 @S8765@ SOUR
1 TITL Hochzeit Litzel-Hartmann, Standesamt Fleinhausen, 1902
1 REPO @R1@
2 CALN LiHa/Biogr/Litzel, Maximilian/Nr. 01
1 REPO @R8773@
2 CALN LiHa/Biogr/Litzel, Maximilian/Nr. 01

Example 2
Source with "single" source repository citations (if using a combined meta repository). In this case, the second source repository citation (for the meta repository) is not needed!

0 @S8765@ SOUR
1 TITL Hochzeit Litzel-Hartmann, Standesamt Fleinhausen, 1902
1 REPO @R1@
2 CALN LiHa/Biogr/Litzel, Maximilian/Nr. 01

It is important to notice that both examples are equaly working and both make use of the meta repository. However, the second example avoids redundancy and needs less work.

Some more background about archive and library management

In archive (or library) management, archival arrangements, library classifications, finding aids, and call numbers are frequently used to:

In the following, some of the typical concepts are briefly described.

Archival Arrangement

Wikipedia: "Arrangement is the manner in which [the archive] has been ordered [...]. Hierarchical levels of arrangement are typically composed of record groups containing series, which in turn contain boxes, folders, and items."

Library classification

Wikipedia: "A library classification is a system of knowledge distribution by which library resources are arranged and ordered systematically."

Finding Aids

Archive Portal Europe: "A finding aid is a structured description of archival materials per collection or fonds up to item level"

Wikipedia: "A finding aid for an archive is an organization tool, a document containing detailed, indexed, and processed metadata and other information about a specific collection of records within an archive."

Call numbers

Wikipedia: "[...] a call number (essentially a book's address) based on the classification system in use at the particular library will be assigned to the work using the notation of the system."

Relationship between Archival Arrangement and Call numbers

A lot of archives (and libraries) map the archival arrangement (or library classification) into the call numbers of the sources.

For example, the archive might have the following arrangement:

In this case, the call numbers might have the following structure:

"Fonds/Record group/Series/Folder/Source"

Therefore, the hierarchy of the archival arrangement is represented in the "route" or the "path" of the call number.

EAD standard for XML export of archival records

Encoded Archival Description (EAD) is a standard for encoding descriptive information regarding archival records. The EAD standard provides a XML export format, which allows to describe and export the content and structure of archives and sources. It also provides data structures to describe and export finding aids.

See also: Wikipedia

apeEAD standard for XML export of finding aids

apeEAD is a standard, which was designed and published by the Archive Portal Europe. As a sub-set of EAD, apeEAD was specifically designed for encoding archival finding aids. As specification of the standard, the Archive Portal Europe provides a table with the used EAD XML tags and a description and best practice guide for usind ape EAD.

The Archive Portal Europe also provides a validation tool. With the apeEAD data preparation tool, EAD XML exports can be validated against the apeEAD standard. Details are described in a manual.

The XML exports of the Repository Hiararchy module were developed and tested to pass the validation of the apeEAD data preparation tool.

How the module maps to Gedcom and to archive management concepts

In order to manage archives and sources, Gedcom and webtrees basically provide the following data structures:

The following table describes how the concepts from archive and library management are mapped to Gedcom/webtrees and the Repository Hierarchy custom module:

Archive/Library Concept Gedcom/webtrees data structures Repository Hierarchy Module
Archive,
Library
Repository Repository
Archival Arrangement,
Library Classification
- Hierarchy of call number categories
Fonds,
Record group,
Series,
Folder
- Call number category
Item,
file,
book
Source Source
Call number Call number Call number
Finding aid List of sources for a selected repository List of sources in a hierarchy of call number categories for a selected repository

Translation

You can help to translate this module. The translation is based on gettext and uses .po files, which can be found in /resources/lang/. You can use a local editor like Poedit or notepad++ to work on translations and provide them in the Github repository of the module. You can do this via a pull request (if you know how to do), or by opening a new issue and attaching a .po file. Updated translations will be included in the next release of this module.

Currently, the following languages are already available:

Bugs and feature requests

If you experience any bugs or have a feature request for this webtrees custom module, you can create a new issue.

License

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Development and Contributions

Github repository

https://github.com/Jefferson49/RepositoryHierarchy