Python packages for downloading the WCVP, and resolving names to it.
As a basic example, the synonym Amsonia tabernaemontana Walter var. gattingeri Woodson is resolved to the accepted
name (as of WCVP v12) Amsonia tabernaemontana var. salicifolia (Pursh) Woodson and a variety of information related to this accepted name is
provided (e.g. family, IPNI id, parent, rank etc..).
Methods for downloading and plotting distributions are also provided.
Note this package has been renamed from automatchnames to wcvpy.
To cite the WCVP: Govaerts R (ed.). 2023. WCVP: World Checklist of Vascular Plants. Facilitated by the Royal Botanic Gardens, Kew. URL http://sftp.kew.org/pub/data-repositories/WCVP/ [accessed XXXX].
Govaerts, R., Nic Lughadha, E. et al. The World Checklist of Vascular Plants, a continuously updated resource for exploring global plant diversity. Sci Data 8, 215 (2021). https://doi.org/10.1038/s41597-021-00997-6
With pip, run:
pip install git+https://github.com/alrichardbollans/wcvpy.git@1.3.2
or for plotting dependencies:
pip install "wcvpy[dist_plots] @ git+https://github.com/alrichardbollans/wcvpy.git@1.3.2"
This is as simple as:
from wcvpy.wcvp_download import get_all_taxa
checklist = get_all_taxa()This will download the checklist into the package directory for repeated use, and the returned pandas Dataframe provides a parsed version of the
checklist that I find a little more user-friendly. When first
downloaded, the most recent version of the checklist will be retrieved and the package will rely on this
version until you force an update (with get_all_taxa(get_new_version=True)).
When running the name matching commands below, the checklist will be automatically downloaded with the get_all_taxa function.
For using distribution data there is one main function get_distributions_for_accepted_taxa:
import pandas as pd
from wcvpy.wcvp_download import get_distributions_for_accepted_taxa
acc_taxa = pd.read_csv('some_data.csv') # a dataframe with a column of accepted names
accepted_name_column = 'accepted_Names'
wcvp_dists = get_distributions_for_accepted_taxa(acc_taxa,
accepted_name_column)This is restricted to accepted names.
This will run get_all_taxa to access the checklist, so will use the version you have downloaded or download the newest version. You can also specify
a specific version if required.
Some other methods for plotting distribution information are provided in plot_distributions.py
import pandas as pd
from wcvpy.wcvp_name_matching import get_accepted_info_from_names_in_column
data_csv = 'path_to_data.csv'
your_data_df = pd.read_csv(data_csv) # Data to use
name_col = 'taxa' # Name of column in data with names to check
# If you know the names in your data are only found in certain families, you can specify them here
# This is optional and will speed up the program, but _may_ result in worse matches. See the notes below.
families_in_occurrences = ['Apocynaceae', 'Rubiaceae']
# Manual resolutions are optional and included by specifying a csv file, in the same format as
# the `manual_match_template.csv` file.
manual_resolution_csv = 'manual_match_template.csv'
# Match level specifies how conservative to be. One of ['full', 'direct', 'fuzzy']
# direct: only include direct matches to wcvp
# fuzzy: Include direct matches to wcvp, matches from KNMS and matches from OpenRefine
# full: include both of the above, and autoresolution step
match_level = 'full'
data_with_accepted_information = get_accepted_info_from_names_in_column(your_data_df, name_col,
families_of_interest=families_in_occurrences,
manual_resolution_csv=manual_resolution_csv,
match_level=match_level)There are a few points to note when specifying families in the matching process. It is recommended to avoid using this unless you also set up some checks of the outputs.
If the restriction of families is not broad enough some species may erroneously be matched to a genus in the incorrect family (during autoresolution
step). There are some genera that are often considered to be in different families. For example, Anthocleista is an accepted genus in Gentianaceae
but is often considered to be in Loganiaceae. If you try to match e.g. Anthocleista procera within Loganiaceae it will be unresolved. Note however
that examples like Anthocleista brieyi are synonyms within Loganiaceae whose accepted family
is Rubiaceae and in this case the program will find the correct resolution.
Accepted names and information about these accepted names is provided in the output. We STRONGLY RECOMMEND using the outputted 'accepted_name_w_author' column where possible to avoid ambiguity.
Output dataframe is the same as the input, with additional columns providing resolved accepted name information. Where names are unresolved, values in these columns are empty.
matched_by column specifies how the name has been resolved. One of:
You can filter your the resulting dataframe to specific types of matches e.g. 'unambiguous'
matches df = df[df['matched_by].isin(['direct_wcvp_unique', 'direct_wcvp_w_author_unique', 'openrefine_unique','openrefine_unique_accepted_name', knms_single', 'knms_multiple_2', 'autoresolution_unique'])]
In the first step, to avoid the program spending time trying to find names we know to be problematic we do
some manual matching. Manual resolutions are optional and included by specifying a csv file, in the same
format as the manual_match_template.csv file. Tag= 'manual'
Once manual matches have been found, we do some very basic cleaning of submitted names (
see tidy_names_in_columnmethod in string_utils). We first try to match names directly to taxa in WCVP.
This finds taxa in WCVP which match our submitted names exactly. This tries combinations of just taxon name (
tag= 'direct_wcvp'), taxon name + taxon authors and taxon name + parenthetical authors + primary author (
tags= '
direct_wcvp_w_author'). To better match submitted names containing author information, we also clean the
submitted names by removing spaces after full stops if the full stop isn't part of an infraspecific epithet
and after the space is a letter (see tidy_authors method in string_utils)
When matching to WCVP, in cases where the there is a single unique match '_unique' is appended to the tags. In cases where multiple taxa are returned for a given submission, taxa are prioritised based on their status ( i.e. Accepted > Artificial Hybrid > Synonym> Illegitimate>...).
Names that are still unresolved are then matched using OpenRefine. OpenRefine matches names to IPNI and the corresponding WCVP matches are then used. Where openrefine provides a single matching name, we use this name tagged: 'openrefine_unique'. Where openrefine finds multiple matches these are resolved by first finding names that all resolve to the same accepted name, tagged: 'openrefine_unique_accepted_name'. Then resolves to names that are given the highest matching score, tagged: 'openrefine_unique_top_score'. Then names are selected based on prioritising taxon status and then taxon rank, tagged: 'openrefine_best_priority'.
Submitted names which aren't found in these first steps are then matched to names using KNMS, which contains multiple steps. Firstly, in simple cases where KNMS returns a single match for a submitted name we use the match IPNI ID to find accepted information from WCVP. Tag = 'knms_single'
Frequently however, submissions will be matched to multiple names in KNMS. In these cases we attempt to find the 'best' match. To do this, first we find accepted info for each of the matches using the match IPNI ID and WCVP. In cases where the accepted name for a given match is the same as the submitted name, we use this match (Tag = 'knms_multiple_1'). Next, in cases where a given submitted name matches (to many) names which all have the same accepted name, we use this accepted name (Tag = 'knms_multiple_2').
Next, for submissions which have been matched in KNMS but haven't been resolved so far we look for matches where the accepted name from the match is contained in the submitted name. This is useful for catching instances where author names have been provided, meaning that the submission may have been unresolved in the previous step. In some cases, for a single submitted name this may return multiple matches, in which case we take the most specific match (i.e. " Subspecies" > "Variety" > "Species"> "Genus"). Tag = 'knms_multiple_3'
Once we have tried to resolve submitted names through KNMS in the above, we may still have some names left over. In these cases we first try to do some automated resolution. In this step we search through WCVP for taxa where the taxon name is contained in the submitted name. This is similar to the previous step but is much slower as many more names must be checked (specifying families of interest really helps here). For each submitted name, we then have a list (possibly empty) of taxa where the taxon name is contained in the submitted name. This list is initially reduced by removing taxa of the same rank but worse taxonomic status than other taxa in the list (i.e. Accepted > Artificial Hybrid > Synonym> Illegitimate>...). Next, we resolve by taking the most specific match from this list i.e. " Subspecies" > "Variety" > "Species"> "Genus". In some cases, a species may be submitted where the species part of the name has been misspelled e.g. Neonauclea observifolia; these cases resolve to the genus which may or may not be desriable depending on the specific application. Some genera names are shared across family names ( e.g. Condylocarpus). Therefore when families have not been specified, we don't match submissions to genera where the genera are known to be contained in multiple families. Note that this is conservative and will cause some good matches to not be matched. Tag= 'autoresolution', including _unique if the match to WCVP was unique
Finally, the resolutions are recompiled and an updated dataframe is returned. Submitted names which haven't been matched at any point are output to a csv file for you to check. Note that unmatched submissions are included in the output dataframe without any accepted information.
The program does some automatic formatting of the input to help resolve name, e.g. removing whitespace, removing unicode characters, fixing capitalisations. However these cleaning method won't catch everything and may lead to unresolved names, in which case it may be worth checking the input data for:
name matching temp outputs
folder.knms_unmatched_accepted_names.csv.Using most up-to-date version of WCVP
Artificial Hybrids are treated as accepted
A note on using families in matching: There are some genera that are often considered to be in different
families. For example, 'Anthocleista' is
an accepted genus in Gentianaceae but is often considered to be in Loganiaceae. If you try to match e.g. '
Anthocleista procera' within Loganiaceae (using families_of_interest argument) it will be unresolved. Note
however that examples like 'Anthocleista brieyi' are synonyms within Loganiaceae whose accepted family is
rubiaceae and in this case the program will find the match. This is particularly relevant for families like
Loganiaceae that have been used as a catch-all
Some records in WCVP are not given accepted information
Sometimes POWO and WCVP don't agree (mostly due to short lag in POWO updates?)
Accepted names are not always unique (without author information) e.g. Helichrysum oligocephalum
Some taxa are not given ipni ids, including some accepted taxa
Some taxa are not given author information
For issues with WCVP, see https://github.com/alrichardbollans/automatchnames/issues/25
See readme in OpenRefine package in this library.
//: # ()
We believe all examples file given in unit_tests/test_inputs resolve correctly, with the exception of known
cases in examples_to_fix.csv.
Govaerts R (ed.). 2023. WCVP: World Checklist of Vascular Plants. Facilitated by the Royal Botanic Gardens, Kew. [WWW document] URL http://sftp.kew.org/pub/data-repositories/WCVP/ Retrieved XX/XX/XX.
KNMS (2023). Kew Names Matching Service. http://namematch.science.kew.org/
OpenRefine https://openrefine.org/
IPNI (2023). International Plant Names Index. Published on the Internet http://www.ipni.org The Royal Botanic Gardens, Kew, Harvard University Herbaria & Libraries and Australian National Herbarium.
//: # ()