search

kartoza / WRODataPlatform

WRC Water Research Observatory Data Platform
0 stars 3 forks source link

XML upload tool not working #53

Open vermeulendivan opened 2 years ago

vermeulendivan commented 2 years ago

Mohab is busy with this issue.

Mohab25 commented 2 years ago

added the XML functionality, if the uploaded file holds the minimum set of required fields (below), the dataset will be created, otherwise if the xml file has more fields than the maximum set of fields (defined as follow) an error would be raised, notice the following:

WRO_METADATA_REQUIRED_FIELDS = [
    'email', 
    'title', 
    'name', 
    'authors-0-author_name',
    'contact_person-0-contact_name',
    'contact_person-0-contact_email', 
    'notes',
    'owner_org', 
    'data_collection_organization', 
    'publisher', 
    'publication_date', 
    'license', 
    'keywords', 
    'spatial', 
    'wro_theme', 
    'data_structure_category', 
    'uploader_estimation_of_extent_of_processing', 
    'data_classification', 
    'agreement'
]
WRO_METADATA_FIELDS = [
    'email', 
    'title', 
    'name', 
    'authors-0-author_name',
    'authors-0-author_surname',
    'authors-0-author_email',
    'authors-0-author_organization',
    'authors-0-author_department',
    'authors-0-contact_same_as_author',
    'contact_person-0-contact_name',
    'contact_person-0-contact_email', 
    'contact_person-0-contact_orgnization', 
    'contact_person-0-contact_department', 
    'notes',
    'owner_org', 
    'citation-0-citation_title', 
    'citation-0-citation_date', 
    'citation-0-citation_identifier', 
    'did_author_or_contact_organization_collect_the_data', 
    'data_collection_organization', 
    'dataset_language', 
    'publisher', 
    'publication_date', 
    'wrc_project_number', 
    'license', 
    'dataset_license_url', 
    'keywords', 
    'spatial', 
    'wro_theme', 
    'data_structure_category', 
    'uploader_estimation_of_extent_of_processing', 
    'data_classification', 
    'data_reference_date-0-data_reference_date_from', 
    'data_reference_date-0-data_reference_date_to', 
    'alternative_identifier', 
    'vertical_extent_datum', 
    'minimum_maximum_extent-0-minimum_vertical_extent', 
    'minimum_maximum_extent-0-maximum_vertical_extent',
    'tags-0-tag_name',
    'tags-0-tag_type',
    'agreement'
    ]

the following file was used to test the xml functionality:

<?xml version="1.0" encoding="UTF-8" ?>
<dataset>
        <email>homab3@gmail.com</email>
        <title>xml testing dataset</title>
        <authors-0-author_name>Dave</authors-0-author_name>
        <authors-0-author_surname>bings</authors-0-author_surname>
        <authors-0-author_email>bings@gmail.com</authors-0-author_email>
        <authors-0-author_organization>kartoza</authors-0-author_organization>
        <authors-0-author_department>Development</authors-0-author_department>
        <authors-0-contact_same_as_author>false</authors-0-contact_same_as_author>
        <contact_person-0-contact_name>Mohab</contact_person-0-contact_name>
        <contact_person-0-contact_email>mohab@gmail.com</contact_person-0-contact_email>
        <contact_person-0-contact_orgnization>kartoza</contact_person-0-contact_orgnization>
        <contact_person-0-contact_department>Development</contact_person-0-contact_department>
        <owner_org>kartoza</owner_org>
        <notes>dataset to test xml dataset creation</notes>
        <citation-0-citation_title>no citation available</citation-0-citation_title>
        <citation-0-citation_date>1994-03-15</citation-0-citation_date>
        <citation-0-citation_identifier>no citation identifier available</citation-0-citation_identifier>
        <did_author_or_contact_organization_collect_the_data>False</did_author_or_contact_organization_collect_the_data>
        <data_collection_organization>Collection Org</data_collection_organization>
        <dataset_language>English</dataset_language>
        <publisher>no-limits publish works</publisher>
        <publication_date>1994-03-15</publication_date>
        <wrc_project_number>Project#12</wrc_project_number>
        <license>Open (Creative Commons)</license>
        <dataset_license_url>No url available</dataset_license_url>
        <keywords>testing, metadata</keywords>
        <spatial>-22.1265, 16.4699, -34.8212, 32.8931</spatial>
        <wro_theme>agriculture</wro_theme>
        <data_structure_category>structured</data_structure_category>
        <uploader_estimation_of_extent_of_processing>raw</uploader_estimation_of_extent_of_processing>
        <data_classification>time series</data_classification>
        <data_reference_date-0-data_reference_date_from>2022-08-08</data_reference_date-0-data_reference_date_from>
        <data_reference_date-0-data_reference_date_to>2022-08-16</data_reference_date-0-data_reference_date_to>
        <alternative_identifier>no calories at all</alternative_identifier>
        <vertical_extent_datum>vertical extent datum</vertical_extent_datum>
        <minimum_maximum_extent-0-minimum_vertical_extent>3</minimum_maximum_extent-0-minimum_vertical_extent>
        <minimum_maximum_extent-0-maximum_vertical_extent>12</minimum_maximum_extent-0-maximum_vertical_extent>
        <tags-0-tag_name>first tag</tags-0-tag_name>
        <tags-0-tag_type>005</tags-0-tag_type>
        <agreement>True</agreement>
</dataset>
mikev3003 commented 2 years ago

@Mohab25 Here is my first stab at identifying the required fields. We recommend that this component be made low priority until we get a better idea of how it will be used by others. https://www.dropbox.com/s/4xbpc9fzd7dpkqz/20221003_xml%20file%20minimum%20requirements.docx?dl=0