search

DOI-USGS / gems-tools-arcmap

Tools for working with the GeMS geologic map database schema in ArcGIS
Creative Commons Zero v1.0 Universal
42 stars 21 forks source link

FGDC metadata, step 2 #65

Open kalexander-wgs opened 3 years ago

kalexander-wgs commented 3 years ago

Hi I am trying to run the FGDC metadata step 2 from the most recent toolbox and im not sure what is going wrong, below are my geoprocessing results:

Running script MetadataFGDC-2... GeMS_FGDC2_Arc10.py, version of 15 March 2021 Parsing J:\gis_editing\projects_current\250_500k_gems\Validation\500k_Metadata\500k_Template-metadata.xml Master record parsed successfully DescriptionOfMapUnits Updating E-A stuff Failed script MetadataFGDC-2...

Traceback (most recent call last): File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 501, in fixObjXML(objName,objType,objLoc,domMR) File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 432, in fixObjXML dom, dataSourceValues = updateTableDom(dom,objLoc,logFile) File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 318, in updateTableDom eainfo = dom.getElementsByTagName('eainfo')[0] IndexError: list index out of range

Failed to execute (MetadataFGDC-2).

Any suggestions would be greatly appreciated. Thanks !

ethoms-usgs commented 3 years ago

It looks to me like the tool is expecting an element in your metadata template and there isn't one. We should change this because that element is not a mandatory CSDGM element and because that section get re-written for each feature class and table anyway, but try adding one, even an empty element and see if it the tool runs.

kalexander-wgs commented 3 years ago

I went back through the Metadata Wizard and I'm getting this error in the metadata validation. Maybe I should post this question with the Wizard but could this be the broken step?

Errors: Spatial_Domain is required in Identification_Information xpath: /metadata

I don't see where the Spatial Domain field is in the wizard.

ethoms-usgs commented 3 years ago

Ha! you know what? I wrote the element name in my last reply but I put it inside angle brackets like it appears in XML and GitHub didn't print it! The element your xml is missing, apparently is the eainfo or Entity And Attribute Information element. Add that one and try it. I don't know Metadata Editor, so I can't advise you on that. Its easy with tkme

kalexander-wgs commented 3 years ago

Hi, Im back at it again with new Errors for step 2. Hoping you can point me in the right direction :) Thanks so much

Executing: MetadataFGDC-2 J:\gis_editing\projects_current\250_500k_gems\Validation\500k_Metadata\500k_Template.gdb J:\gis_editing\projects_current\250_500k_gems\Validation\500k_Metadata\500k_Template-metadata.xml # Start Time: Mon May 3 12:55:01 2021 Running script MetadataFGDC-2... GeMS_FGDC2_Arc10.py, version of 15 March 2021 Parsing J:\gis_editing\projects_current\250_500k_gems\Validation\500k_Metadata\500k_Template-metadata.xml Master record parsed successfully DescriptionOfMapUnits Updating E-A stuff Field: OBJECTID Field: MapUnit Field: Name Field: FullName Field: Age Field: Description Field: HierarchyKey Field: ParagraphStyle Field: Label Field: Symbol Field: AreaFillRGB Field: AreaFillPatternDescription Field: DescriptionSourceID Field: GeoMaterial Field: GeoMaterialConfidence Field: DescriptionOfMapUnits_ID Field: created_user Field: created_date Field: last_edited_user Field: last_edited_date Pruning unused srcinfo branches Replacing title and suppplinf Writing XML to 500k_Template_DescriptionOfMapUnits-metadata.xml Failed script MetadataFGDC-2...

Traceback (most recent call last): File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 501, in fixObjXML(objName,objType,objLoc,domMR) File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 445, in fixObjXML writeDomToFile(wksp,dom,domName) File "J:\gis_editing\projects_current\250_500k_gems\gems-tools-arcmap-1.6\gems-tools-arcmap-1.6\Scripts\GeMS_FGDC2_Arc10.py", line 457, in writeDomToFile dom.writexml(outf) File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 1749, in writexml node.writexml(writer, indent, addindent, newl) File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 814, in writexml node.writexml(writer, indent+addindent, addindent, newl) File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 814, in writexml node.writexml(writer, indent+addindent, addindent, newl) File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 810, in writexml self.childNodes[0].writexml(writer, '', '', '') File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 1038, in writexml _write_data(writer, "%s%s%s" % (indent, self.data, newl)) File "C:\Python27\ArcGIS10.6\Lib\xml\dom\minidom.py", line 298, in _write_data writer.write(data) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2003' in position 17: ordinal not in range(128)

Failed to execute (MetadataFGDC-2)."

ethoms-usgs commented 3 years ago

The offending character looks to be an em-space, which I would imagine could be hard to find. Try the fix I pushed to branch issue65. Ultimately, we would like the tool to work with non-ASCII characters, so it's worth figuring this out. If you want to send the files you are working with, that's fine too.

kalexander-wgs commented 3 years ago

500k_WA_Geology.zip

I opened the metadata file using the tkme software and wasn't able to find the error. I attached a folder with the geodatabase I am using and the metadata file that is generated from the FGDC metadata step 1 which I have filled out with all the appropriate info. Thanks so muck for helping me out with this

ethoms-usgs commented 3 years ago

I got past your initial error with new code on the issue65 branch, but then ran into one that I cannot solve. Four definitions of faults in the Glossary use the degree symbol which I have not been successful yet in getting written to the XML metadata. It will take more time to track down so, in the meanwhile, the workaround is to replace those symbols with some text, degrees or deg., for example.

We have some other problems with the metadata scripts and we really need to get this non-ASCII character encoding worked out, but for now, at least, the xml files that I got out of the script could be run through mp or the USGS online validator to be validated.

My changes in the issue65 branch have been merged into the latest version, v1.6.3. Try that and let me know.

kalexander-wgs commented 3 years ago

I was able to get past my issues this morning without any further errors, but if you ran into an error that I haven't I wonder how I actually did that.

thanks for getting me past all that.

ethoms-usgs commented 3 years ago

I have to admit, since the changes I made didn't fix the degree symbol problem, it doesn't seem to me like they should have had any effect on your first error either, so I am not sure what is going on, except that it's clear that I need to learn up on UTF-8 encoding.

chhalsted commented 3 years ago

I think I'm running into a similar issue. Here is snippet of the error: UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 60: ordinal not in range(128) The code seems to be tripping over the '±' character (ascii code 241). I replaced it with '+/-' and was able to continue running the script.