What is the data federation effort all about? What am I looking to get out of it?
This is a collaborative research project with GSA's Office of Products & Platforms and 18F. The goal is to build a toolkit / playbook for undertaking intra-governmental data collection / aggregation projects, such as data.gov, code.gov, and NIEM. Or goal is to find out what works, what doesn't, and what tools are appropriate for what circumstances, in order to accelerate similar efforts in the future.
Notes & Report will be public
Any questions before we get started?
What is X, in your own words?
Federal geographic data committee office of secretariat. Made up for 32 federal agencies, most have geospatial part of portfolio. Steering committee - executives who have geo spatial as part of portfolio. FGDC governed by A-16, WW2 effort trying to organize geospatial for civilians. Geospatial has been around for a long time "it's what the government does." A-16 revised in 1990, FGDC organized as coordinating body. National Spatial Data Infrastructure (NSDI) - includes non-federal and non-governmental datasets as well. About 200 members, cross-cutting view across different sectors. Metadata standards, geospatial standards. Content standards for digital geospatial metadata (CSDGM). Different agencies had already started. Established clearing house, a way to aggregate information by pulling it into database you could search. 2011 / 2013, geospatial world faced with open data policy. Old problem in geo world! Complementary to geo efforts. Data.gov has done a good job allowing geospatial community to keep going with what . ISO 19115 - new standard. Did crosswalk between those ISO standards and data.gov schema. data.gov allows harvest sources to be tagged as geospatial.
What was impetus or driving force for this effort: policy, user needs, etc (perhaps after the first question)
[covered]
In building X, what were the biggest challenges, and what went smoothly?
challenge for agencies: asking them to "do it twice". Success: data.gov is "the place. But some agencies question why we need to do geospatial thing? (answer: geospatial standards a lot more robust). Can transform using crosswalk, but it's extra work. Project open data requires project / program codes, not represented in geo standards. Also licensing was a problem. On policy side: monthly meetings, get together, bring up issues. e.g., errors in data.gov, teams of interagency folks got together to do crosswalks. People / policy / technical.
What tools and technologies do you use for this effort?
"each agency is a little bit different", but data.gov sets standards. For creation, using vendor tools. But ultimately everything is in XML going through pipeline. ISO standards use XML. Meta data creation: "hundreds" of tools. tools all over the place. "A lot of it is what data.gov offers" - everything has to be in json or ISO formats for metadata. Old standards crosswalked into ISO.
any gaps or tools / technologies that could have helped?
"you think you have it covered, then somebody throws a curveball at you." started in 1998, services / APIs didn't exist, technologies change over time. Things that are now in the data that were part of it. "Of anything, it would be take the time to do it right." better definitions, dictionaries, all that stuff. The more you can spend time up front, the better. Using common vocabularies, "gosh, that would have been nice" - to have some structure. In early days, people didn't want to do it. acronyms don't even make sense over time. "Stop putting people into these things" - think about organization, not individual. "Be more universal in any way that you can."
What are the political and organizational dynamics of collecting this data?
"That's part of the role of FGDC, is to provide that facilitation across agencies." For example, addresses. but when you start putting it in context of 911 emergency response, can be a lot more complex. Need for national address database, every state / locality does it a little differently. Trying to get a broader picture is really hard. DOT taking lead along with U.S. census bureau. They've said they needed it, then trying to get right people at the table. "Sharing information can always be an issue" - some states due to privacy regs in their states, can't share it.
Who were the relevant stakeholders for this project, how were they identified and convened?
"I would start with any known standards bodies, and there's a lot of them out their for different topics area" (in geospatial, it's ISO technical committee 211). in U.S., American Standards National Institute (ANSI). See what those organizations are doing, trying to talk to some folks. See website, ask people if they're working in standards arena. "The more consistent you can be, everyone benefits from that."
Introductory comments
What is X, in your own words?
Federal geographic data committee office of secretariat. Made up for 32 federal agencies, most have geospatial part of portfolio. Steering committee - executives who have geo spatial as part of portfolio. FGDC governed by A-16, WW2 effort trying to organize geospatial for civilians. Geospatial has been around for a long time "it's what the government does." A-16 revised in 1990, FGDC organized as coordinating body. National Spatial Data Infrastructure (NSDI) - includes non-federal and non-governmental datasets as well. About 200 members, cross-cutting view across different sectors. Metadata standards, geospatial standards. Content standards for digital geospatial metadata (CSDGM). Different agencies had already started. Established clearing house, a way to aggregate information by pulling it into database you could search. 2011 / 2013, geospatial world faced with open data policy. Old problem in geo world! Complementary to geo efforts. Data.gov has done a good job allowing geospatial community to keep going with what . ISO 19115 - new standard. Did crosswalk between those ISO standards and data.gov schema. data.gov allows harvest sources to be tagged as geospatial.
What was impetus or driving force for this effort: policy, user needs, etc (perhaps after the first question)
[covered]
In building X, what were the biggest challenges, and what went smoothly?
challenge for agencies: asking them to "do it twice". Success: data.gov is "the place. But some agencies question why we need to do geospatial thing? (answer: geospatial standards a lot more robust). Can transform using crosswalk, but it's extra work. Project open data requires project / program codes, not represented in geo standards. Also licensing was a problem. On policy side: monthly meetings, get together, bring up issues. e.g., errors in data.gov, teams of interagency folks got together to do crosswalks. People / policy / technical.
What tools and technologies do you use for this effort?
"each agency is a little bit different", but data.gov sets standards. For creation, using vendor tools. But ultimately everything is in XML going through pipeline. ISO standards use XML. Meta data creation: "hundreds" of tools. tools all over the place. "A lot of it is what data.gov offers" - everything has to be in json or ISO formats for metadata. Old standards crosswalked into ISO.
any gaps or tools / technologies that could have helped?
"you think you have it covered, then somebody throws a curveball at you." started in 1998, services / APIs didn't exist, technologies change over time. Things that are now in the data that were part of it. "Of anything, it would be take the time to do it right." better definitions, dictionaries, all that stuff. The more you can spend time up front, the better. Using common vocabularies, "gosh, that would have been nice" - to have some structure. In early days, people didn't want to do it. acronyms don't even make sense over time. "Stop putting people into these things" - think about organization, not individual. "Be more universal in any way that you can."
What are the political and organizational dynamics of collecting this data?
"That's part of the role of FGDC, is to provide that facilitation across agencies." For example, addresses. but when you start putting it in context of 911 emergency response, can be a lot more complex. Need for national address database, every state / locality does it a little differently. Trying to get a broader picture is really hard. DOT taking lead along with U.S. census bureau. They've said they needed it, then trying to get right people at the table. "Sharing information can always be an issue" - some states due to privacy regs in their states, can't share it.
Who were the relevant stakeholders for this project, how were they identified and convened?
"I would start with any known standards bodies, and there's a lot of them out their for different topics area" (in geospatial, it's ISO technical committee 211). in U.S., American Standards National Institute (ANSI). See what those organizations are doing, trying to talk to some folks. See website, ask people if they're working in standards arena. "The more consistent you can be, everyone benefits from that."