fedspendingtransparency / fedspendingtransparency.github.io

Federal Spending Transparency
http://fedspendingtransparency.github.io
Creative Commons Zero v1.0 Universal
54 stars 118 forks source link

What are some of the considerations that Treasury and OMB should take into account when establishing data standards? #8

Open rmaziarz opened 9 years ago

bsweger commented 9 years ago

Though not directly related to the data standards, it would be helpful to know if there are plans to include any derived/calculated elements in the next iteration of federal spending information.

For example:

Obviously you can’t be in the business of handling every indiviual request for a calculated field, but doing a few that are widely beneficial would make it easier and more efficient to use the data.

Either way, as you finalize the standards, it would be useful to know the plans for derived fields and, if they’re in play, what the guidelines are for determining their inclusion.

HerschelC commented 9 years ago

One of the most important considerations with any standard is how to audit compliance with the standard. The standard must be comprehensive to leave little gray area or wiggle room. More specifically in the case of the DATA Act - an important consideration is ensuring the quality of the standardized data. Quality likely being defined as fitness for purpose.

The process of establishing a standard is of little value without rigorous quality control measures. How you will measure data quality should be part of data standard definitions. How data defects are adjudicated should be a process established alongside the data standard since the people setting the standard are likely the people to deal with violations of the standard.

HerschelC commented 9 years ago

When dealing with a migration of existing data standards to a new standard, the idea of "global" and "local" standards could be implemented. A global standard applies across all stakeholder communities whereas a local standard applies to a subset that for whatever reason is not able to meet the global definition. In theory we prefer global standards for everything; in practice there are sometimes nuances because either the data does not exist, is not easily accessible, or the local organization cannot use the global definition for operational reasons. That's ok - so long as there is a clear local definition. Don’t stop progress waiting on perfect data alignment. The significant downside to local definitions is that it prevents cross-organization analysis which requires apples-to-apples comparisons. I often use the "local" option as a stepping stone for organizations working towards the global definition but aren't quite there yet. Local definitions should be the rare exception. Using a github analogy, the local definitions could be “forks” from the master definition. Work on migrating the local definition to the global definition progresses until the local definitions are eventually pulled back into the master. The group using the local definition continues to take a hit on quality until they’ve migrated to the global definition.
An option often considered in data standards is to simply have both a global definition AND a local definition for cases where the local organization prefers to view the world in their own way. This is a different scenario than the above where the organization can’t get to the global standard. In this case it’s a different view that is desired, generally for operational reasons. This is fine and even encouraged since the local view likely enables the local organization to better manage their business while at the same time aligning with the global view of the world – it’s just a data crosswalk.

klguentert commented 9 years ago

As someone that will be asked to provide data, my concern is with reducing administrative burden while providing accurate, consistent, reliable data. Much of the data is already in federal systems and some of that data is data the recipients originally provided when asking for the funding.

For research institutions, we provide much of the data through agency systems (like SAM or grants.gov). It's critical that the data standards consistently utilize what has already been built (grants.gov data standards or RPPR data standards).

Agencies should be required to consistently use these data standards and there should be oversight to ensure that agencies are using them as intended.

Not related to data standards, please do not use FFATA reporting as the model for how the reporting will work. It is a clunky system that does not utilize existing federal data very efficiently.

In addition, something to consider is the zip plus 4 requirement. That does not always get you the "location of the work" information that you want. What happens if the work is being done in the Gulf of Mexico by an institution in Connecticut. Zip plus 4 will give you the institution in Connecticut that's receiving the funding but, the work is conducted somewhere else (without a zip plus 4) and impacting a completely different economy.

justinduncan commented 9 years ago

Treasury and OMB should adopt nonproprietary data standards for federal spending information.

Treasury and OMB should commit to using the new data standards within the existing reporting systems that they already manage.

HerschelC commented 9 years ago

@justinduncan Having a second standard for reporting the same data reminds me of the lesson learned at the SEC in allowing businesses to continue sending PDF reports as well as the new XBRL standard. Data quality suffers.

JimMcKay1 commented 9 years ago

Hope this has not already been addressed. I assume one of the goals of the DA is to achieve the President's goal for cost-based performance reporting. If we perfectly identify financial data elements, but do not focus equally on program and performance data elements, we will end up combining perfect data with imperfect program and performance data elements and be no better off than before. When will be begin defining terms like "program," "increased capacity," "response time," "attendance," "numbers served," etc...................................

HerschelC commented 9 years ago

@JimMcKay1 - I've wondered the same thing. In data element discussions it seems to have been decided not to define "program". I believe that with the current definitions that we're going to be left which a huge number of account transactions that don't roll up into anything that would be logically associated with a program to which we could attach outcome metrics for analysis. Having said that, does the law require this type of capability? I think enabling this type of outcome analysis was the intent, but not sure if that expectation will be met with the initial iteration of data elements. Also not sure if there will be a second iteration.

JimMcKay1 commented 9 years ago

There is a bill in congress that will require definition of program performance elements

Jim McKay, MBA, CICA Senior Director Kforce Government Solutions 703.638.5640

On May 14, 2015 8:40 AM, Herschel Chandler notifications@github.com wrote:

@JimMcKay1https://github.com/JimMcKay1 - I've wondered the same thing. In data element discussions it seems to have been decided not to define "program". I believe that with the current definitions that we're going to be left which a huge number of account transactions that don't roll up into anything that would be logically associated with a program to which we could attach outcome metrics for analysis. Having said that, does the law require this type of capability? I think enabling this type of outcome analysis was the intent, but not sure if that expectation will be met with the initial iteration of data elements. Also not sure if there will be a second iteration.

Reply to this email directly or view it on GitHubhttps://github.com/fedspendingtransparency/fedspendingtransparency.github.io/issues/8#issuecomment-102025511.

catalinareyes commented 9 years ago

Publish What You Fund and the Modernizing Foreign Assistance Network submitted comments to the DATA Act consultation last November. Please see: http://publishwhatyoufund.org/files/2014/11/MFAN-PWYF-response-to-Federal-Register-Notice-on-DATA-Act.pdf

Our main concern remains "tagging" information. Tagging allows different types of information to be easily identified. For example domestic versus foreign assistance spending.

All foreign assistance information should be tagged so that it can be easily consumed by others including the Foreign Assistance Dashboard if needed/wanted. The tagging system used by the DATA Act appears to be the North American Industry Classification System (NAICS). From the NAICS we would recommend all foreign assistance information to be tagged as: 928120 International Affairs.

HerschelC commented 9 years ago

@catalinareyes What you say makes perfect sense. Being able to report on data "by" multiple dimensions/attributes is only a good thing.

Isn't this already accomplished with the given standards? An award is tied to a NAICS, in this case 928120. Or are you saying that each award/payment/expense should be tied to/tagged with multiple NAICS for descriptive purposes? For example, 541511 for computer services as well as 928120 for computer services for foreign assistance.

Would this purpose be served by having a separate data element that denotes foreign assistance? Simply Yes or No if foreign assistance.

catalinareyes commented 9 years ago

@HerschelC Having a separate element that denotes foreign assistance would probably be preferable as it could allow the data to be classified more specifically. It may also make sense to allow multiple NAICS classifications (with % for each one), as awards could be quite large and have several components (e.g. health plus education), which can't always be disaggregated.

HerschelC commented 9 years ago

@catalinareyes This makes sense. I think to ensure the proposed element is captured for discussion that you should add it https://github.com/fedspendingtransparency/fedspendingtransparency.github.io/issues with the subject beginning with "Proposed Element". You'll see some others on there. Your good point on this new element may be lost within this thread.

I like your thinking with regards to NAICS codes. I wish we could have had this discussion while the public comment period was open. I wonder the process for assigning NAICS. I've often found it difficult to find relevant contracts because it seems that the work we do could be assigned any number of NAICS codes (and therefore the spending would be associated with those NAICS). I started a new issue thread on this topic; though we don't seem to get a lot of feedback on our issues. https://github.com/fedspendingtransparency/fedspendingtransparency.github.io/issues/83 please comment if I didn't capture the whole concept.

ecyiing commented 9 years ago

Hi, I'm new to this database and I was wondering if anyone can help me reference the right category for actual federal funds that were transferred in a fiscal year. I have currently aggregated "dollarsobligated" for contracts and I'm concerned that I may be double counting some of the funds. In other words, are the contract awards reported here the amount for the fiscal year or should I appropriate them across the duration of the contract? Thanks.

bsweger commented 9 years ago

Welcome, @ecyiing! Is this a question about using the data that's currently on USAspending.gov? If so, the best way to get an assist is to post a new issue on the USAspending issue tracker (look for the green New Issue button in the upper left corner) or to fill out the form on their contact page.

If you're looking for guidance on DATA Act implementation, then you can e-mail the folks at DATAPMO@fiscal.treasury.gov.

Sorry to bounce you around--feel free to post back here if you have more questions about the best place to get help.

ecyiing commented 9 years ago

Thanks for the lead @bsweger. Really appreciate it.