Open jsfenfen opened 6 years ago
We need a protocol for updating the concordance. That should be part of our powwow.
On Oct 26, 2017 11:47 PM, "Jacob Fenton" notifications@github.com wrote:
I think the current version of the concordance leaves out elements introduced in 2016v3.0, including the following. That may be deliberate--but at some point I thought this had been discussed and was going to include this stuff?
/ReturnHeader/FilingSecurityInformation/IPAddress/IPv4AddressTxt /ReturnHeader/FilingSecurityInformation/IPAddress/IPv6AddressTxt /ReturnHeader/FilingSecurityInformation/IPDt /ReturnHeader/FilingSecurityInformation/IPTm /ReturnHeader/FilingSecurityInformation/IPTimezoneCd /ReturnHeader/FilingSecurityInformation/FederalOriginalSubmissionId /ReturnHeader/FilingSecurityInformation/FederalOriginalSubmissionIdDt /ReturnHeader/FilingSecurityInformation/FilingLicenseTypeCd /ReturnHeader/FilingSecurityInformation/AtSubmissionCreationDeviceId /ReturnHeader/FilingSecurityInformation/AtSubmissionFilingDeviceId
/IRS990ScheduleA/AgriculturalResearchOrgInd /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CollegeUniversityName/ BusinessNameLine1Txt /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CollegeUniversityName/ BusinessNameLine2Txt /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CityNm /IRS990ScheduleA/AgriculturalNameAndAddressGrp/StateAbbreviationCd /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CountryCd
/IRS990ScheduleA/DistributionAllocationsGrp/ExcessDistributionCyovYr3Amt /IRS990ScheduleA/DistributionAllocationsGrp/ExcessFromYear4Amt
/IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/PriorCHNAImpactInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ NotifiedFAPCopyBillDisplayInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/FAPTranslatedInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ PermitDeferDenyRqrPaymentInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ EngageDeferDenyRqrPaymentInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ProvidedWrittenNoticeInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/MadeEffortOrallyNotifyInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ProcessedFAPApplicationInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ MadePresumptiveEligDetermInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/LookBackMedicareInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/LookBackMedicarePrivateInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ LookBackMedicaidMedcrPrvtInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ ProspectiveMedicareMedicaidInd
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/15, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn50yLMf3me0pn_5vxV-RW2b_eugMks5swVJjgaJpZM4QIgtO .
I thought about updating the concordance with these--and can do so eventually--but what's missing is the variable name generation code. It would be awesome if two people added the same xpath they got back the same automatically generated variable name...
I don't think that's possible, because multiple xpaths map to the same variable. That's why the first validatathon was a manual process--we were making up arbitrary variables.
On Oct 27, 2017 11:46 AM, "Jacob Fenton" notifications@github.com wrote:
I thought about updating the concordance with these--and can do so eventually--but what's missing is the variable name generation code. It would be awesome if two people added the same xpath they got back the same automatically generated variable name...
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/15#issuecomment-340008976, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn_Oy_nEZuljwbviR9uDznRh0Y9Y4ks5swfq_gaJpZM4QIgtO .
This is an issue I'm running into as well. My company is looking to keep an up to date database of the filed 990s and the lack of support for 2016 and 2017 schemas mean's I'll have to do this manually. Is there any plan for how to keep this concordance up to date or alternatively some documentation on how to add new rows to the concordance document? Since I'll be doing the work anyway, I'd be interested in contributing my schema updates back to this project, but will definitely need some guidance about how to best do that.
regarding the variable name generation question, wouldn't it be most sane to generate the variable name from the earliest occurrence (aka the field name that said piece of data appeared under in the schema with the lowest year and version number)? That would allow anyone to have a clear rule to add new variables without having to consult with everyone here beforehand.
@atartanian: 2016 and 2017 support is coming!
regarding the variable name generation question, wouldn't it be most sane to generate the variable name from the earliest occurrence
Yeah that's an approach I take in my version of the metadata here: https://github.com/jsfenfen/990-xml-metadata/ .
@jsfenfen any idea how far off that is? I'm currently weighing if I need to write a mapping myself for the fields i'm interested in or if I can wait for an official update to this project.
@atartanian would love to hear more about your project.
We are focused on the most useful fields on the base 990 (990 and 990-EZ) for the moment, with a few misc schedules. We should have a significant update in a couple of weeks, including more consistent and readable variable names.
@lecy my company provides philanthropy information to non-profits who are looking for donors. one of the ways we do that is to connect a given philanthropic gift to the non-profit who received it and provide additional information about said organization. we use the 990 data to do that. I'm looking to automate the process of updating the 990 data into our database and this concordance is saving me from having to write my own. We're hoping to keep our 990 dataset up to date in at least a quarterly cadence, so I'm looking to see how frequently this concordance will be updated as well as how I can participate to help on that front.
Can you share which fields from the 2016 and 2017 schemas you are planning to include for the upcoming update?
@atartanian believe this data is complete through 2016v3.0, see more here: https://github.com/jsfenfen/990-xml-reader/ ; That project standardizes 2013 forward and will be crosswalked to Jesse's concordance variables when they're out. All variables in all lettered schedules from schema year 2013 through 2016v3 are standardized, I should have 2017 soon.
@jsfenfen @lecy ok great that sounds like it's worth me putting this on the back burner for a little bit to avoid reinventing the wheel. thanks!
@jsfenfen @lecy hey folks, just looping back to this and wanted to check to see if the post-2016v3 schemas have made it into the concordance?
Hey @atartanian the 2017 schema changes are up here: https://github.com/jsfenfen/990-xml-metadata/ . The changes seem pretty minimal, btw, and as of May IRS had yet to release any xml in 2017 schemas. Not sure when they'll hit the concordance, @lecy would know best.
Hi @atartanian, I will be pushing some changes to the concordance later this week that impact some variable names, mapping, and meta-data on the main 990-PC and 990-EZ forms (not the 990-PF or Schedules).
I think the current version of the concordance leaves out elements introduced in 2016v3.0, including the following. That may be deliberate--but at some point I thought this had been discussed and was going to include this stuff?
/ReturnHeader/FilingSecurityInformation/IPAddress/IPv4AddressTxt /ReturnHeader/FilingSecurityInformation/IPAddress/IPv6AddressTxt /ReturnHeader/FilingSecurityInformation/IPDt /ReturnHeader/FilingSecurityInformation/IPTm /ReturnHeader/FilingSecurityInformation/IPTimezoneCd /ReturnHeader/FilingSecurityInformation/FederalOriginalSubmissionId /ReturnHeader/FilingSecurityInformation/FederalOriginalSubmissionIdDt /ReturnHeader/FilingSecurityInformation/FilingLicenseTypeCd /ReturnHeader/FilingSecurityInformation/AtSubmissionCreationDeviceId /ReturnHeader/FilingSecurityInformation/AtSubmissionFilingDeviceId
/IRS990ScheduleA/AgriculturalResearchOrgInd /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CollegeUniversityName/BusinessNameLine1Txt /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CollegeUniversityName/BusinessNameLine2Txt /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CityNm /IRS990ScheduleA/AgriculturalNameAndAddressGrp/StateAbbreviationCd /IRS990ScheduleA/AgriculturalNameAndAddressGrp/CountryCd
/IRS990ScheduleA/DistributionAllocationsGrp/ExcessDistributionCyovYr3Amt /IRS990ScheduleA/DistributionAllocationsGrp/ExcessFromYear4Amt
/IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/PriorCHNAImpactInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/NotifiedFAPCopyBillDisplayInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/FAPTranslatedInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/PermitDeferDenyRqrPaymentInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/EngageDeferDenyRqrPaymentInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ProvidedWrittenNoticeInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/MadeEffortOrallyNotifyInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ProcessedFAPApplicationInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/MadePresumptiveEligDetermInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/LookBackMedicareInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/LookBackMedicarePrivateInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/LookBackMedicaidMedcrPrvtInd /IRS990ScheduleH/HospitalFcltyPoliciesPrctcGrp/ProspectiveMedicareMedicaidInd