search

Nonprofit-Open-Data-Collective / irs-efile-master-concordance-file

The Master Concordance File defines standards and provides documentation necessary to build structured databases from the IRS E-File XML files posted on AWS.
https://nonprofit-open-data-collective.github.io/irs-efile-master-concordance-file/
40 stars 6 forks source link

MCF: Missing 'Organization527Ind'? #14

Closed jsfenfen closed 6 years ago

jsfenfen commented 6 years ago

I noticed that the 527 checkboxes in 990, 990EZ seem to be missing in the master concordance?

/Return/ReturnData/IRS990EZ/Organization527Ind /Return/ReturnData/IRS990/Organization527Ind

Dunno if anyone's ever said yes to this, but seems to be possible? Am I missing something?

image

borenstein commented 6 years ago

They must have gotten removed in one of the updates. This is consistent with my impression that there were fewer variables, and less data, than there had been in the past. Can you check the version history to see if they got clobbered at some point?

-- David Bruce Borenstein, PhD 781.710.2789 (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 7:51 PM, Jacob Fenton notifications@github.com wrote:

I noticed that the 527 checkboxes in 990, 990EZ seem to be missing in the master concordance?

/Return/ReturnData/IRS990EZ/Organization527Ind /Return/ReturnData/IRS990/Organization527Ind

Dunno if anyone's ever said yes to this, but seems to be possible? Am I missing something?

[image: image] https://user-images.githubusercontent.com/639012/32082032-aca7f7fc-ba6d-11e7-96b6-eeeea7502530.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn7Jd4Oi_IU_kFGZhglLdA1ba5_yQks5swRr9gaJpZM4QIXp7 .

borenstein commented 6 years ago

Unfortunately, we'll have to work with what we have for the validatathon; I don't think I'll have time to re-generate the master data set.

-- David Bruce Borenstein, PhD 781.710.2789 (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 7:59 PM, David Bruce Borenstein < david.bruce.borenstein@gmail.com> wrote:

They must have gotten removed in one of the updates. This is consistent with my impression that there were fewer variables, and less data, than there had been in the past. Can you check the version history to see if they got clobbered at some point?

-- David Bruce Borenstein, PhD 781.710.2789 <(781)%20710-2789> (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 7:51 PM, Jacob Fenton notifications@github.com wrote:

I noticed that the 527 checkboxes in 990, 990EZ seem to be missing in the master concordance?

/Return/ReturnData/IRS990EZ/Organization527Ind /Return/ReturnData/IRS990/Organization527Ind

Dunno if anyone's ever said yes to this, but seems to be possible? Am I missing something?

[image: image] https://user-images.githubusercontent.com/639012/32082032-aca7f7fc-ba6d-11e7-96b6-eeeea7502530.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn7Jd4Oi_IU_kFGZhglLdA1ba5_yQks5swRr9gaJpZM4QIXp7 .

lecy commented 6 years ago

I just did a quick search in the source files - the base spreadsheet david created for the datathon and the spreadsheet miguel contributed, and did not find these in either.

https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/tree/master/development

is there something unique about the way the path is presented that would have caused a problem?

borenstein commented 6 years ago

So we have not had these since May? Yikes!

-- David Bruce Borenstein, PhD 781.710.2789 (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 8:32 PM, Jesse Lecy notifications@github.com wrote:

I just did a quick search in the source files - the base spreadsheet david created for the datathon and the spreadsheet miguel contributed, and did not find these in either.

https://github.com/Nonprofit-Open-Data-Collective/irs- efile-master-concordance-file/tree/master/development

is there something unique about the way the path is presented that would have caused a problem?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14#issuecomment-339839123, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn-obzK-lnF1PPL--WOJ-sDC2HjJEks5swSS5gaJpZM4QIXp7 .

borenstein commented 6 years ago

I don't think this is necessarily a bug. The check box's element only appears in the return if it is actually checked. Not all of the items in the schema are actually represented in the data set, and we deliberately excluded anything that was indicated in the schema but not attested in the data.

I just checked for several of the other check box fields (eg Organization501c3Ind, AllAffiliatesIncludedInd) and I found that they were indeed there as expected. So unless we can find an XML file that contains the 527 field, I'm going to say the program performed correctly.

I'm going to close this ticket for now. If we do find the a 990 containing the missing 527 field, then there is indeed a bug and I will need to investigate.

-- David Bruce Borenstein, PhD 781.710.2789 (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 8:36 PM, David Bruce Borenstein < david.bruce.borenstein@gmail.com> wrote:

So we have not had these since May? Yikes!

-- David Bruce Borenstein, PhD 781.710.2789 <(781)%20710-2789> (m) https://www.linkedin.com/in/davidborenstein

On Thu, Oct 26, 2017 at 8:32 PM, Jesse Lecy notifications@github.com wrote:

I just did a quick search in the source files - the base spreadsheet david created for the datathon and the spreadsheet miguel contributed, and did not find these in either.

https://github.com/Nonprofit-Open-Data-Collective/irs-efile- master-concordance-file/tree/master/development

is there something unique about the way the path is presented that would have caused a problem?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14#issuecomment-339839123, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn-obzK-lnF1PPL--WOJ-sDC2HjJEks5swSS5gaJpZM4QIXp7 .

jsfenfen commented 6 years ago

Yeah, I see no instances in the 2013 and on era of this variable ever appearing. Whether or not this is a bug, it's definitely worth following up. I believe 527 orgs have the option of filing 8871/8872 reports as well, but unclear if that removes this requirement?

lecy commented 6 years ago

That makes sense, and I'm good with closing the ticket for now.

I'm just glad we are not missing 527 check boxes!

Jesse Lecy Associate Professor • Arizona State University Academic Director: M.S. in Program Evaluation and Data Analytics M: +1 801 462 1648 / *@: jdlecy@asu.edu / W: *publicservice.asu.edu

On Thu, Oct 26, 2017 at 7:45 PM, Jacob Fenton notifications@github.com wrote:

Yeah, I see no instances in the 2013 and on era of this variable ever appearing. Whether or not this is a bug, it's definitely worth following up. I believe 527 orgs have the option of filing 8871/8872 reports as well, but unclear if that removes this requirement?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14#issuecomment-339857408, or mute the thread https://github.com/notifications/unsubscribe-auth/ABJzCyjeHRI_EH5C1uPzhpYFW2lOBbC4ks5swUO2gaJpZM4QIXp7 .

jsfenfen commented 6 years ago

Not sure what to make of this pdf 900817449-990POL-01.pdf

This is an efile generated graphic for ty 2012 with the 527 box checked. What's odd is that I got it from the IRS 527 search, which said that it was a paper result, even though clearly it's generated. image

borenstein commented 6 years ago

Is it in the Amazon index?

On Oct 27, 2017 12:45 AM, "Jacob Fenton" notifications@github.com wrote:

Not sure what to make of this pdf 900817449-990POL-01.pdf https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/files/1420798/900817449-990POL-01.pdf

This is an efile generated graphic for ty 2012 with the 527 box checked. What's odd is that I got it from the IRS 527 search, which said that it was a paper result, even though clearly it's generated. [image: image] https://user-images.githubusercontent.com/639012/32088282-e8cec728-ba96-11e7-809d-b2fc206c28fe.png

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/Nonprofit-Open-Data-Collective/irs-efile-master-concordance-file/issues/14#issuecomment-339871319, or mute the thread https://github.com/notifications/unsubscribe-auth/AEPgn6bp3SuqOcYDx5z7Q7cTHDrW5TsWks5swV_zgaJpZM4QIXp7 .

jsfenfen commented 6 years ago

Nope it looks like the efiled 990's from 527 orgs have been systematically excluded from the amazon dump? Ask me about the file Sean O'Reilly took down showing the number of 990s submitted electronically but not released electronically.

Here's another: doh, the link doesn't work here's the file (it's 2015): PDF File

And another 460769544-990POL-05.pdf

jsfenfen commented 6 years ago

I'm closing this for now, as it doesn't seem to be a problem with the code reading the xml, but a larger problem.