Determine what we should do with empty assistance listings

[chouinar]

Summary

Opportunities have assistance listings which consist of a program title + assistance listing number. An opportunity can have several of these assistance listings.

The legacy data has tens of thousands of these records (~15% of all records) where both values are blank. There doesn't appear to be any purpose to these values. They filter it out on their frontend.

Should we just skip importing these records entirely?

Acceptance criteria

• [ ] Figure out whether we should keep these
• [ ] If we decide to filter them out, implement that in the transformation logic

[mxk0]

@margaretspring to intro @mdragon and @mxk0 to HHS stakeholder for product discovery.

[chouinar]

Adding an example:

Here is what data we have for an assistance listing coming out of our API at the moment when we hit this case:

Here is how the existing grants.gov website renders that data:

I need to check their APIs, but as far as I can tell from the logic, the API would return these empty assistance listings as well, and their front-end just doesn't display them.

[chouinar]

In an unrelated discussion, I found at least some clue to why these blank assistance listings exist. The assistance listings are used to link an opportunity to the TCOMPETITION table and then to various application package tables (all of which we don't consume yet).

In some cases, its linked through these null assistance listings (and the code even references nullCFDA as a concept a few times. Why exactly it is needed, I'm not sure, but it looks like they do serve some other purpose.

You can see this a bit on the API.

This opportunity gets to its package via an ordinary assistance listing number https://grants.gov/search-results-detail/355417

This opportunity gets to its package via the weird null one https://grants.gov/search-results-detail/356641

I think the question now is - why?