Add sub-agencies back to Department of Energy in inventory.data.gov

[hkdctol]

User Story

In order to get a consolidated data.json, Department of Energy needs to have sub-agencies added to the admin UI on inventory.data.gov.

Acceptance Criteria

• [x] GIVEN a user for Department of Energy goes to admin UI for main DOE organization THEN the field for sub-agencies is populated with the names of sub-agencies (other organizations within inventory for the agency)

Background

With the move to new inventory.data.gov, looks like the sub-agencies field and values were lost for some organizations.

Sketch

Follow the pattern on the organization that did not have the sub-agencies field cleared as a result of the new inventory launch. See screenshot for example.

[mogul]

We need to verify whether we've actually solved this problem.

[hkdctol]

This doesn't seem to be solved. Department of Energy contact reported (and I confirmed) when you download a data.json from main Department of Energy organization in inventory.data.gov, you get a small file 59Kb - it's not the consolidated data.json for all the sub-agencies in Department of Energy, even though we've added the sub-agencies through the admin UI for Department of Energy organization.

[FuhuXia]

@hkdctol I did not see sub-agencies was added to DOE. I added it just now and gave it value as follows then I got data.json of size 778Kb.

olm-doe,fossil-energy,cfo-energy-gov,chco-energy-gov,eere-gov,em-energy-gov,eia-gov,nnsa-gov,gc-doe-gov,osti-gov,doe-ma,nrel-gov,ornl-energy-gov,office-of-nuclear-energy-department-of-energy,office-of-electricity

Do you need me to go thru all other organizations and add back missing sub-agencies values?

[hkdctol]

@FuhuXia thanks let's discuss tomorrow so we're clear on what happened with this.

[FuhuXia]

DOE lost its sub-agencies field. Happened a few times over the past a few months. Need to troubleshoot why the Custom Field sub-agencies does not stick.

The first thing to look is to see whether script https://github.com/GSA/ckanext-dcat_usmetadata#publishers-import ignores sub-agencies field during its organization update.

[UPDATE] It is verified that every time publishers-import is run, sub-agencies and any other extras field are lost.

[FuhuXia]

Fix is verified on prod. DOE sub-agencies stays after publishers-import is run.

The following is a list of organizations that have custom fields sub-agencies but their values are lost due to this bug. We need to manually add them back.

organization  | eda-doc-gov
sub-agencies  | department-of-commerce,bea-gov,bis-doc-gov,census-bureau,trade-gov,national-technical-information-service-department-of-commerce,ntia-doc-gov,uspto-gov,national-oceanic-and-atmospheric-administration-department-of-commerce

organization  | bis-doc-gov
sub-agencies  | nist-gov,national-oceanic-and-atmospheric-administration-department-of-commerce

organization  | department-of-commerce
sub-agencies  | bea-gov,bis-doc-gov,census-bureau,trade-gov,national-technical-information-service-department-of-commerce,ntia-doc-gov,uspto-gov,national-oceanic-and-atmospheric-administration-department-of-commerce

organization  | department-of-energy
sub-agencies  | olm-doe,fossil-energy,cfo-energy-gov,chco-energy-gov,eere-gov,em-energy-gov,eia-gov,nnsa-gov,gc-doe-gov,osti-gov,doe-ma,nrel-gov,ornl-energy-gov,office-of-nuclear-energy-department-of-energy,office-of-electricity

organization  | usda-gov
sub-agencies  | agricultural-marketing-service-department-of-agriculture,aphis-usda-gov,ers-usda-gov,fas-usda-gov,fns-usda-gov,fsa-usda-gov,fsis-usda-gov,grain-inspection-packers-stockyards-administration,nda-usda-gov,nass-usda-gov,nifa-usda-gov,nrcs-usda-gov,stl-usda-gov,risk-management-agency-department-of-agriculture

organization  | department-of-labor
sub-agencies  | us-department-of-labor-benefits-gov-program,us-department-of-labor-bureau-of-labor-statistics,us-department-of-labor-mine-safety-and-health-administration,us-department-of-labor-employee-benefits-security-administration,us-department-of-labor-employment-and-training-administration,us-department-of-labor-office-of-job-corps,us-department-of-labor-occupational-safety-and-health-administration,us-department-of-labor-office-of-federal-contract-compliance-programs,us-department-of-labor-office-of-public-affairs,us-department-of-labor-office-of-labor-management-standards,us-department-of-labor-office-of-the-assistant-secretary-for-policy,us-department-of-labor-office-of-the-solicitor,us-department-of-labor-office-of-worker-s-compensation-program,us-department-of-labor-office-of-workers-compensation-program,us-department-of-labor-wage-and-hour-division,us-department-of-labor-bureau-of-international-affairs,us-department-of-labor-veterans-employment-and-training-service,u-s-department-of-labor-office-of-the-chief-financial-officer

[hkdctol]

Thanks @FuhuXia this tracks with what I see when looking through inventory.data.gov organizations