ddrindex publish improperly publishes "in progress" objects

[GeoffFroh]

ddr-densho-441 is marked as inprocess and public and live on the public site. this should not be live. 441 was created june 15 2018 with a last mod date of july 29 2019

confirmed that ddrindex publishes status==inprocess entities.

(first reported in: https://github.com/densho/ddr-public/issues/129)

[GeoffFroh]

@pkikawa

TODO: try publishing a test entity and a test collection with status==inprocess to stage

[pkikawa]

Testing results

attempting to publish a collection with the collection status set to inprocess results in it NOT being published. similarly, attempting to publish child entities, when their parent object is set to inprocess results in the child NOT being published, but the parent object will still be published.

[pkikawa]

Test with a single inprocess entity with no child entities using ddr-testing-40289 resulted in the correct behavior for ddrindex -- i.e., did not publish the entity.

[GeoffFroh]

In summary:

• collection set to inprocess did not publish the collection or any entities (PASS)
• entity set to inprocess with no child entities did not publish (PASS)
• entity set to inprocess with child entities did not publish child entities, but did publish parent entity (FAIL)

[gjost]

Thx for the summary, was trying to figure things out.

[pkikawa]

2019-08-08 09:20:27.614265-07:00 | 1/5 SKIP ddr-testing-40289-2 parent unpublishable
2019-08-08 09:20:27.628923-07:00 | 2/5 SKIP ddr-testing-40289-1 parent unpublishable
2019-08-08 09:20:27.638252-07:00 | 3/5 SKIP ddr-testing-40289-1-2 parent unpublishable
2019-08-08 09:20:27.648155-07:00 | 4/5 SKIP ddr-testing-40289-1-1 parent unpublishable
2019-08-08 09:20:27.657055-07:00 | 5/5 SKIP ddr-testing-40289 parent unpublishable
{'successful': 0, 'skipped': 5, 'total': 5, 'bad': []}

^^ from attempting to publish a collection marked as in-process

[pkikawa]

2019-08-08 09:22:10.857851-07:00 | 1/5 SKIP ddr-testing-40289-2 status
2019-08-08 09:22:10.871454-07:00 | 2/5 SKIP ddr-testing-40289-1 status
2019-08-08 09:22:10.881261-07:00 | 3/5 SKIP ddr-testing-40289-1-2 parent unpublishable
2019-08-08 09:22:10.891571-07:00 | 4/5 SKIP ddr-testing-40289-1-1 parent unpublishable
2019-08-08 09:22:10.900637-07:00 | 5/5 POST ddr-testing-40289
{'successful': 1, 'skipped': 4, 'total': 5, 'bad': []}

where the parent entity and children were inprocess. And the children were not published but the parent was

[pkikawa]

though in actuality those parents were not published : https://ddrstage.densho.org/ddr-testing-40289-1/

[GeoffFroh]

UPDATE: in re-testing, we were unable to replicate the failure behavior.

We surmise that the original improperly published data (ddr-densho-1000-441) was likely indexed a long time ago when this bug may have existed in ddrindex; but that the issue does not exist in the current version.

@gjost is taking the opportunity to add a unit test to the DDR.docstore._publishable function, however.