unable to create new collections via either webui or new-manual

[pkikawa]

This issue has been around for awhile on older versions (2.8.0MA) but had only affected certain workstations (silverstar, pahto - the physical machines themselves - not vm or user specific) but has since expanded to affect all workstations tested so far (loowit, rainier, silverstar, argonaut, dragontail) on the current master version 2.8.6

clicking on the new collection button results in a Page not found (404) the url of the page is http://192.168.56.101/ui/collection/ddr-testing/new-idservice/

using the new-manual method results in the same manner. (same 404 on the same url) local.log local.log

[pkikawa]

celeryd.log

[gjost]

The real problem is a 502 Bad Gateway error somewhere. Note the fine print saying that 404 is being shown because there is no 502 template: "The current path, 502.html, didn't match any of these." Note to self: add a 502 template (https://github.com/densho/ddr-local/issues/260).

I'm unable to duplicate this bug on my end - I was able to successfully create new collections using both the new-idservice and new-manual methods. Here's info on the setup I tested with:

Rendered at 2019-06-10T11:08:15.970740-07:00 by PID 14715 on deb9a
loc: 6fe987c3db7e87a81ee4ee7ba56f71765cf244dc  (HEAD -> master, tag: v2.8.6, origin/master) 2019-05-29 15:50:07 -0700
cmd: f1a3064 (HEAD -> master, tag: v2.8.6, origin/master, origin/HEAD)
def:   63f9ee8 (HEAD -> master, tag: v2.8.5, tag: v2.8.3, tag: v2.8.2, ...
def: /opt/ddr-local-develop/ddr-defs/

[pkikawa]

I fired up the old vm (2.8.0 MA which was working) and it no longer works for me. 2.8.6 HAD been working until sometime in the last (couple?) week(s)

[pkikawa]

at this time, we have no way of creating new collections from the web ui via the button or new-manual

[gjost]

That the error is being served by Django and not by Nginx indicates that the ddrlocal Django process is trying to connect to another server. My guess is the ID service, but it might possibly be Celery.

[gjost]

Apologies: This error is because Nginx is getting a 502 when it tries to connect with the main ddrlocal Django process. There's some timeout or error happening inside the new-idservice and new-manual views. I'll review those and see what might be causing that.

[gjost]

I'm working on a couple changes to help surface the cause of this error. One is to remove the whole 404 on 502.html error. Another is to add timeouts to all outgoing HTTP requests from the ddrlocal process that are shorter than the Nginx timeouts. Hopefully the result will be a 500 error message that tells us which URL request (if any) is causing the problem.

[pkikawa]

ddrlocal.log nginx log

[gjost]

The Nginx logs tell us that POST requests to new-idservice and new-manual are returning 502 errors.

[pkikawa]

on 2.8.7-2DEV now getting 502 bad gateway on http://192.168.56.101/ui/collection/ddr-testing/new-manual/

ddrlocal.zip celeryd.log local.log

[gjost]

In local.log it looks like the new-idservice procedure cuts off around 2019-06-10 14:22:21,688. There are two ssh git@mits.densho.org info calls and then nothing. In the following new-manual procedure ssh git@mits.densho.org info is followed by git clone git@mits.densho.org:ddr-testing-40250.git and other operations.

[gjost]

Collection and entity creation happens within the HTTP request cycle. I've been meaning to move these out to Celery tasks (https://github.com/densho/ddr-local/issues/253). Maybe it's time to do that.

[pkikawa]

on 2.8.7-3DEV - created a new collection via the new collection button - collection was successfully created and the idservice updated, but i was dumped at this page rather than being directed to the edit page: UnboundLocalError at _uicollections.zip

using the new-manual method resulted in the successful creation of the collection, but the only indication of this was the gitstatus notification. after entering the ID on the new-manual page i was sent back to the collections page - not the edit page for the newly created collection.

[gjost]

So... a step closer then?

[pkikawa]

On dragontail, the behavior i am seeing is different: instead of throwing the unboundlocalerrror it just dumps me back at the collections page with the gitstaus notifying me when the collection is created (successfully)

creating a new collection via new-manual on dragontail exhibits identical behavior as on my workstaion (rainier)

[gjost]

The reason it sends you to the collections screen is that it's immediately sending the collection create process to the Celery backend. The page has to do something and even if it knows the collection ID (for new-idservice it doesn't) it can't redirect to a collection that doesn't exist.

Ultimately (https://github.com/densho/ddr-local/issues/253) the plan is to have an intermediate page with a spinner or something and to redirect to the new collection once it's created, but I don't know how to do that yet. At least now collections are getting created?

[pkikawa]

yes, collections are being successfully created.

[GeoffFroh]

We need to have a UI message that let you know the collection had been created and give a link to the page. Otherwise, there is no way of knowing whether it worked (especially since the collections list page is cached so the new collection does not show up immediately).

[pkikawa]

^^ the gitstatus notifications can fulfill this need for now?

[gjost]

The gitstatus messages contain a link to the new collection when a collection has been successfully created.