Implement record submission flow

[torrin47]

From @torrin47 on August 31, 2018 23:11

Upon completion a user will be able to click a submit button that sends the finished document to the EDG team for review and posting to the EDG. How does this work? Do we simply send the json file as an email to edg@epa.gov via server CGI script (python example already implemented for EnviroAtlas) or something similar? Is there any more elegant solution? If we are implementing an ORCID Authentication, could we allow record to be directly posted into EDG for review? If we do, we'd still need some sort of notification.

_Copied from original issue: USEPA/EPA_Environmental_DatasetGateway#66

[torrin47]

Extremely bare-bones submission handler configured and working on EDG server, currently performs no validation and sends submission only to Torrin. Accepts 4 parameters - publisher, sponsor, metadata, and token:

https://edg.epa.gov/nongeoeditor/submithandler/sendMetadata.py?token=recaptchaToken&sponsor=greene.ana@epa.gov&metadata=%22This%20metadata%20is%20excellent!%22&publisher=External%20Researcher

I'm expecting that the token will be whatever we decide on for CAPTCHA. reCaptcha seems like a pretty reasonable solution: https://developers.google.com/recaptcha/intro but I'm open to alternatives if you have any to recommend @aergul. Once we decide on a solution, I'll implement the backend verification and wire up the sponsor email. Right now it will return a boolean status message of success or failure, it wouldn't be hard to make the errors more helpful, but I'm not sure whether we really want to.

[aergul]

First attempt appeared to be successful:

Nice. Expect many more...

[torrin47]

Roger that!

[aergul]

Uploaded a somewhat primitive version. Succeeds submitting a record with little entry. Failed on a rather large record with HTTP 404 NOT FOUND. Large record attached. @torrin47

4b74aab6-d0ce-456c-a73b-68521dc16e80 (1).json.txt

[torrin47]

Ok, that's not unexpected for a GET request, but the endpoint should also accept POST - should be an easy switch?

[aergul]

Now POSTing, no more 404 which is great. Still getting failure, as in {"status":"failure"}, with above record. It's valid per pod validator but it does have two datasets, is that the problem? @torrin47

Deployed latest, if you would like to experiment.

[torrin47]

I added a basic error message to the response, which helps slightly. The "Not Indexable" message indicates it's not able to find a parameter - in this case I believe it's looking for "metadata" in the payload, but when I look in Chrome dev tools, I'm seeing the raw record as the payload, not referenced as "metadata". This stackoverflow posting seems to clarify slightly:

https://stackoverflow.com/questions/23118249/whats-the-difference-between-request-payload-vs-form-data-as-seen-in-chrome

form = cgi.FieldStorage() metadata = form.getvalue('metadata')

When I test using Postman, I'm able to get it to work with both Content-Type: application/x-www-form-urlencoded and Content-Type: multipart/form-data but all of the elements are necessary.

[aergul]

Ah, ok, didn't know it was expecting multi-part form submission. Now it works. At least on this end. Your side?

[torrin47]

Yeah, neither did I until we saw the errors. Working smoothly now, I'm receiving the emails... and realizing that I probably need to work on the formatting to not confuse the sponsors.
One adjustment, the tool shows a success message following submission, but the Submit to EPA modal stays open - can we have that close if the submission is successful?

[torrin47]

Oh, i'm still getting sponsor hardcoded as greene.ana@epa.gov - can we wire that up to epa_contact? It's not sending to them yet, but I do print out that element for testing.

[aergul]

I wasn't sure about what you wanted to with it. Right now sponsor is optional. Do we want to make it mandatory or do we want to keep it optional for validation purposes but required for submit purposes? The latter requiring some custom logic.

[torrin47]

I think sponsor is going to be mandatory for external users. No need for custom logic.

[aergul]

Sponsor now mandatory. Modal closes if submission successful.

[torrin47]

Awesome.