Elements: Distribution

[torrin47]

From @torrin47 on September 10, 2018 23:44

At least one distribution URL is mandatory, but the editor must allow a user to specify an unlimited number of distribution sections, and every URL must have a corresponding title. EPA doesn't specify any additional guidance above and beyond the project open data schema: https://project-open-data.cio.gov/v1.1/schema/#distribution although the referenced list of IANA Media Types is so epically long, we've only offered users a very short list of choices: text/plain, text/csv, text/html, application/xml, application/zip. Maybe we show the short list and then allow them to enter custom types if they wish? The whole concept is a little goofy, as many downloads are zip files but a user might really benefit from knowing the format(s) of the file(s) inside the zip.

_Copied from original issue: USEPA/EPA_Environmental_DatasetGateway#75

[torrin47]

In the EPA tech spec, Distribution URL pretty much maps to an entire Dataset Distribution section in a POD record, with either a downloadURL or accessURL populated. I'm again puzzled over why the tech spec says no equivalent - I'm going to fix that now. I have mixed feelings about this Dataset Distribution section. I love that a distribution URL can have a title and description, and that a dataset can have an infinite number of distributions. I think it's odd and unhelpful that there is a choice of URL fields, and the definitions or conformsTo, describedBy, format, and mediaType are ambiguous and overlapping- the schema would be more powerful with a single URL and better standards around those elements that characterize the URL. But for now they're what we have to work with. For the purposes of this tool, I think I'd be fine with grouping all of the distribution subelements under a single guidance element, rather than giving each distribution subelement a separate help. Not quite sure how that'll fly, but it's my initial inclination.

[torrin47]

Access/distribution URLs will be mutually exclusive (via a toggle?) and all other elements plain text. Title and a URL are mandatory, everything else is optional.
MediaType lookup will be separate enhancement ticket.

[aergul]

Distribution implemented. Currently it's the first element on the page for testing convenience. User can add multiple distribution entries. Only one entry can be editable at any given time. Validation section displays validation results for currently edited entry only if one is being edited. If not, validation results for the first entry that's not validating is displayed.

Probably needs some tweaking...

[torrin47]

Wow, looks great. Clean, straightforward, intuitive. Validation seems to make sense. And I see the subtle color coding within each entry to match the top level status icons. Solid. That just leaves the optional elements?

[jzichichi]

@aergul - I really like how this works - the validation works well and is intuitive. Hopefully the addition of the elements Torrin mentions above won't make the clean approach more clunky, as it's nice and straightforward at the moment.

[aergul]

The optional elements are implemented per @torrin47 above and icons during edit per @jzichichi as discussed on standup.

The visuals and validation at both field level and distribution section level could use some discussion.

[jzichichi]

@aergul - I like it. Moving over for @torrin47 to review. Will note for today's meeting.

[torrin47]

Capturing notes from today's call.

When AccessURL is selected:

• Let's display Format as a binary choice between API and Other, with free text next to the Other option. This is to comply with the POD guidance that "Format" should be set to "API" for any web APIs.
• Hide mediaType element.
• Show optional conformsTo, describedBy and describedByType elements

When DownloadURL is selected:

• mediaType element becomes mandatory.
• Format element is hidden, but is populated by label of mediaType value.
• Show optional conformsTo, describedBy and describedByType elements

conformsTo and describedBy elements should validate as URLs.

[aergul]

addressed

[jzichichi]

@aergul @torrin47, I believe these are all addressed with the only item that I couldn't verify being the requirement "free text next to the Other option". It seemed that when I selected "Other" for the format choice, I didn't have a choice to also enter free text, but it's also possible that I'm overlooking something. Will put in the next column for @torrin47 to review

[torrin47]

Looking very close. I'm seeing the free-text space when Other is chosen, and am realizing we probably need slightly different wording in the prompt. Perhaps: "Please enter a human-readable description of this indirect means of accessing the data such as: web page, request form, or query tool."

conformsTo still isn't validating as a URL. Otherwise everything looks great!

[jzichichi]

@torrin47 - conformsTo indicates URI for this field: https://project-open-data.cio.gov/v1.1/schema/#conformsTo

Do we want to intentionally limit this field to a URL?

[torrin47]

Yes, this is a case where data.gov's validator deviates from the documentation. If you try entering anything other than a URL in this field and run it through their validator: https://labs.data.gov/dashboard/validate it causes the record to be considered invalid. We've never had anyone use this element, so it's not the hill we want to die on.

[torrin47]

Minor note, but can we place Distribution between Publisher Email and Identifier?

[aergul]

conformsTo validates as URL and Distribution moved

[jzichichi]

@aergul - looks like you covered both; moving over for @torrin47 to review

[torrin47]

Should have grey "empty/optional" icon at pageload. Otherwise perfect, thanks!

[jzichichi]

default for page load looks good; when I click on add new distribution entry, the icon shows the entry as valid, even without content entered.

[jzichichi]

better. Moving over

[torrin47]

👍