Closed BlackthornYugen closed 3 years ago
You're trying to parse HTML using an XML parser. Unfortunately that won't work out of the box. You can just google your error to see extensive discussions of why HTML escape sequences are not valid XML.
There are many hacky workarounds for this, but none are built in to the Python standard library XML parsers. I'm on the fence about what to do here, since I'd like yq to be able to parse HTML files out of the box, but I'd like to find a minimally hacky way to configure the XML parser to do so.
Ah, so & isn't valid?
╭─🕐─>
╰─$ echo "<xml>nbsp;</xml>" | xq 1 ↵
{
"xml": "nbsp;"
}
╭─🕐─>
╰─$ echo "<xml> </xml>" | xq
xq: Error running jq: ExpatError: undefined entity: line 1, column 5.
Not sure what the least hacky way to solve this would be in yq. For now passing through sed first is fine for me. Here's what I went with:
function parseNexus() {
curl "https://nexus.mysite.tld/service/rest/repository/browse/${1}/" -s | \
tidy -asxml 2> /dev/null | \
sed 's/ //' | \
xq -r '.html.body.table.tr | map(select(. | has("td") and (.td | length) == 4 )) | .[] | .td | .[0] | .a["@href"]'
}
PS: I know that nexus has a rest api; but I didn't want to deal with their pagination stuffs. :)
Tidy seems happy with my XML but xq won't parse until I remove
.