ietf-ribose / bibxml-data-iana

BibXML bibliographic data for IANA registries
0 stars 1 forks source link

Potentially create a full dataset for IANA registries #1

Closed ronaldtse closed 3 years ago

ronaldtse commented 3 years ago

There is a limited number of IANA registries -- they rarely change.

$ wget https://www.iana.org/protocols
$ grep -o 'href="/assignments/.*"' protocols | cut -d '"' -f 2 | cut -d '#' -f 1 | sort | uniq | wc -l
    581

According to the mechanism used by today's IANA fetching script: https://trac.ietf.org/trac/xml2rfc/browser/website/public/rfc/bibxml-iana/nph-index.pl

Every registry corresponds to a single BibXML entry: https://xml2rfc.tools.ietf.org/public/rfc/bibxml-iana/reference.IANA.sip-parameters.xml

=>

<reference anchor="SIP-PARAMETERS" target="http://www.iana.org/assignments/sip-parameters">
<front>
<title>Session Initiation Protocol (SIP) Parameters</title>
<author>
<organization>IANA</organization>
</author>
<date/>
</front>
</reference>

Given that there are only 581 registries, performing 581 requests per-day isn't so bad. Instead of needing a 24-hour expiring cache, it would be more effective to scan the 581 pages daily and generate a static BibXML dataset.

These are the 581 registries today:

/assignments/_6lowpan-parameters/_6lowpan-parameters.xhtml
/assignments/_6tisch/_6tisch.xhtml
/assignments/aaa-parameters/aaa-parameters.xhtml
/assignments/abfab-parameters/abfab-parameters.xhtml
/assignments/about-uri-tokens/about-uri-tokens.xhtml
/assignments/acap-registrations/acap-registrations.xhtml
/assignments/access-types/access-types.xhtml
/assignments/ace/ace.xhtml
/assignments/acme/acme.xhtml
/assignments/acp/acp.xhtml
/assignments/address-family-numbers/address-family-numbers.xhtml
/assignments/address-literal-tags/address-literal-tags.xhtml
/assignments/adsp-parameters/adsp-parameters.xhtml
/assignments/aead-parameters/aead-parameters.xhtml
/assignments/aka-version-namespace/aka-version-namespace.xhtml
/assignments/alert-urns/alert-urns.xhtml
/assignments/alto-protocol/alto-protocol.xhtml
/assignments/amtrelay-resource-record/amtrelay-resource-record.xhtml
/assignments/ancp/ancp.xhtml
/assignments/aodv-parameters/aodv-parameters.xhtml
/assignments/apex/apex.xhtml
/assignments/arp-parameters/arp-parameters.xhtml
/assignments/as-numbers/as-numbers.xhtml
/assignments/audio-telephone-event-registry/audio-telephone-event-registry.xhtml
/assignments/auth-namespaces/auth-namespaces.xhtml
/assignments/authentication-cryptographic-protocol-id/authentication-cryptographic-protocol-id.xhtml
/assignments/authentication-method-reference-values/authentication-method-reference-values.xhtml
/assignments/auto-response-parameters/auto-response-parameters.xhtml
/assignments/auto-submitted-keywords/auto-submitted-keywords.xhtml
/assignments/babel/babel.xhtml
/assignments/bandwidth-constraints-model-ids/bandwidth-constraints-model-ids.xhtml
/assignments/battery-technologies/battery-technologies.xhtml
/assignments/beep-parameters/beep-parameters.xhtml
/assignments/bfcp-parameters/bfcp-parameters.xhtml
/assignments/bfd-parameters/bfd-parameters.xhtml
/assignments/bgp-data-collection-communities-std/bgp-data-collection-communities-std.xhtml
/assignments/bgp-extended-communities/bgp-extended-communities.xhtml
/assignments/bgp-ls-parameters/bgp-ls-parameters.xhtml
/assignments/bgp-parameters/bgp-parameters.xhtml
/assignments/bgp-tunnel-encapsulation/bgp-tunnel-encapsulation.xhtml
/assignments/bgp-well-known-communities/bgp-well-known-communities.xhtml
/assignments/bier/bier.xhtml
/assignments/bmp-parameters/bmp-parameters.xhtml
/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml
/assignments/brski-parameters/brski-parameters.xhtml
/assignments/building-blocks/building-blocks.xhtml
/assignments/bundle/bundle.xhtml
/assignments/c-dns/c-dns.xhtml
/assignments/calipso/calipso.xhtml
/assignments/caller-id-plans/caller-id-plans.xhtml
/assignments/capability-codes/capability-codes.xhtml
/assignments/captive-portals/captive-portals.xhtml
/assignments/capwap-parameters/capwap-parameters.xhtml
/assignments/card-parameters/card-parameters.xhtml
/assignments/cbor-simple-values/cbor-simple-values.xhtml
/assignments/cbor-tags/cbor-tags.xhtml
/assignments/ccmp-parameters/ccmp-parameters.xhtml
/assignments/ccnx/ccnx.xhtml
/assignments/cddl/cddl.xhtml
/assignments/cdni-parameters/cdni-parameters.xhtml
/assignments/cert-rr-types/cert-rr-types.xhtml
/assignments/cfm-oam/cfm-oam.xhtml
/assignments/cga-message-types/cga-message-types.xhtml
/assignments/channel-binding-types/channel-binding-types.xhtml
/assignments/character-sets/character-sets.xhtml
/assignments/charset-info/charset-info.xhtml
/assignments/charset-reg/charset-reg.xhtml
/assignments/cisco-sla-protocol/cisco-sla-protocol.xhtml
/assignments/civic-address-types-registry/civic-address-types-registry.xhtml
/assignments/clue/clue.xhtml
/assignments/cnrp-parameters/cnrp-parameters.xhtml
/assignments/comp-meth-ids/comp-meth-ids.xhtml
/assignments/cont-disp/cont-disp.xhtml
/assignments/content-security-policy-directives/content-security-policy-directives.xhtml
/assignments/cops-parameters/cops-parameters.xhtml
/assignments/core-parameters/core-parameters.xhtml
/assignments/cose/cose.xhtml
/assignments/cpim-headers/cpim-headers.xhtml
/assignments/crypto-suites/crypto-suites.xhtml
/assignments/cwt/cwt.xhtml
/assignments/cxtp-parameters/cxtp-parameters.xhtml
/assignments/dane-parameters/dane-parameters.xhtml
/assignments/dcap-parameters/dcap-parameters.xhtml
/assignments/dccp-ccid2-parameters/dccp-ccid2-parameters.xhtml
/assignments/dccp-ccid3-parameters/dccp-ccid3-parameters.xhtml
/assignments/dccp-ccid4-parameters/dccp-ccid4-parameters.xhtml
/assignments/dccp-parameters/dccp-parameters.xhtml
/assignments/dcep-parameters/dcep-parameters.xhtml
/assignments/device-identification/device-identification.xhtml
/assignments/dhcpv6-parameters/dhcpv6-parameters.xhtml
/assignments/directory-system-names/directory-system-names.xhtml
/assignments/dkim-parameters/dkim-parameters.xhtml
/assignments/dlep-parameters/dlep-parameters.xhtml
/assignments/dmarc-parameters/dmarc-parameters.xhtml
/assignments/dncp-registry/dncp-registry.xhtml
/assignments/dns-key-rr/dns-key-rr.xhtml
/assignments/dns-parameters/dns-parameters.xhtml
/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xhtml
/assignments/dns-sshfp-rr-parameters/dns-sshfp-rr-parameters.xhtml
/assignments/dnskey-flags/dnskey-flags.xhtml
/assignments/dnssec-nsec3-parameters/dnssec-nsec3-parameters.xhtml
/assignments/dots/dots.xhtml
/assignments/ds-rr-types/ds-rr-types.xhtml
/assignments/dscp-registry/dscp-registry.xhtml
/assignments/dskpp/dskpp.xhtml
/assignments/dsn-types/dsn-types.xhtml
/assignments/dsr-parameters/dsr-parameters.xhtml
/assignments/dssc/dssc.xhtml
/assignments/eap-channel-binding-parameters/eap-channel-binding-parameters.xhtml
/assignments/eap-eke/eap-eke.xhtml
/assignments/eap-fast-parameters/eap-fast-parameters.xhtml
/assignments/eap-gpsk-parameters/eap-gpsk-parameters.xhtml
/assignments/eap-ikev2-payloads/eap-ikev2-payloads.xhtml
/assignments/eap-numbers/eap-numbers.xhtml
/assignments/eap-pax/eap-pax.xhtml
/assignments/eap-psk-parameters/eap-psk-parameters.xhtml
/assignments/eap-pwd-parameters/eap-pwd-parameters.xhtml
/assignments/eap-sake-parameters/eap-sake-parameters.xhtml
/assignments/eappotp-identifiers/eappotp-identifiers.xhtml
/assignments/eapsimaka-numbers/eapsimaka-numbers.xhtml
/assignments/ebml/ebml.xhtml
/assignments/ecml-parameters/ecml-parameters.xhtml
/assignments/ecmlv2-parameters/ecmlv2-parameters.xhtml
/assignments/email-auth/email-auth.xhtml
/assignments/emergency-call-additional-data/emergency-call-additional-data.xhtml
/assignments/emergency-call-metadata-control-data/emergency-call-metadata-control-data.xhtml
/assignments/emsk-parameters/emsk-parameters.xhtml
/assignments/enum-services/enum-services.xhtml
/assignments/epp-extension-role-values/epp-extension-role-values.xhtml
/assignments/epp-extensions/epp-extensions.xhtml
/assignments/epp-repository-ids/epp-repository-ids.xhtml
/assignments/ethernet-numbers/ethernet-numbers.xhtml
/assignments/evpn/evpn.xhtml
/assignments/ex-mobility-subtypes/ex-mobility-subtypes.xhtml
/assignments/fc-port-types/fc-port-types.xhtml
/assignments/fcast/fcast.xhtml
/assignments/fedfs-parameters/fedfs-parameters.xhtml
/assignments/flow-spec/flow-spec.xhtml
/assignments/flute-parameters/flute-parameters.xhtml
/assignments/foobar-af-numbers/foobar-af-numbers.xhtml
/assignments/forces/forces.xhtml
/assignments/ftp-commands-extensions/ftp-commands-extensions.xhtml
/assignments/g-ach-parameters/g-ach-parameters.xhtml
/assignments/gdoi-payloads/gdoi-payloads.xhtml
/assignments/geo-uri-parameters/geo-uri-parameters.xhtml
/assignments/geolocation-policy/geolocation-policy.xhtml
/assignments/geopriv-identifiers/geopriv-identifiers.xhtml
/assignments/gist-parameters/gist-parameters.xhtml
/assignments/gmpls-sig-parameters/gmpls-sig-parameters.xhtml
/assignments/gmpls-wson/gmpls-wson.xhtml
/assignments/gnss/gnss.xhtml
/assignments/grasp-parameters/grasp-parameters.xhtml
/assignments/gre-parameters/gre-parameters.xhtml
/assignments/gsakmp-parameters/gsakmp-parameters.xhtml
/assignments/gsmpv3-adapt/gsmpv3-adapt.xhtml
/assignments/gsmpv3-event/gsmpv3-event.xhtml
/assignments/gsmpv3-failure/gsmpv3-failure.xhtml
/assignments/gsmpv3-label/gsmpv3-label.xhtml
/assignments/gsmpv3-message/gsmpv3-message.xhtml
/assignments/gsmpv3-model/gsmpv3-model.xhtml
/assignments/gsmpv3-port/gsmpv3-port.xhtml
/assignments/gsmpv3-result/gsmpv3-result.xhtml
/assignments/gsmpv3-service/gsmpv3-service.xhtml
/assignments/gsmpv3-traffic/gsmpv3-traffic.xhtml
/assignments/gss-api-parameters/gss-api-parameters.xhtml
/assignments/gss-eap-parameters/gss-eap-parameters.xhtml
/assignments/gssapi-service-names/gssapi-service-names.xhtml
/assignments/gstn-extensions/gstn-extensions.xhtml
/assignments/hash-function-text-names/hash-function-text-names.xhtml
/assignments/held-parameters/held-parameters.xhtml
/assignments/hip-parameters/hip-parameters.xhtml
/assignments/hoba-device-identifiers/hoba-device-identifiers.xhtml
/assignments/hoba-key-identifiers/hoba-key-identifiers.xhtml
/assignments/hoba-signature-algorithms/hoba-signature-algorithms.xhtml
/assignments/http-alt-svc-parameters/http-alt-svc-parameters.xhtml
/assignments/http-authentication-control-parameters/http-authentication-control-parameters.xhtml
/assignments/http-authschemes/http-authschemes.xhtml
/assignments/http-cache-directives/http-cache-directives.xhtml
/assignments/http-dig-alg/http-dig-alg.xhtml
/assignments/http-methods/http-methods.xhtml
/assignments/http-parameters/http-parameters.xhtml
/assignments/http-status-codes/http-status-codes.xhtml
/assignments/http-upgrade-tokens/http-upgrade-tokens.xhtml
/assignments/http-warn-codes/http-warn-codes.xhtml
/assignments/http2-parameters/http2-parameters.xhtml
/assignments/http3-parameters/http3-parameters.xhtml
/assignments/iana-as-numbers-special-registry/iana-as-numbers-special-registry.xhtml
/assignments/iana-bfd-types/iana-bfd-types.xhtml
/assignments/iana-crypt-hash/iana-crypt-hash.xhtml
/assignments/iana-dns-class-rr-type/iana-dns-class-rr-type.xhtml
/assignments/iana-dots-signal-channel/iana-dots-signal-channel.xhtml
/assignments/iana-hardware/iana-hardware.xhtml
/assignments/iana-if-type/iana-if-type.xhtml
/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml
/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml
/assignments/iana-routing-types/iana-routing-types.xhtml
/assignments/iana-tunnel-type/iana-tunnel-type.xhtml
/assignments/ianaaddressfamilynumbers-mib/ianaaddressfamilynumbers-mib.xhtml
/assignments/ianabfdtcstd-mib/ianabfdtcstd-mib.xhtml
/assignments/ianacharset-mib/ianacharset-mib.xhtml
/assignments/ianaenergyrelation-mib/ianaenergyrelation-mib.xhtml
/assignments/ianaentity-mib/ianaentity-mib.xhtml
/assignments/ianafinisher-mib/ianafinisher-mib.xhtml
/assignments/ianagmplstc-mib/ianagmplstc-mib.xhtml
/assignments/ianaiftype-mib/ianaiftype-mib.xhtml
/assignments/ianaippmmetricsregistry-mib/ianaippmmetricsregistry-mib.xhtml
/assignments/ianaiprouteprotocol-mib/ianaiprouteprotocol-mib.xhtml
/assignments/ianaitualarmtc-mib/ianaitualarmtc-mib.xhtml
/assignments/ianalanguage-mib/ianalanguage-mib.xhtml
/assignments/ianamalloc-mib/ianamalloc-mib.xhtml
/assignments/ianamau-mib/ianamau-mib.xhtml
/assignments/ianaolsrv2linkmetrictype-mib/ianaolsrv2linkmetrictype-mib.xhtml
/assignments/ianapowerstateset-mib/ianapowerstateset-mib.xhtml
/assignments/ianaprinter-mib/ianaprinter-mib.xhtml
/assignments/ianasmf-mib/ianasmf-mib.xhtml
/assignments/ianastoragemediatype-mib/ianastoragemediatype-mib.xhtml
/assignments/ianatn3270etc-mib/ianatn3270etc-mib.xhtml
/assignments/iax-parameters/iax-parameters.xhtml
/assignments/icalendar/icalendar.xhtml
/assignments/ice/ice.xhtml
/assignments/icmp-parameters/icmp-parameters.xhtml
/assignments/icmpv6-parameters/icmpv6-parameters.xhtml
/assignments/idmef-parameters/idmef-parameters.xhtml
/assignments/idna-tables-11.0.0/idna-tables-11.0.0.xhtml
/assignments/idxp-options/idxp-options.xhtml
/assignments/ieee-802-numbers/ieee-802-numbers.xhtml
/assignments/ieee-std-802.15.4-ietf-ie-subtype-ids/ieee-std-802.15.4-ietf-ie-subtype-ids.xhtml
/assignments/iesg-recognized-organizations/iesg-recognized-organizations.xhtml
/assignments/igmp-type-numbers/igmp-type-numbers.xhtml
/assignments/igp-parameters/igp-parameters.xhtml
/assignments/ikev2-parameters/ikev2-parameters.xhtml
/assignments/im-srv-labels/im-srv-labels.xhtml
/assignments/imap-annotate-extension/imap-annotate-extension.xhtml
/assignments/imap-capabilities/imap-capabilities.xhtml
/assignments/imap-jmap-keywords/imap-jmap-keywords.xhtml
/assignments/imap-list-extended/imap-list-extended.xhtml
/assignments/imap-mailbox-name-attributes/imap-mailbox-name-attributes.xhtml
/assignments/imap-metadata/imap-metadata.xhtml
/assignments/imap-response-codes/imap-response-codes.xhtml
/assignments/imap-threading-algorithms/imap-threading-algorithms.xhtml
/assignments/imdn/imdn.xhtml
/assignments/inst-man-values/inst-man-values.xhtml
/assignments/integ-serv/integ-serv.xhtml
/assignments/iodef/iodef.xhtml
/assignments/iodef2/iodef2.xhtml
/assignments/iotp-codes/iotp-codes.xhtml
/assignments/ip-over-IEEE1394/ip-over-IEEE1394.xhtml
/assignments/ip-parameters/ip-parameters.xhtml
/assignments/ip-xns-mapping/ip-xns-mapping.xhtml
/assignments/ipfix/ipfix.xhtml
/assignments/ipp-registrations/ipp-registrations.xhtml
/assignments/ips-protocols/ips-protocols.xhtml
/assignments/ipsec-registry/ipsec-registry.xhtml
/assignments/ipseckey-rr-parameters/ipseckey-rr-parameters.xhtml
/assignments/ipv4-address-space/ipv4-address-space.xhtml
/assignments/ipv4-recovered-address-space/ipv4-recovered-address-space.xhtml
/assignments/ipv6-address-space/ipv6-address-space.xhtml
/assignments/ipv6-anycast-addresses/ipv6-anycast-addresses.xhtml
/assignments/ipv6-interface-ids/ipv6-interface-ids.xhtml
/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml
/assignments/ipv6-parameters/ipv6-parameters.xhtml
/assignments/ipv6-routeralert-values/ipv6-routeralert-values.xhtml
/assignments/ipv6-tla-assignments/ipv6-tla-assignments.xhtml
/assignments/ipv6-unicast-address-assignments/ipv6-unicast-address-assignments.xhtml
/assignments/isakmp-registry/isakmp-registry.xhtml
/assignments/iscsi-parameters/iscsi-parameters.xhtml
/assignments/isis-mt-parameters/isis-mt-parameters.xhtml
/assignments/isis-pdu/isis-pdu.xhtml
/assignments/isis-tlv-codepoints/isis-tlv-codepoints.xhtml
/assignments/isns-parameters/isns-parameters.xhtml
/assignments/jmap/jmap.xhtml
/assignments/jms-uri-variants/jms-uri-variants.xhtml
/assignments/jose/jose.xhtml
/assignments/jscalendar/jscalendar.xhtml
/assignments/jwt/jwt.xhtml
/assignments/kerberos-parameters/kerberos-parameters.xhtml
/assignments/kerberos-v-gss-api/kerberos-v-gss-api.xhtml
/assignments/keynote/keynote.xhtml
/assignments/keytable/keytable.xhtml
/assignments/kink-parameters/kink-parameters.xhtml
/assignments/l2tp-parameters/l2tp-parameters.xhtml
/assignments/lang-subtags-templates/lang-subtags-templates.xhtml
/assignments/lang-tag-apps/lang-tag-apps.xhtml
/assignments/language-subtag-registry/language-subtag-registry
/assignments/language-tag-extensions-registry/language-tag-extensions-registry
/assignments/language-tags/language-tags.xhtml
/assignments/lct-header-extensions/lct-header-extensions.xhtml
/assignments/ldap-parameters/ldap-parameters.xhtml
/assignments/ldp-namespaces/ldp-namespaces.xhtml
/assignments/leighton-micali-signatures/leighton-micali-signatures.xhtml
/assignments/lgr-dispositions/lgr-dispositions.xhtml
/assignments/link-relations/link-relations.xhtml
/assignments/lisp-parameters/lisp-parameters.xhtml
/assignments/lmp-parameters/lmp-parameters.xhtml
/assignments/loa-profiles/loa-profiles.xhtml
/assignments/locally-served-dns-zones/locally-served-dns-zones.xhtml
/assignments/location-type-registry/location-type-registry.xhtml
/assignments/lost-location-profiles/lost-location-profiles.xhtml
/assignments/ltp-parameters/ltp-parameters.xhtml
/assignments/machine-names/machine-names.xhtml
/assignments/madcap-parameters/madcap-parameters.xhtml
/assignments/mail-encoding/mail-encoding.xhtml
/assignments/mail-parameters/mail-parameters.xhtml
/assignments/managesieve/managesieve.xhtml
/assignments/manet-parameters/manet-parameters.xhtml
/assignments/marf-parameters/marf-parameters.xhtml
/assignments/markdown-variants/markdown-variants.xhtml
/assignments/masc-parameters/masc-parameters.xhtml
/assignments/mdn/mdn.xhtml
/assignments/media-control-channel/media-control-channel.xhtml
/assignments/media-feature-tags/media-feature-tags.xhtml
/assignments/media-type-structured-suffix/media-type-structured-suffix.xhtml
/assignments/media-type-sub-parameters/media-type-sub-parameters.xhtml
/assignments/media-types-parameters/media-types-parameters.xhtml
/assignments/media-types/media-types.xhtml
/assignments/megaco-h248/megaco-h248.xhtml
/assignments/message-header-types/message-header-types.xhtml
/assignments/message-headers/message-headers.xhtml
/assignments/message-store-events/message-store-events.xhtml
/assignments/method-tokens/method-tokens.xhtml
/assignments/mgcp-localconnectionoptions/mgcp-localconnectionoptions.xhtml
/assignments/mgcp-packages/mgcp-packages.xhtml
/assignments/mih/mih.xhtml
/assignments/mikey-payloads/mikey-payloads.xhtml
/assignments/milnet-parameters/milnet-parameters.xhtml
/assignments/mobileip-numbers/mobileip-numbers.xhtml
/assignments/mobility-parameters/mobility-parameters.xhtml
/assignments/mpls-id-type/mpls-id-type.xhtml
/assignments/mpls-label-values/mpls-label-values.xhtml
/assignments/mpls-lsp-ping-parameters/mpls-lsp-ping-parameters.xhtml
/assignments/mpls-multi-topology-parameters/mpls-multi-topology-parameters.xhtml
/assignments/mrcpv2-parameters/mrcpv2-parameters.xhtml
/assignments/mrt-parameters/mrt-parameters.xhtml
/assignments/mrt/mrt.xhtml
/assignments/msdp-tlv-values/msdp-tlv-values.xhtml
/assignments/msrp-parameters/msrp-parameters.xhtml
/assignments/mta-sts/mta-sts.xhtml
/assignments/mtqp-options/mtqp-options.xhtml
/assignments/mtrace/mtrace.xhtml
/assignments/mud/mud.xhtml
/assignments/multicast-acquisition/multicast-acquisition.xhtml
/assignments/multicast-addresses/multicast-addresses.xhtml
/assignments/multicast-ping/multicast-ping.xhtml
/assignments/named-information/named-information.xhtml
/assignments/netconf-capability-urns/netconf-capability-urns.xhtml
/assignments/netnews-parameters/netnews-parameters.xhtml
/assignments/nfsv4-device-id-notifications/nfsv4-device-id-notifications.xhtml
/assignments/nfsv4-named-attributes/nfsv4-named-attributes.xhtml
/assignments/nfsv4-path-variables/nfsv4-path-variables.xhtml
/assignments/nfsv4-recallable-object-types/nfsv4-recallable-object-types.xhtml
/assignments/nhrp-types/nhrp-types.xhtml
/assignments/nlpids/nlpids.xhtml
/assignments/nntp-parameters/nntp-parameters.xhtml
/assignments/norm-parameters/norm-parameters.xhtml
/assignments/notification-capability-parameters/notification-capability-parameters.xhtml
/assignments/novell-sap-numbers/novell-sap-numbers.xhtml
/assignments/nsh/nsh.xhtml
/assignments/nslp-parameters/nslp-parameters.xhtml
/assignments/ntp-parameters/ntp-parameters.xhtml
/assignments/nts/nts.xhtml
/assignments/nvo3/nvo3.xhtml
/assignments/oauth-parameters/oauth-parameters.xhtml
/assignments/olsr-types/olsr-types.xhtml
/assignments/oncrpc-ids/oncrpc-ids.xhtml
/assignments/operating-system-names/operating-system-names.xhtml
/assignments/opes/opes.xhtml
/assignments/opus-channel-mapping-families/opus-channel-mapping-families.xhtml
/assignments/os-specific-parameters/os-specific-parameters.xhtml
/assignments/osi-nsapa-numbers/osi-nsapa-numbers.xhtml
/assignments/ospf-authentication-codes/ospf-authentication-codes.xhtml
/assignments/ospf-dd-packet-flags/ospf-dd-packet-flags.xhtml
/assignments/ospf-lls-tlvs/ospf-lls-tlvs.xhtml
/assignments/ospf-mt-routing/ospf-mt-routing.xhtml
/assignments/ospf-opaque-types/ospf-opaque-types.xhtml
/assignments/ospf-parameters/ospf-parameters.xhtml
/assignments/ospf-sig-alg/ospf-sig-alg.xhtml
/assignments/ospf-traffic-eng-tlvs/ospf-traffic-eng-tlvs.xhtml
/assignments/ospfv2-parameters/ospfv2-parameters.xhtml
/assignments/ospfv3-authentication-trailer-options/ospfv3-authentication-trailer-options.xhtml
/assignments/ospfv3-parameters/ospfv3-parameters.xhtml
/assignments/otp-parameters/otp-parameters.xhtml
/assignments/owamp-parameters/owamp-parameters.xhtml
/assignments/pa-tnc-parameters/pa-tnc-parameters.xhtml
/assignments/pal-package-types/pal-package-types.xhtml
/assignments/pana-parameters/pana-parameters.xhtml
/assignments/params/params.xhtml
/assignments/passport/passport.xhtml
/assignments/paws/paws.xhtml
/assignments/pb-tnc-parameters/pb-tnc-parameters.xhtml
/assignments/pcep/pcep.xhtml
/assignments/pcim-oids/pcim-oids.xhtml
/assignments/pcp-parameters/pcp-parameters.xhtml
/assignments/performance-metrics/performance-metrics.xhtml
/assignments/perm-mcast-groupids/perm-mcast-groupids.xhtml
/assignments/pgp-parameters/pgp-parameters.xhtml
/assignments/phbid-codes/phbid-codes.xhtml
/assignments/pidf-lo-civic-address-considerations/pidf-lo-civic-address-considerations.xhtml
/assignments/pim-parameters/pim-parameters.xhtml
/assignments/pkix-parameters/pkix-parameters.xhtml
/assignments/pnfs-layout-types/pnfs-layout-types.xhtml
/assignments/pop3-extension-mechanism/pop3-extension-mechanism.xhtml
/assignments/posh-service-names/posh-service-names.xhtml
/assignments/power-state-sets/power-state-sets.xhtml
/assignments/ppp-numbers/ppp-numbers.xhtml
/assignments/pppoe-parameters/pppoe-parameters.xhtml
/assignments/ppsp-tp/ppsp-tp.xhtml
/assignments/ppspp/ppspp.xhtml
/assignments/precis-parameters/precis-parameters.xhtml
/assignments/precis-tables-6.3.0/precis-tables-6.3.0.xhtml
/assignments/preemption-namespace/preemption-namespace.xhtml
/assignments/pres-srv-labels/pres-srv-labels.xhtml
/assignments/printer-language-numbers/printer-language-numbers.xhtml
/assignments/profile-uris/profile-uris.xhtml
/assignments/pronet80-type-numbers/pronet80-type-numbers.xhtml
/assignments/prophet/prophet.xhtml
/assignments/protocol-numbers/protocol-numbers.xhtml
/assignments/provisional-standard-media-types/provisional-standard-media-types.xhtml
/assignments/proxy-languages/proxy-languages.xhtml
/assignments/psamp-parameters/psamp-parameters.xhtml
/assignments/pskc/pskc.xhtml
/assignments/pt-eap/pt-eap.xhtml
/assignments/pt-tls-parameters/pt-tls-parameters.xhtml
/assignments/public-data-network-numbers/public-data-network-numbers.xhtml
/assignments/pvds/pvds.xhtml
/assignments/pwe3-parameters/pwe3-parameters.xhtml
/assignments/qspec/qspec.xhtml
/assignments/quic/quic.xhtml
/assignments/radius-types/radius-types.xhtml
/assignments/rdap-asn/rdap-asn.xhtml
/assignments/rdap-dns/rdap-dns.xhtml
/assignments/rdap-extensions/rdap-extensions.xhtml
/assignments/rdap-ipv4/rdap-ipv4.xhtml
/assignments/rdap-ipv6/rdap-ipv6.xhtml
/assignments/rdap-json-values/rdap-json-values.xhtml
/assignments/rdap-object-tags/rdap-object-tags.xhtml
/assignments/rdap-provider-object-tags/rdap-provider-object-tags.xhtml
/assignments/rddp/rddp.xhtml
/assignments/registrar-ids/registrar-ids.xhtml
/assignments/reload/reload.xhtml
/assignments/remote-direct-data-placement/remote-direct-data-placement.xhtml
/assignments/reputation-parameters/reputation-parameters.xhtml
/assignments/restconf-capability-urns/restconf-capability-urns.xhtml
/assignments/rfb/rfb.xhtml
/assignments/rid/rid.xhtml
/assignments/rip-types/rip-types.xhtml
/assignments/rmt-fec-parameters/rmt-fec-parameters.xhtml
/assignments/rohc-pro-ids/rohc-pro-ids.xhtml
/assignments/rohc-sub-ids/rohc-sub-ids.xhtml
/assignments/rolie/rolie.xhtml
/assignments/route-distinguisher-types/route-distinguisher-types.xhtml
/assignments/rpc-authentication-numbers/rpc-authentication-numbers.xhtml
/assignments/rpc-netids/rpc-netids.xhtml
/assignments/rpc-program-numbers/rpc-program-numbers.xhtml
/assignments/rpcbind-transport-parameters/rpcbind-transport-parameters.xhtml
/assignments/rpki/rpki.xhtml
/assignments/rpl-routing-metric-constraint/rpl-routing-metric-constraint.xhtml
/assignments/rpl/rpl.xhtml
/assignments/rserpool-parameters/rserpool-parameters.xhtml
/assignments/rsip-parameters/rsip-parameters.xhtml
/assignments/rsvp-parameters/rsvp-parameters.xhtml
/assignments/rsvp-te-oam/rsvp-te-oam.xhtml
/assignments/rsvp-te-parameters/rsvp-te-parameters.xhtml
/assignments/rtcp-xr-block-types/rtcp-xr-block-types.xhtml
/assignments/rtcp-xr-sdp-parameters/rtcp-xr-sdp-parameters.xhtml
/assignments/rtfm/rtfm.xhtml
/assignments/rtp-parameters/rtp-parameters.xhtml
/assignments/rtsp-parameters/rtsp-parameters.xhtml
/assignments/rtspv2-parameters/rtspv2-parameters.xhtml
/assignments/s-naptr-parameters/s-naptr-parameters.xhtml
/assignments/s-pmsi-parameters/s-pmsi-parameters.xhtml
/assignments/safi-namespace/safi-namespace.xhtml
/assignments/sam-baseline/sam-baseline.xhtml
/assignments/sasl-mechanisms/sasl-mechanisms.xhtml
/assignments/scep/scep.xhtml
/assignments/scim/scim.xhtml
/assignments/scsp-numbers/scsp-numbers.xhtml
/assignments/sctp-parameters/sctp-parameters.xhtml
/assignments/sdp-parameters/sdp-parameters.xhtml
/assignments/sdp-security-descriptions/sdp-security-descriptions.xhtml
/assignments/sdxf-parameters/sdxf-parameters.xhtml
/assignments/sec-ext-enum/sec-ext-enum.xhtml
/assignments/secevent/secevent.xhtml
/assignments/security-label-format-selection/security-label-format-selection.xhtml
/assignments/security-txt-fields/security-txt-fields.xhtml
/assignments/segment-routing/segment-routing.xhtml
/assignments/senml/senml.xhtml
/assignments/service-codes/service-codes.xhtml
/assignments/service-function-chaining-service-function-types/service-function-chaining-service-function-types.xhtml
/assignments/service-names-port-numbers/service-names-port-numbers.xhtml
/assignments/sgmp-vendor-specific-codes/sgmp-vendor-specific-codes.xhtml
/assignments/shim6/shim6.xhtml
/assignments/sieve-environment-items/sieve-environment-items.xhtml
/assignments/sieve-extensions/sieve-extensions.xhtml
/assignments/sieve-notification/sieve-notification.xhtml
/assignments/sig-alg-numbers/sig-alg-numbers.xhtml
/assignments/sigcomp-namespace/sigcomp-namespace.xhtml
/assignments/sigtran-adapt/sigtran-adapt.xhtml
/assignments/sip-clf-parameters/sip-clf-parameters.xhtml
/assignments/sip-events/sip-events.xhtml
/assignments/sip-parameters/sip-parameters.xhtml
/assignments/sip-precond-types/sip-precond-types.xhtml
/assignments/sip-priv-values/sip-priv-values.xhtml
/assignments/sip-table/sip-table.xhtml
/assignments/slp-da-service/slp-da-service.xhtml
/assignments/smi-numbers/smi-numbers.xhtml
/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xhtml
/assignments/snmp-number-spaces/snmp-number-spaces.xhtml
/assignments/snoop-datalink-types/snoop-datalink-types.xhtml
/assignments/socks-methods/socks-methods.xhtml
/assignments/special-registry/special-registry.xhtml
/assignments/special-use-domain-names/special-use-domain-names.xhtml
/assignments/spf-parameters/spf-parameters.xhtml
/assignments/spi-numbers/spi-numbers.xhtml
/assignments/sppf-orgidtype-namespace/sppf-orgidtype-namespace.xhtml
/assignments/srtp-protection/srtp-protection.xhtml
/assignments/ssh-parameters/ssh-parameters.xhtml
/assignments/stamp-tlv-types/stamp-tlv-types.xhtml
/assignments/starttls-validation-result-types/starttls-validation-result-types.xhtml
/assignments/stringprep-profiles/stringprep-profiles.xhtml
/assignments/stun-parameters/stun-parameters.xhtml
/assignments/sun-rpc-numbers/sun-rpc-numbers.xhtml
/assignments/svrloc-cryptographic-bsd/svrloc-cryptographic-bsd.xhtml
/assignments/svrloc-error-numbers/svrloc-error-numbers.xhtml
/assignments/svrloc-extensions/svrloc-extensions.xhtml
/assignments/svrloc-function-ids/svrloc-function-ids.xhtml
/assignments/svrloc-templates/svrloc-templates.xhtml
/assignments/syslog-parameters/syslog-parameters.xhtml
/assignments/tcp-convert-protocol-parameters/tcp-convert-protocol-parameters.xhtml
/assignments/tcp-header-flags/tcp-header-flags.xhtml
/assignments/tcp-parameters/tcp-parameters.xhtml
/assignments/te-link-capabilities/te-link-capabilities.xhtml
/assignments/te-types/te-types.xhtml
/assignments/teap-parameters/teap-parameters.xhtml
/assignments/tel-uri-parameters/tel-uri-parameters.xhtml
/assignments/telnet-options/telnet-options.xhtml
/assignments/terminal-type-names/terminal-type-names.xhtml
/assignments/tesla-parameters/tesla-parameters.xhtml
/assignments/text-directory-registrations/text-directory-registrations.xhtml
/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml
/assignments/tls-parameters/tls-parameters.xhtml
/assignments/token-binding-protocol/token-binding-protocol.xhtml
/assignments/trailer-types/trailer-types.xhtml
/assignments/trans/trans.xhtml
/assignments/transfer-encodings/transfer-encodings.xhtml
/assignments/trill-parameters/trill-parameters.xhtml
/assignments/trip-parameters/trip-parameters.xhtml
/assignments/tsig-algorithm-names/tsig-algorithm-names.xhtml
/assignments/tunnel-setup-protocol/tunnel-setup-protocol.xhtml
/assignments/twamp-parameters/twamp-parameters.xhtml
/assignments/tzdist-actions/tzdist-actions.xhtml
/assignments/tzdist-identifiers/tzdist-identifiers.xhtml
/assignments/uaddr-formats/uaddr-formats.xhtml
/assignments/ule-next-headers/ule-next-headers.xhtml
/assignments/uri-schemes/uri-schemes.xhtml
/assignments/urlauth-access-ids/urlauth-access-ids.xhtml
/assignments/urlauth-authorization-mechanism-registry/urlauth-authorization-mechanism-registry.xhtml
/assignments/urn-namespaces/urn-namespaces.xhtml
/assignments/urn-serviceid-labels/urn-serviceid-labels.xhtml
/assignments/vcard-elements/vcard-elements.xhtml
/assignments/vcdiff-comp-ids/vcdiff-comp-ids.xhtml
/assignments/version-numbers/version-numbers.xhtml
/assignments/version-symbols/version-symbols.xhtml
/assignments/vnc-uri/vnc-uri.xhtml
/assignments/vot/vot.xhtml
/assignments/wave-avi-codec-registry/wave-avi-codec-registry.xhtml
/assignments/webauthn/webauthn.xhtml
/assignments/webfinger/webfinger.xhtml
/assignments/webpush-parameters/webpush-parameters.xhtml
/assignments/websocket/websocket.xhtml
/assignments/well-known-uris/well-known-uris.xhtml
/assignments/wesp-flags/wesp-flags.xhtml
/assignments/x25-type-numbers/x25-type-numbers.xhtml
/assignments/xcap-parameters/xcap-parameters.xhtml
/assignments/xml-ns-provided-by/xml-ns-provided-by.xhtml
/assignments/xml-registry/xml-registry.xhtml
/assignments/xml-security-uris/xml-security-uris.xhtml
/assignments/xmlers/xmlers.xhtml
/assignments/xmss-extended-hash-based-signatures/xmss-extended-hash-based-signatures.xhtml
/assignments/xns-protocol-types/xns-protocol-types.xhtml
/assignments/yang-module-tags/yang-module-tags.xhtml
/assignments/yang-parameters/yang-parameters.xhtml
ronaldtse commented 3 years ago

For every registry, there is an XML file describing its contents, which is where we will get the dates and details of this registry.

e.g. for sip-parameters:

$ wget https://www.iana.org/assignments/sip-parameters/sip-parameters.xml

The only metadata to obtain is this:

<?xml version='1.0' encoding='UTF-8'?>
<?xml-stylesheet type="text/xsl" href="sip-parameters.xsl"?>
<?xml-model href="sip-parameters.rng" schematypens="http://relaxng.org/ns/structure/1.0" ?>
<registry xmlns="http://www.iana.org/assignments" id="sip-parameters">
  <title>Session Initiation Protocol (SIP) Parameters</title>
  <created>2002-01</created>
  <updated>2021-01-20</updated>

The rest are "sub-registry" information. There are 80 "sub-registries" in this particular registry:

$ grep '<registry' sip-parameters.xml | wc -l
      81

e.g.

  <registry id="sip-parameters-2">
    <title>Header Fields</title>
    <expert>Adam Roach</expert>
    <xref type="rfc" data="rfc3261"/>
    <xref type="rfc" data="rfc3427"/>
    <xref type="rfc" data="rfc5727"/>
    <note>The table below lists the header fields currently defined for the
Session Initiation Protocol (SIP) <xref type="rfc" data="rfc3261"/>. Some headers have
single-letter compact forms (Section 7.3 of RFC 3261). Header field
names are case-insensitive.

Standard header fields and messages MUST NOT begin with the leading
characters "P-".  Existing "P-" header field registrations are
considered grandfathered, but new registrations of Informational
header fields should not begin with the leading characters "P-"
(unless the "P-" would preserve compatibility with an pre-existing
unregistered usage of the header field, at the discretion of the
Designated Expert).  Short forms of header fields MUST only be
assigned to standards track header fields. At the discretion of the 
Designated Expert, a header registration may require a Standards 
Action.
    </note>
ronaldtse commented 3 years ago

@rjsparks I'd like to check if it's okay to make the IANA BibXML dataset a static one -- only 581 pages, crawled once a day, seems like the best approach. Thanks!

rjsparks commented 3 years ago

I'm not particularly comfortable with that without asking IANA. The RFP as specified would only fetch a registry from IANA when someone wanted bibtex related to it - it's quite likely that some IANA registries could go days or longer without being accessed that way. We would only be caching the results that were actually requested. This plan would force us adding a daily refresh load, which I admit isn't large, but I would want to ask them (which I will do and report back).

If we were to proceed this way, Given that the cache period from the RFP is configurable, the refresh scan rate would need to be configurable, and it would be best if there were a way to signal that a particular registry should be refreshed to avoid hitting all the IANA pages again to pick up one wanted changed registry.

rjsparks commented 3 years ago

btw - when registries do change, authors tend to be very impatient with the ability to reference the results.

ronaldtse commented 3 years ago

Thanks for consulting IANA on this!

Re: authors being impatient. The current IANA BibXML only provides the name of the registry and nothing more, so the impatience should only apply when there's a new registry that an author can't wait to cite.

ronaldtse commented 3 years ago

@rjsparks I just found out that Lars Eggert is already rsynching all the IANA registries on GitHub: https://github.com/larseggert/iana-assignments

So it's easy to rsync in this case, too.

ronaldtse commented 3 years ago

@rjsparks we've implemented a mirror using the IANA rsync endpoint, with the full data available here: https://github.com/ietf-ribose/iana-registries . It's fast, and presumably won't be an extra burden to them (since others are already publicly doing so...).

I will file another issue for consideration of sync cadence: