investigate more modern redirector

[ksonda]

The PID service probably needs to be migrated to something else in the next 18 months as geoconnex development spins up. The PIDsvc app is based on JRE 6, and as far as I can tell is not being maintained and the database design seems inefficient. Local testing with 300,000 registered redirects reveals an internal database lookup for a 1:1 match of 100ms, with a total RTT of 500ms for the 303. Failover/replication is difficult with the postgres backend. Along with all the other issues, including the flash deprecation in the UI, I need to investigate migrating the redirector to a modern URL shortener, preferably one that is under active maintenance by a reasonably sustainable organization or active community.

Minimal Requirements:

• custom slugs (e.g. not just short auto uuids)
• 302 and 303 response codes
• querystring forwarding
• REST API/ bulk import
• logging
• <100ms RTT to 303 with 1 million registered pids
• capability to handle synchronous requests a la chyld-pilot

Wishlists:

• regex matching (if not, can handle in proxy server or with Public Cloud load balancer)
• arbitrary tagging (for things like author, organization, description, lat/lon)
• GUI
• distributed DB backend-capable
• history logging

Preliminary options:

https://yourls.org
https://shlink.io/ https://github.com/thedevs-network/kutt

[ksonda]

yourls redirects ~10x faster in a DB with 1 million 1:1 entries from PIDsvc.

However,

• upload is slower
• Regex would need a new plugin or be implemented upstream in a proxy server
• querystring functionality is limited to "pass any arbitrary query string as-is"
• no other conditions, http header settings, etc. possible

[ksonda]

@Webb-Ben did a great job implementing most of the wishlist here. Need to test against GSIP workload.

[ksonda]

This is implemented on the server now alongside PIDsvc, and in A/B testing seems to send the 303s 100-150x faster than PIDsvc. ChyLD also functions with concurrent linked data requests on the full database. Not quite as fast as the Canadians, but functional.

@dblodgett-usgs and I should discuss the metadata requirements for something on the actual PID server/database vs what can be captured in GitHub only. YOURLs is faster, but only allows one metadata field ("title"), and doesn't keep a version history (or "retired" PIDs), whereas PIDsvc has "description" and "author" and keeps a full history.

[webb-ben]

I know that yourls has plans in the future to include metadata tables to the database structure which can be seen here. A temporary workaround could be implemented with a custom metadata table and replacing the title descriptors in yourls_url with mappings to the metadata table. I don't have any ideas at the moment for how to handle deprecated mappings...

[ksonda]

I am closing this issue, and consider YOURLs to be suitable for geoconnex for this stage. It will be maintained here until a full migration has taken place. https://pids.geoconnex.us/admin is where PID registration can take place. The PIDsvc deployment will be maintained as a backup service if we come upon a use case YOURLS cannot handle. I will make a new issue regarding YOURLS API documentation and automation.