Identify URL:s uniquely

[oscarb]

When two different URL:s point to the same content, the URL:s should be considered the same.

This is so that features such as avoiding getting duplicate links and calculating statistic for links can be supported.

How can the URL be stored on the back-end in unique and identifying way?

[oscarb]

The suggested solution is to make use of:

• Follow and resolve URL
• URL normalization
• Canonical URL

[oscarb]

Canonical URL

Two ways have been found for websites to provide the canonical URL:

Canonicial <link rel="canonical" href="http://www.imdb.com/title/tt0117500/">

Open Graph <meta property="og:url" content="http://www.imdb.com/title/tt0117500/" />

Resources:

• Use canonical URLs - Search Console Help
• Canonical URL - RESTful API for obtain the Canonical URL for a given URL
• agavish/parse-canonical-url: Parse canonical url of cross device supported web sites
• canonical-url.js
• The Open Graph protocol