thedevs-network / kutt

Free Modern URL Shortener.
https://kutt.it
MIT License
8.4k stars 1.09k forks source link

Bulk import (e.g. from Bitly) #382

Open w4rner opened 4 years ago

w4rner commented 4 years ago

If we export data from another provider (most notably Bitly), can we bulk upload url-destination pairs? Can this be handled already by the API?

trgwii commented 4 years ago

I'm not sure what file formats other URL shorteners export, but one could for sure write some code to iterate over the exported files and add the short links one by one.

trgwii commented 4 years ago

If you provide a sample of an export file we can probably write some command-line client to import everything from the file.

brownsugar commented 4 years ago

@trgwii Glad to hear this. I'm wondering which data fields are required for kutt to store a link?

trgwii commented 4 years ago

I think you misunderstand, for this tool to be worth writing it should work with whatever formats other URL shorteners export to, meaning the data shouldn't need to be converted to anything beforehand

brownsugar commented 3 years ago

I use https://api-ssl.bitly.com/v4/groups/{groupid}/bitlinks api to get all bit.ly links (ref). The data will be:

{
    "links": [
        {
            "created_at": "2020-10-26T17:20:40+0000",
            "id": "bit.ly/XXXXX",
            "link": "https://bit.ly/XXXXX",
            "custom_bitlinks": [
                "https://bit.ly/my-custom-url"
            ],
            "long_url": "https://www.youtube.com",
            "archived": false,
            "created_by": "o_xxxxx",
            "client_id": "xxxxxxxxxx",
            "tags": [],
            "deeplinks": [],
            "references": {
                "group": "https://api-ssl.bitly.com/v4/groups/{groupid}"
            }
        },
        ...
    ],
    "pagination": {
        "prev": "",
        "next": "https://api-ssl.bitly.com/v4/groups/{groupid}/bitlinks?page=2",
        "size": 50,
        "page": 1,
        "total": 160
    }
}

To get the click stats of a bitlink, use https://api-ssl.bitly.com/v4/bitlinks/{linkid}/clicks (ref):

{
    "unit_reference": "2020-12-26T11:57:06+0000",
    "link_clicks": [
        {
            "date": "2020-12-26T00:00:00+0000",
            "clicks": 10
        },
        {
            "date": "2020-12-25T00:00:00+0000",
            "clicks": 10
        },
        ...
    ],
    "units": 30,
    "unit": "day"
}
trgwii commented 3 years ago

Here's a basic script to export data from the official instance (or at least the first page of data):

const Kutt = require('kutt');

const [mode, arg] = process.argv.slice(2);

const kutt = new Kutt();

kutt.setKey('your-api-key');

(async () => {
    if (mode === 'links') {
        const links = (await kutt.list()).list.map(x => ({
            created_at: x.created_at || null,
            id: x.id,
            custom_bitlinks: [],
            long_url: x.target,
            archived: Boolean(x.banned),
            created_by: String(x.user_id || 'unknown_user'),
            client_id: String(x.user_id || 'unknown_user'),
            tags: [],
            deeplinks: [],
            references: {}
        }));
        console.log(JSON.stringify({ links }, null, 2));
    }
})();

Usage:

  1. Install nodejs
  2. Create a folder
  3. Save the above code in a file called exporter.js
  4. Fill in your API key in the file
  5. Open a terminal in the folder and run: npm init -y
  6. Run: npm i kutt
  7. Run: node exporter.js links You should see something similar to your first JSON sample.

In order to improve this, I'll need to update the node-kutt library a bit, as it seems to be getting quite outdated, I might look into it over the next few days.