only hardcode one url in updater

[illwieckz]

Currently, those url are hardcoded in updater:

• http://www.unvanquished.net/?json=get_recent_posts
• http://dl.unvanquished.net/versions.json (it advertises unvanquished and updater version numbers)
• http://github.com/Unvanquished/updater2/releases/download (it's a prefix to compute final updater url using updater version number from versions.json)

Once the updater is built and ship, we loose all control on those url, and we are vulnerable to any change or redirect (some parts in updater does not support redirects for sure), for example github may redirect the HTTP url to HTTPS and may break the updater update.

The idea would be to only hardcode one url in updater (basically a json file like version.json one) and put the other ones in that json file.

Note that our web server already knows the full path to the latest updater, so we may store the full updater url in that json file instead of a prefix url. That would also allow us to quit github, change the zip name, or do strong things like that.

I suggest to extend the versions.json file (perhaps under another name).

Edit: Here is the new ones: http://cdn.unvanquished.net/current.torrent

[illwieckz]

So I suggest a release.json file with a content like that:

{
    "posts": "https://unvanquished.net/api/get_recent_posts/",
    "game": {
        "version": "0.51.1",
        "torrent": "https://cdn.unvanquished.net/unvanquished_0.51.1.torrent"
    },
    "updater": {
        "version": "0.0.5",
        "zip": {
            "linux-amd64": "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterLinux.zip",
            "macos-amd64": "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterOSX.zip",
            "windows-amd64": "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterWin.zip"
        }
    }
}

Note the url are unique ones, we can because we would be able to properly produce the json file server side, the same way we already have to do it for the versions.json file.

[illwieckz]

So I wrote a tool named announce.sh which is expected to live alongside existing get.sh and publish.sh.

This announce.sh tool reads a config file and produces this:

• https://cdn.unvanquished.net/current.json (the proposed json file for next updater)
• http://dl.unvanquished.net/versions.json (the current json file read by current updater)
• http://dl.unvanquished.net/current.txt (the version file read by legacy updater)
• an nginx snippet to configure the redirects for https://unvanquished.net/download/
• the current.torrent symlink in place of publish.sh (so publish.sh only publishes things but does not announce them)

[slipher]

Sounds good. Where can we see this publish.sh and friends?

[illwieckz]

it's currently stored in server, we may have to make a git repo for those

[DolceTriade]

We should. It should be easy.

[illwieckz]

This is the current proposal (see https://cdn.unvanquished.net/current.json):

{
    "newsfeed": {
        "urls": [ "https://unvanquished.net/api/get_recent_posts/" ],
        "format": "wordpress-json-recent-posts"
    },
    "game": {
        "name": "Unvanquished",
        "version": "0.51.1",
        "parcels": {
            "universal": {
                "urls": [ "https://cdn.unvanquished.net/unvanquished_0.51.1.torrent" ],
                "content": "mixed-files",
                "container": "torrent"
            }
        }
    },
    "updater": {
        "version": "0.0.5",
        "parcels": {
            "linux-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterLinux.zip" ],
                "content": "single-binary",
                "container": "zip"
            },
            "windows-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterWin.zip" ],
                "content": "single-binary",
                "container": "zip"
            },
            "macos-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterOSX.zip" ],
                "content": "single-macos-appdir",
                "container": "zip"
            }
        }
    }
}

Note that format, content and container are unused, those are just demos for keys we may create, as a showcase for the ability to extend the proposed format.

Thinking about them helped me to design a format that would be future-proof and enable us to change many things in future updaters without breaking the format itself. For example we may also think about including checksum keys for downloaded files, especially when we would add support for mirrors.

Also the urls field is now thought to be a list: that would allow us to make the updater more reliable in the future by being able to download torrents or other things from our CDN.

[illwieckz]

For a start, updater would only have to read that:

{
    "newsfeed": {
        "urls": [ "https://unvanquished.net/api/get_recent_posts/" ]
    },
    "game": {
        "version": "0.51.1",
        "parcels": {
            "universal": {
                "urls": [ "https://cdn.unvanquished.net/unvanquished_0.51.1.torrent" ]
            }
        }
    },
    "updater": {
        "version": "0.0.5",
        "parcels": {
            "linux-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterLinux.zip" ]
            },
            "windows-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterWin.zip" ]
            },
            "macos-amd64": {
                "urls": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5/UnvUpdaterOSX.zip" ]
            }
        }
    }
}

[illwieckz]

After having implemented it in #78, this is the current format:

{
    "news": {
        "version": "rolling",
        "mirrors": [ "https://unvanquished.net" ],
        "parcels": {
            "all-all": {
                "path": "api/get_recent_posts",
                "content": "raw",
                "container": "json"
            }
        }
    },
    "game": {
        "version": "0.51.1",
        "mirrors": [ "https://cdn.unvanquished.net" ],
        "parcels": {
            "all-all": {
                "path": "unvanquished_0.51.1.torrent",
                "content": "mixed-files",
                "container": "torrent"
            }
        }
    },
    "updater": {
        "version": "0.0.5",
        "mirrors": [ "https://github.com/Unvanquished/updater/releases/download/v0.0.5" ],
        "parcels": {
            "linux-amd64": {
                "path": "UnvUpdaterLinux.zip",
                "content": "single-file",
                "container": "zip"
            },
            "windows-i686": {
                "path": "UnvUpdaterWin.zip",
                "content": "single-file",
                "container": "zip"
            },
            "macos-amd64": {
                "path": "UnvUpdaterOSX.zip",
                "content": "single-directory",
                "container": "zip"
            }
        }
    }
}

The format is meant to use a same reusable function for both news, updater and the game, it would even works if we added an option to download a level editor or things like that as an addition.

Keys like content or container are just examples done to help me design the format to be future-proof.

Support of mirrors is not implemented but nothing in the format prevents to do it one day.