Verifiable history of canaries

[carrotcypher]

Without a verifiable trail of at least the last canary's states, it would be difficult or impossible to tell if there had been any changes to a canary. This was originally intended to be solved by a network that shared and stored the canaries, but a more efficient solution may exist as proposed by u/xAptive in this thread, by simply hashing or including some elements from a previous iteration of the same canary. This needs some thought.

[xAptive]

To be clear, I think a network of monitors will always be necessary. A hash chain can't solve all problems. For example, someone could publish a state, then revert to a previous state a month later. The old state would check out perfectly, and without monitoring, people might not notice the reversion, which is certainly extremely suspicious.

When I first encountered this problem, my first thought was that this was a good application for a block chain. But a custom chain would have weak hashing power, and therefore low security, so that was immediately out. But I still think there is value in piggybacking on one of the other popular chains. Maybe using OP_RETURN on Bitcoin or Bitcoin Cash (which is somewhat frowned upon), or Ethereum. This would allow a user to publish a latest hash. Monitors could pull that, verify it matches the hash of canary.json, and they know they have the latest.

At this point, only one thing is missing that would necessitate a network, and that's history. If you pull canary.json, you know the current state of the system, but you don't know past states. Knowing past states is actually somewhat vital if you want to check for changes. If you only have one state, and that state includes something like { "CODES": "DURESS": 1}}, then obviously you know enough to alert. But you can't detect things like a switched PUBKEY.

History could be exchanged among monitoring stations PTP, or it could be on the server to keep it, maybe at a URL like: https:///.well-known/canary/

[carrotcypher]

@xAptive As the PoC gets closer to being functional for key generation, I've returned focus to this problem. In reading up more on Merkle trees as well as Certificate Transparency, I've come to the conclusion that while later versions will likely involve some decentralized network of constant auditing, the easiest way to accomplish it without such a network is already a part of the original concept for canarytail — multi-party signing.

By requiring a canary have another trustworthy signing party for each update, any changes or time traveling would be instantly noticed by that party (depending on the UX of the verification, it may even block signing it while providing a warning), who can then alert the world of such. This could be done by forcing the signing functionality to retain a record of anything it signed — or at least the hash(es) of it.

Obviously this moves the trust from crypto to humans, but at the end of the day the canary is all about human trust anyway (who is to say that any canary is not just lying?).

What are your thoughts?

[carrotcypher]

@xAptive The more I think on this, I think we shouldn't attempt to solve this problem with the standard itself, but rather push it off on the storage medium. If for example all canaries are backed up in Github, IPFS, TahoeLAFS, etc, then the CLI should be able to fetch that data and compare accordingly.

By requiring the standard itself to keep a history, it over complicates the standard and may introduce some legal liabilities (e.g. canaries outing themselves as being tripped and thus falling afoul of gag orders). Besides, there seems to be no incentive for a compromised entity to tell the truth of the history anyway, so until we can come up with an elegant way to do it that doesn't require a lot of overhead or complications, I think relying on the storage medium to provide history and working to integrate with that is a safer approach.

Example: instead of just checking latest_canary.json, you feed it the URLs of both previous_canary.json and latest_canary.json.

[egberts]

State tracking to ensure proper detection of revocation is a tricky thing.

[carrotcypher]

I'm currently experimenting with a method of embedding the history of the canary without it causing too much bloat. Welcome thoughts on this direction!

One way could be by using unix timestamps (for example) to sort the oldest~newest canaries in the main array:

{
"CANARY": {

    "1609359875": {

        "CLAIMS": {
            "DOMAIN": "cryptanalys.is",
            "PUBKEY": "Y4PsjneDjxckrgibojs38VDWFBTyvlVtTQR3Z9RTRw0=",
            "NEWPUBKEY": "venYyx8=Rok4VP90RWsiD3QbVTR3sTZjjgltBjTrcwFD",
            "PANICKEY": "",
            "NEWPANICKEY": "",
            "VERSION": "0.1",
            "RELEASE": "2020-09-30T19:17:40+09:00",
            "EXPIRY": "2030-09-30T19:17:40+09:00",
            "FRESHNESS": "00000000000000000008b4177c1302331aa546635d65306d35c97a8b9bb3461d",
            "CODES": "CEASE, DURESS, GAG, RAID, SEIZE, SEPPU, SUBP, TRAP, WAR, XCRED, XOPERS"
        },
        "SIGNATURES":{
            "SIGNED_DOMAIN": "Eajg00cYjeP1Tt//6moo2eMpfHnrd8HZnOqY1XcawoqgwBrF4pIQ984kxN1uLQQHaKfStoWJTM5MKhR5aujTBA==",
            "SIGNED_PUBKEY": "S=j6gRr8H0toY5EqT4kTMoKPILHdh95tqaueZ1M0KWOQFMH1mwnefQXN/oBcQjYa2uapcaArp=gj81x/oJwfnTB4",
            "SIGNED_NEWPUBKEY": "mTqMKB=OuS4wQJHatpo5Nrexrooa0pMQF/k0A2BdjERcjI/e681qW1gnH4af8L9=XnTjZauKthQfYYcwHogM1T5P,
            "SIGNED_PANICKEY": "I8MRN5YcTrnhMEaHKwkp5A0wnogqqjerM9B/T/uLpegjao14StQfP8B2t1=oYfJ6T1HZxmaQQOcd=aoH0KXWFj4u",
            "SIGNED_NEWPANICKEY": "u5Q6FZrQNcMooaEq5Mrtp0tSX8T4=9Hnc2gKWM1KBh=nLao1jja0QTx4BJPH/m/AjOfdIkeoTp8qfaRY1wewYguH",
            "SIGNED_VERSION": "HamapANgBeFR/oM2oXahS58H8P1W4qJnjfZkB=ErKK0ocLQTueqI56jujx0QcQ=41oMOMgadYTnwwpr9YHtft/1T",
            "SIGNED_RELEASE": "wJa2xfcSTPFk1BYMd5Tpgo/HB=rK48QWjQXHr8OQImct1ea5R9gjTtufu0Y/oaqHh0KqoZ6jwMLn4p=1EMNeAnoa",
            "SIGNED_EXPIRY": "=ngLZnT1eojPMF0MKaq8TaTQ=waO/1YrIRc4jNHBg0xja1or5J9fKBtqM46QQpwHfcueWYdEm25oAS/pohktXu8H",
            "SIGNED_FRESHNESS": "YoT=h1Bnw5KMWRH9muLQSF8HaYc5kJZ/QpXcEBjpfg1/uoa4w=qnxTaNTtHoPQ2daKj8gjMIOMf0Art6e1oe0r4q",
            "SIGNED_CODES": "1J4joZKaQYorhqcOQkSwK8T1qt8=mog25IXr1e6LHF0Ba0/nu/HguQMf59dxoYMAMjfPaTpHRatWpETNnc4Bewj="
        }
    },
    "1609359912": {

        "CLAIMS": {
            "DOMAIN": "cryptanalys.is",
            "PUBKEY": "Y4PsjneDjxckrgibojs38VDWFBTyvlVtTQR3Z9RTRw0=",
            "NEWPUBKEY": "venYyx8=Rok4VP90RWsiD3QbVTR3sTZjjgltBjTrcwFD",
            "PANICKEY": "",
            "NEWPANICKEY": "",
            "VERSION": "0.1",
            "RELEASE": "2020-09-15T17:19:40+09:00",
            "EXPIRY": "2030-09-30T19:17:40+09:00",
            "FRESHNESS": "00000000000000000008b4177c1302331aa546635d65306d35c97a8b9bb3461d",
            "CODES": "CEASE, DURESS, GAG, RAID, SEIZE, SEPPU, SUBP, TRAP, WAR, XCRED, XOPERS"
        },
        "SIGNATURES":{
            "SIGNED_DOMAIN": "Eajg00cYjeP1Tt//6moo2eMpfHnrd8HZnOqY1XcawoqgwBrF4pIQ984kxN1uLQQHaKfStoWJTM5MKhR5aujTBA==",
            "SIGNED_PUBKEY": "S=j6gRr8H0toY5EqT4kTMoKPILHdh95tqaueZ1M0KWOQFMH1mwnefQXN/oBcQjYa2uapcaArp=gj81x/oJwfnTB4",
            "SIGNED_NEWPUBKEY": "mTqMKB=OuS4wQJHatpo5Nrexrooa0pMQF/k0A2BdjERcjI/e681qW1gnH4af8L9=XnTjZauKthQfYYcwHogM1T5P,
            "SIGNED_PANICKEY": "I8MRN5YcTrnhMEaHKwkp5A0wnogqqjerM9B/T/uLpegjao14StQfP8B2t1=oYfJ6T1HZxmaQQOcd=aoH0KXWFj4u",
            "SIGNED_NEWPANICKEY": "u5Q6FZrQNcMooaEq5Mrtp0tSX8T4=9Hnc2gKWM1KBh=nLao1jja0QTx4BJPH/m/AjOfdIkeoTp8qfaRY1wewYguH",
            "SIGNED_VERSION": "HamapANgBeFR/oM2oXahS58H8P1W4qJnjfZkB=ErKK0ocLQTueqI56jujx0QcQ=41oMOMgadYTnwwpr9YHtft/1T",
            "SIGNED_RELEASE": "wJa2xfcSTPFk1BYMd5Tpgo/HB=rK48QWjQXHr8OQImct1ea5R9gjTtufu0Y/oaqHh0KqoZ6jwMLn4p=1EMNeAnoa",
            "SIGNED_EXPIRY": "=ngLZnT1eojPMF0MKaq8TaTQ=waO/1YrIRc4jNHBg0xja1or5J9fKBtqM46QQpwHfcueWYdEm25oAS/pohktXu8H",
            "SIGNED_FRESHNESS": "YoT=h1Bnw5KMWRH9muLQSF8HaYc5kJZ/QpXcEBjpfg1/uoa4w=qnxTaNTtHoPQ2daKj8gjMIOMf0Art6e1oe0r4q",
            "SIGNED_CODES": "1J4joZKaQYorhqcOQkSwK8T1qt8=mog25IXr1e6LHF0Ba0/nu/HguQMf59dxoYMAMjfPaTpHRatWpETNnc4Bewj="
        }
    }
}
}

[carrotcypher]

And if that gets too bloated, potentially could gzip all entries that aren't the newest:

{
"CANARY": {

    "1609359875": {

        "CLAIMS": {
            "DOMAIN": "cryptanalys.is",
            "PUBKEY": "Y4PsjneDjxckrgibojs38VDWFBTyvlVtTQR3Z9RTRw0=",
            "NEWPUBKEY": "venYyx8=Rok4VP90RWsiD3QbVTR3sTZjjgltBjTrcwFD",
            "PANICKEY": "",
            "NEWPANICKEY": "",
            "VERSION": "0.1",
            "RELEASE": "2020-09-30T19:17:40+09:00",
            "EXPIRY": "2030-09-30T19:17:40+09:00",
            "FRESHNESS": "00000000000000000008b4177c1302331aa546635d65306d35c97a8b9bb3461d",
            "CODES": "CEASE, DURESS, GAG, RAID, SEIZE, SEPPU, SUBP, TRAP, WAR, XCRED, XOPERS"
        },
        "SIGNATURES":{
            "SIGNED_DOMAIN": "Eajg00cYjeP1Tt//6moo2eMpfHnrd8HZnOqY1XcawoqgwBrF4pIQ984kxN1uLQQHaKfStoWJTM5MKhR5aujTBA==",
            "SIGNED_PUBKEY": "S=j6gRr8H0toY5EqT4kTMoKPILHdh95tqaueZ1M0KWOQFMH1mwnefQXN/oBcQjYa2uapcaArp=gj81x/oJwfnTB4",
            "SIGNED_NEWPUBKEY": "mTqMKB=OuS4wQJHatpo5Nrexrooa0pMQF/k0A2BdjERcjI/e681qW1gnH4af8L9=XnTjZauKthQfYYcwHogM1T5P,
            "SIGNED_PANICKEY": "I8MRN5YcTrnhMEaHKwkp5A0wnogqqjerM9B/T/uLpegjao14StQfP8B2t1=oYfJ6T1HZxmaQQOcd=aoH0KXWFj4u",
            "SIGNED_NEWPANICKEY": "u5Q6FZrQNcMooaEq5Mrtp0tSX8T4=9Hnc2gKWM1KBh=nLao1jja0QTx4BJPH/m/AjOfdIkeoTp8qfaRY1wewYguH",
            "SIGNED_VERSION": "HamapANgBeFR/oM2oXahS58H8P1W4qJnjfZkB=ErKK0ocLQTueqI56jujx0QcQ=41oMOMgadYTnwwpr9YHtft/1T",
            "SIGNED_RELEASE": "wJa2xfcSTPFk1BYMd5Tpgo/HB=rK48QWjQXHr8OQImct1ea5R9gjTtufu0Y/oaqHh0KqoZ6jwMLn4p=1EMNeAnoa",
            "SIGNED_EXPIRY": "=ngLZnT1eojPMF0MKaq8TaTQ=waO/1YrIRc4jNHBg0xja1or5J9fKBtqM46QQpwHfcueWYdEm25oAS/pohktXu8H",
            "SIGNED_FRESHNESS": "YoT=h1Bnw5KMWRH9muLQSF8HaYc5kJZ/QpXcEBjpfg1/uoa4w=qnxTaNTtHoPQ2daKj8gjMIOMf0Art6e1oe0r4q",
            "SIGNED_CODES": "1J4joZKaQYorhqcOQkSwK8T1qt8=mog25IXr1e6LHF0Ba0/nu/HguQMf59dxoYMAMjfPaTpHRatWpETNnc4Bewj="
        }
    },
    "1609359912": {

        "GZIP": "eNqNVE1zozgQvW/V/geXr+sZJCwIpIoDxHiEsQCBEoxvCl8eeYyAwODU1Pz3dZKtmtnNjnd1UKv6qV+rVd3v2++/zS5rfmcHdpzNb2ff3hxvzq3tkeTvzldgFRLbCy7APO+f24E3/Mvz08fPT/PFPy5G947vvtDOMxQ9iaZciXN+7OvPj1I8LY2HVbp22PPXLw8Do/Fyb8YsnoD1jiZw0x9MX8smez4bViyP6CEyQZw+fV4t6eMDi5dPbC9E/WVwBOvzab16/yA78O7+IvrXNNfwBzdOvPC1bvARvoNjd+vaifsCq0AFH4D5YQkYNG/hzS0CfwDzFoB3Qe4u8l4//hKz/J8x69hNcOAmyetL3i/jEcGbmxwugbpcQs41pOtLrdC1JdCLpZabN9x4NB8fl0iHxTv2u3DlvjLfvVSzmK3uL+mSxeyT/Wkxi21vtZglrrd3X0wU3V/MvRMtZiy2L3tqx4vZ7i52L7d2YXT5sPkP+u8/pZon3qfAZi/c89tf95fLRQ1AnokygmxQFP0kpVqStsJNXxh434RdBnc5n2RXT06/Rq1HTQMdzwEct5Ri7lfJINMNIxrxD7HGR8Ec27KudGpiCb2OewODQWaa2zF0ZET6kbfFxcHUho6P5R4S4KchXRMMT1NTVnQXKNLJqci4OvI253bfWrUw4FmRm6lqmIOud/WJdcR3rHBM0EQ3mA+t1IK+PPdSctASulaOwFadQrhxLjyl1A3YpbBuMOKVsTWtXcPEno/+cKBVluUTljWBTIuuDYBnkDjQspz1zYG4HPvTsdVsMDWy7jpR9sR0FKaM27asBZcQJQOtIsNRB2jJrNroDOL9+cQpDfPC4hIDf5euBRr/a7BGjerrfU+DnFyqczuN9EMLhmRnMGSZuMnV2k8J9J2D1WwviYXggLIzcjYRVk6KLcKq8I6lZK3RVTzO4FROWT3iawOL+Ym3dlA75TpWJFHljh8SzcBGBFPUbRpR7Y+O5fa+D2S+pWwsO0/TxSjOgObUQlCSkNS8yFgzTW1vZnioBgWyayowbbh6rvKEResjdDJSaKytpYIdq/eRQVNBd7g3Quqd8gGWXIvNWrBhrEaQKZJ3+AD8Tu51MZFtg1oLuiQo7UbyKyJiNfV23zBYShGRNSA+7wzGGbUmHiow6704RyLATg3OgkPZaxuz8p2hI0intJ1wlY9lmhXuSdWknSitPByH3Wjg6xqUSWYdoNNMmk/SGJuny+wlawPzLNeOm71C213uOqKtaqiMkqPJ6poz4wEbsIyoWnBfGLUgXkgqYPeDXkJZgh51v9YmuEFC7n1OM9kfujykx2TyDQa7wbBOslY1b9fDUt/iNXA4UJpRwfVISaWZxVlmxCaiijhrccyHtHVZ0OTIKSdh/axWb8eL+f4ntXHo8w=="
    }
}
}

then the client software could unzip and process the json appropriately.

[egberts]

Just read a snippet over at HN,

... US law makes a distinction between 'prior restraint' and 'compelled speech'.

• Prior restraint is the government forcing you to not say something.

• Compelled speech is the opposite, forcing you to say something.

• Both have a high bar to meet, and meeting one does not mean you that you meet the other. In the case of compelled speech, I don't believe a US court has ever ordered someone to lie, which is what would be required here.

In short, the data field "DURESS" will violate the spirit of United States NSL. The only mechanism that has seems to be working is outright removal of the canary.txt file.

Reference: https://news.ycombinator.com/item?id=34193285

[carrotcypher]

Just read a snippet over at HN,

... US law makes a distinction between 'prior restraint' and 'compelled speech'.

• Prior restraint is the government forcing you to not say something.

• Compelled speech is the opposite, forcing you to say something.

• Both have a high bar to meet, and meeting one does not mean you that you meet the other. In the case of compelled speech, I don't believe a US court has ever ordered someone to lie, which is what would be required here.

In short, the data field "DURESS" will violate the spirit of United States NSL. The only mechanism that has seems to be working is outright removal of the canary.txt file.

Reference: https://news.ycombinator.com/item?id=34193285

It's a good discussion and I'll reiterate what I've said before: some lawyers find the concept of a warrant canary "dubious at best".

That said, the design goal is to design one that provides as much protection as possible.

If you make no positive claims, start with negative claims, and remove them as necessary, it should it theory meet the requirements. If a court argues that simply republishing a file without a tag is making a statement and that deleting the file altogether was the better path, I wonder if making each claim as an individual file resolves that. It'd be the same thing but abstracted out.

[egberts]

In short, not all fields would be used within JSON, depending on the country that is based out of, right?