ok I think I've got something feasible, this is very rough probably doesn't work needs much testing
private chatrooms have slashes in their name
names are of the format org/team where org is a github organization and team is a team within that organization
upon joining the chat you download the list of users in the org, generate a secret, encrypt the secret with all of the users public keys, and post the message
subsequent messages are encrypted with the key and instead of linking to the head you link to the message where the keys are stored (and the head too? to limit number of heads) (they are signed the same as normal)
on receiving a message you just look up the message it linked to to find the key to decrypt it, also you verify it against the public key before decrypting it
Things this solves:
confidential (it encrypts the chat)
Authentication (randos can't spam the channel with gibberish and participants can't pretend to be each other).
user management (though the org team)
end to end encryption (nobody else, not even github can decrypt the messages, github can manage the users, i.e. they could remove or add somebody)
Things this doesn't solve
Deniability: if somebody is able to decrypt the message they have mathematical proof you sent it
forward security: no session keys or anything you're key is leaked you leak everything
other issues
requires api access to github
wouldn't work with (ec)dsa keys from github
need direct access to the private key which you currently don't have (easily)
other protocols
otr: this is set up for a one on one chat it is also interactive meaning you wouldn't be able to set it up with a user who is never online at the same time as before
mpOTR: this is still being developed from what I can figure out, I believe it has an interactive setup phase which all parties must be available to use
telehash: this seems to just be communication security aka making sure my connection with my peer is encrypted, this is already handled by webrtc via DTLS
closing this as we don't have an easy way to get direct access to the keys without putting in a password, same with direct access to the github api, ow well
ok I think I've got something feasible, this is very rough probably doesn't work needs much testing
Things this solves:
Things this doesn't solve
other issues
other protocols