You always have to install Simple Voice Chat on your client and your server. If you don't have it installed on your client, you can still join, but you won't be able to use any of the mod's features.
When on Forge 1.12.2 and the server has Simple Voice Chat Installed, the client will be kicked by Forge's network check if the modid isn't present.
Note: this will cause the server to allow any version of the mod to join. I'm assuming this is acceptable because of Voicechat.COMPATIBILITY_VERSION.
According to the Wiki:
When on Forge 1.12.2 and the server has Simple Voice Chat Installed, the client will be kicked by Forge's network check if the modid isn't present.
Note: this will cause the server to allow any version of the mod to join. I'm assuming this is acceptable because of
Voicechat.COMPATIBILITY_VERSION
.Not sure if this is needed for 1.16.5 too?