fishjam-dev / membrane_rtc_engine

Customizable Real-time Communication Engine/SFU library focused on WebRTC.
Apache License 2.0
137 stars 11 forks source link

Fly.io support? #171

Closed aphillipo closed 6 months ago

aphillipo commented 1 year ago

Hi Guys, it appears that ex_dtls uses a feature of Erlang that cannot be used on IPv6 networks and because internally fly.io uses IPv6 (questionable decision, I know) we cannot use the whole of the membrane framework. With everyone using Elixir with Fly.io I do wonder if this limits this excellent framework's potential. I'm interested in helping if I can but I'm really not that sure I understand what ex_dtls is doing 😂

madawei2699 commented 1 year ago

I try to use fly.io to deploy the server, but here are some stumbling blocks that make it difficult to deploy to Fly.io:

I do not know how to resolve the TURN issue, if anyone can help, thanks!

aphillipo commented 1 year ago

I think just deploy to Digital Ocean and give up on fly.io. Not all your services need to be in one place so you could deploy the main API stuff + web (Phoenix) on fly.io and the video infrastructure on digital ocean. It probably makes sense to make your video infrastructure hard separated from your main app anyway.

Better deployment instructions would be excellent. I tried CNode but it didn't work for me (did you do anything specific?). lib cluster works fine/great for me after following these instructions https://fly.io/docs/elixir/getting-started/#clustering-your-application - if you want your app to not produce errors when deploying about connecting to the previous cluster you need to deploy with a static cookie: https://fly.io/docs/app-guides/elixir-static-cookie/

madawei2699 commented 1 year ago

I deploy my app to AWS EC2 and the NIF mode works fine, you can refer the repo: https://github.com/madawei2699/free4chat

daviaws commented 1 year ago
  • IPV6: Fly.io internal is IPV6 private network, but ei_connect does not handle IPv6 which make it hard to start our backend server, because it need use CNode to start ex_dtls. Althoght we can sue NIF mode to start ex_dtls, but it seems libcluster still can not connect the node in Fly.io IPV6.

For this part of the problem I found: https://fly.io/docs/elixir/the-basics/clustering/

  • TURN: WebRTC need TURN server to relay the package behind on the NAT network, and the TURN need open lots of UDP ports, but Fly.io needs our server to listen on fly-global-services which is conflict with the TURN server address.

For this one I still do not know.

mickel8 commented 1 year ago

Take a look at https://jellyfish-dev.github.io/jellyfish-docs/deploying/fly_io