Urgent: CloudKibo is not working

[jekram]

@sojharo

Basic cloud Kibo is not working. I think this has to do with the recent changes that have been made. So start investigating from that end. I could not type chat properly. Something in the background is looping and eating up resources.

[sojharo]

I had worked on this on that day. I could not test it today as it was off day for all and I had to do interviews. We would finally test it tomorrow then hopefully we would be able to close the error.

Also in this I improved the quality of log messages and added meaningful logs which will help more in this. For now, I have removed the callstats.io. We had tested alot with interns on friday.

[jekram]

Great job @sojharo.Thanks :-)

[sojharo]

I worked more on it. It was becoming slow after adding the callstats.io back.

I then tried it only on audio channel and it was working fine. I think it is error on our side. I created separate instances of callstats for each channel audio, video, data and screen.

I need to think more on it on how we should design the logic that we should make only instance of callstats for all.

[jekram]

Thanks for the update.

It is good think to think more about it from design perspective before we reintroduce it.

[sojharo]

I have added it using Angularjs service way. Now it doesn't create several instances for each channel and person in the call. I hope now we don't have processing issues with CallStats.io in our calls.

However, I noticed one thing, it sends data to server after each second. It keeps doing this during the entire call. This may slow down the call on slower networks. I have reduced the processing load now but I could not increase this time interval. This is set inside callstats.io library and it keeps sending data after every one second.

[sojharo]

We have removed the Callstats.io. As discussed in above comment, that it sends the information to callstats server every 1 second. This might be cause of problems we are having on call.

Similar service to this is getstats.io

[jekram]

@sojharo

Is this what you want. I have on both the servers. on TCP.

[sojharo]

Yes, we need to have these ports open. On Mar 19, 2016 6:19 AM, "Cloudkibo" notifications@github.com wrote:

@sojharo https://github.com/sojharo

Is this what you want. I have on both the servers. on TCP.

[image: image] https://cloud.githubusercontent.com/assets/5336012/13895777/ad12f23e-ed35-11e5-96b1-d0d0f4fb901e.png

[image: image] https://cloud.githubusercontent.com/assets/5336012/13895800/c245e2b0-ed35-11e5-9d61-32ed1074cc9e.png

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/Cloudkibo/CloudKibo/issues/423#issuecomment-198598375

[jekram]

They are open and let me know if you still cannot access it

[sojharo]

Yes, I can access them now.

[jekram]

I do not think the test for isChrome is correct. Are we setting the variable correctly.

var isChrome = !!navigator.webkitGetUserMedia;

  return {'iceServers': [

• {url: 'turn:45.55.232.65:3478?transport=udp', username: 'cloudkibo', credential: 'cloudkibo'},
• {url: (isChrome
• ? 'turn:turn.cloudkibo.com:3478?transport=udp'
• : 'turn:45.55.232.65:3478?transport=udp'), username: 'cloudkibo', credential: 'cloudkibo'},
• {url: (isChrome
• ? 'turn:turn.cloudkibo.com:3478?transport=tcp'
• : 'turn:45.55.232.65:3478?transport=tcp'), username: 'cloudkibo', credential: 'cloudkibo'}, +
• /*{url: 'turn:45.55.232.65:3478?transport=udp', username: 'cloudkibo', credential: 'cloudkibo'},

Please see the example here for simplicity.

[jekram]

Let's also discuss this:

https://webrtchacks.com/coturn/

https://github.com/coturn/coturn

[jekram]

Let's also implement ohter TURN services (multiple) of them today.

Including Xirsys. Here is the internal link.

https://docs.google.com/spreadsheets/d/1YbJGyGUoVSevsfDrlUZTe_KQNPSkskHP7MkyG24o3cE/edit#gid=0

[jekram]

Here is a sample of Sturn and Turn URL. We should keep things simple.

https://gist.github.com/yetithefoot/7592580

[sojharo]

I have verified from the adapter.js code too, they also use same webkit prefixed names to detect between firefox and chrome.

[sojharo]

Some parameters were missing in the XirSys TURN settings, however I have added the Viagenie now.

[sojharo]

numb.viagenie.ca did not work.

Also they have not given the port number with the url. like we give 3478

I am trying again with XirSys now. They have not given complete parameter values. they have given token which is not used in documentation. They talk of secret, domain name in documentation which is not given to us.

[sojharo]

I worked with XirSys. It did not work and did not give me credentials in response.

However, numb.viagenie.com is now working. I did tests with Sumaira. And it did not give any issues now. To confirm, I would once remove numb.viagenei.com and then test and then add it again.

[sojharo]

For final test on this, it would be easier if I test with you twice. Once behind IBA proxy and once behind my 4G device.

I tested again by removing numb turn and then adding it again.

[jekram]

Please provide what changes have been implemented and what use cases we have tested in detail.

[jekram]

Show the test cases in a matrix

[sojharo]

Here is the document of Test Case Matrix. Need one more test which need to be done between us and your computer.

CloudKibo TURN Test.docx

[jekram]

So our Turn Server is still not working. What is the plan to fix it?

[sojharo]

Yes. It only works with FireFox and not chrome.

I will try with IP address only first and then URL only to diagnose it. Instead of putting condition, i would simply test with IP address and the test with URL. It would give more understanding on this issue.

[jekram]

Sojharo said before Firefox is not interesting. We need to get Chrome working.

What is the next step here?

[sojharo]

I have discussed the next step in previous comment. I am trying to fix that.

[jekram]

I am online now. You want to test

[sojharo]

Yes, Our class is going to end in 15 minutes. I would reach home till 10 pm. I would test till 1 am tonight

[jekram]

I am on the road and I had taken an exit to test. Email me when you are ready to test.

[jekram]

How are we going? When can we test?

[jekram]

@sojharo

I am making this medium as we have a work around. You should continue to resolve but not full time.

Two things I would do in this order:

1) Our Turn server is getting validated but external tool as valid. So I would focus on the interface between our code and Turn Server.

2) We should get the Turn Server on Azure as it would give better performance. Ports are open and if you need other ports to be open then let me know.

[sojharo]

Test Case 1:

I removed all the TURN server configuration and kept only one following configuration. It is our TURN server:

{url: 'turn:45.55.232.65:3478', username: 'cloudkibo', credential: 'cloudkibo'}

We removed the URL and just kept IP address.

We tested chrome to chrome and then firefox to chrome.

I was on 4G USB wingle in IBA main campus. Sumaira was behind IBA proxy in IBA main campus.

Sumaira is helping me to prepare test matrix for all such test cases.

We are testing one by one all test cases. The above test case did not work.

[sojharo]

Tested other test cases:

1. With IP address TURN settings only
2. With URL address TURN settings only
3. Numb Turn tested
4. tested google stun
5. tested bistri turn

All these were tested on both chrome and firefox.

When I was on 4G device, I couldn't hear. When I was on IBA Internet (without proxy) I was able to hear sound from Sir.

[sumairasaeed]

Here is the are the results of testing matrix for webmeeting:

testingg.xlsx

[jekram]

All of these are showing that STUN is working. However, TURN is either not working or not called. Is this a correct summary?

Can we put debug statement before and after calling TURN to see what is returned?

[jekram]

Thanks for these testing.

Let's discuss what are the logical next steps. Please get on Slack

[jekram]

Is this summary correct from the matrix: ?

1. both behind proxy it is working ?

When both sides are on proxy then TURN server is being used a relay box. Correct?

1. both non proxy it is working ?

In the case STUN resolved the address and it is peer to peer. correct?

1. one proxy and other not proxy it is not working ?

Is the expected behaviour in mixed mode is that it would use TURN server as an relay box. Correct?

As a test let's force all traffic to use TURN server as an relay box and see what happens.

[jekram]

Changing this back to high until we have a workaround

[sojharo]

I just saw these comments. I think we discussed most on slack

[jekram]

Yes

[jekram]

@sojharo

Please provide what is the status here

[sojharo]

I have completed installing of TURN servers on given two droplets. I could not find Google's open source TURN server project. However, I installed following two TURN servers on given projects.

• https://github.com/coturn/coturn
• http://www.resiprocate.org/ReTurn_Overview

I also found one other TURN server project on sourceforge. It is not actively maintained. Last update was in 2013.

https://sourceforge.net/projects/turnserver/

I carefully read the documentations of both Coturn and reTurn before installation. I still got unreachable issues on Coturn. For reTurn, I am still working on it, it seems that it is not running.

Further, I read the following explanation on TURN and STUN by Muaz Khan.

https://www.webrtc-experiment.com/docs/STUN-or-TURN.html

Furthermore, I read one more article and came to know about following TURN servers:

http://www.creytiv.com/restund.html

This one is also available for AWS and it had setup steps for Google Compute Engine.

After this, I also looked into Docker and found that there were several docker images built for TURN server which we can pull and just run. They will run TURN server inside them. We would just need to run docker image.

https://hub.docker.com/search/?isAutomated=0&isOfficial=0&page=2&pullCount=0&q=turn&starCount=0

There are so many of them that is becoming difficult to understand which one should we opt. I think the one with so many pulls and stars will be the right option.

I am currently trying to work with the installations I have done on droplets. If they are not done, then we would go for docker containers of TURN as last option. I would have gone for them as first option but I thought to search dockers for TURN server after I tried all above things.

[jekram]

Thanks for the detailed update.

Let's discuss these tomorrow in our status meeting. In the mean time it will give me time to comprehend and come up with a plan.

Thanks again

[jekram]

@sojharo

I have thought about this, and this is what we need to do. So let's discuss this in the status meeting.

There is two kind of priority and answer. What we do in the short term and what we do in the long term.

In the Short term we should work in this order:

1. Buy a Turn Service from xrysis or somebody else. So we have a solution at hand.
2. Deploy Docker
3. Deploy our Turn Server

In the long term our priority should be:

1. Deploy our Turn Server
2. Deploy Docker
3. Buy as a service

In the short term, we should just buy the service, and it is only a cost issue. So it should be just us pointing to one or more service provider. It will allow us time deploy Docker or our own Turn Server.

In the long term, the answer is between Docker and our own Turn Server. In the long term, I prefer that we understand Turn Server so we can fine tune it and we can optimize it. TuRN Server is a critical part of our offering. It will become expensive is we use service. I do not know how much control we would have with Docker Turn.

[jekram]

What is the mid day status?

[sojharo]

Working with service of XirSys. I have managed to get TURN credentials from them now. We had to create domain name, applications and rooms in dashboard of xirsys. Now, it is just 20 minutes more work. I am trying to fix it in angularjs code.

[sojharo]

I have completely integrated XirSys now with both conference and webmeeting. I need to test it with Sumaira. She says her Internet is not working now. We would test and then report here.

[sojharo]

I just tested with Sumaira and we could not hear each other and it seems that issue is still not resolved by using XirSys TURN servers. This is the list of TURN server and credentials given to me by XirSys which I added to our peer connection object:

[sojharo]

I have finally make it work behind the proxy now. What I did was this:

1. I got TURN server credentials from the XirSys. It gave me list of 5 TURN server addresses and credentials.
2. Then I attached to this list addresses of our own TURN server (old one)
3. After this, I attached numb turn credentials to this list.

Finally, this is the list of turn servers which I give to our webrtc peer connection object:

{"iceServers":[
    {"url":"stun:turn01.uswest.xirsys.com"},
    {"username":"bcd62532-f1cd-11e5-a87b-5883419cfa3a","url":"turn:turn01.uswest.xirsys.com:443?transport=udp","credential":"bcd625be-f1cd-11e5-b663-8968345edd51"},
    {"username":"bcd62532-f1cd-11e5-a87b-5883419cfa3a","url":"turn:turn01.uswest.xirsys.com:443?transport=tcp","credential":"bcd625be-f1cd-11e5-b663-8968345edd51"},
    {"username":"bcd62532-f1cd-11e5-a87b-5883419cfa3a","url":"turn:turn01.uswest.xirsys.com:5349?transport=udp","credential":"bcd625be-f1cd-11e5-b663-8968345edd51"},
    {"username":"bcd62532-f1cd-11e5-a87b-5883419cfa3a","url":"turn:turn01.uswest.xirsys.com:5349?transport=tcp","credential":"bcd625be-f1cd-11e5-b663-8968345edd51"},
    {"url":"turn:turn.cloudkibo.com:3478?transport=tcp","username":"cloudkibo","credential":"cloudkibo"},
    {"url":"turn:turn.cloudkibo.com:3478?transport=udp","username":"cloudkibo","credential":"cloudkibo"},
    {"url":"turn:numb.viagenie.ca:3478","username":"support@cloudkibo.com","credential":"cloudkibo"}
]}

I did multiple tests on this. I did with Zarmeen, Sumaira and one other friend in other city and this is working now.

The next work in this would be, according to plan, go for docker images and then try to work with our own TURN servers.

[jekram]

Great. Thanks for the update.

What made it work? It was not working before.