Critical: API integration with CloudKibo

[jekram]

@sojharo

There is an opportunity to do an API integration with CloudKibo for that we have make the following changes.

1. Currently CloudKibo is muti-user but not multi-company. We do not need to modify our front-end at this stage only the back-end. They are not looking to use the front end and want to access our system via API only. Like KiboSupport all the back-end data should be segregated by each customer. So in the MangoDB we need Customer name, Client ID and Client Secret.
2. We need to maintain old API with new API.
3. Customer will request meeting URL and they would also pass their internal id. We need to pass them two URL (one for agent and one for customer) or should it be one URL? How do we do it today
4. When the call is complete we need to inform them via webhook
5. Then through API they would request meeting data
6. They need screen sharing
7. Agent will be on Chrome, and user would be on IE, FireFox or Chrome
8. What will happen if user does not have mic,speaker and camera (would screen sharing still work)?
9. Server side recording.

I would like to see a design document on what need to be changed.to make it happen. In concept this is very similar to KiboSupport where they control the widget and the call center dashboard.

We have most of the pieces, we need to lay out in in sequence (what is available now, easy to do and then where we need more development)

I am hoping we can start the trial and then complete the rest.

We need to do what ShightCall is doing for Browser.

We need to go back to them with a document and v1 working by next week.

More on this in the meeting.

[sojharo]

Here is the design document for this.

CloudKibo Company Integration Design.docx

[jekram]

@sojharo

Thanks. I have read it once, and I would reread later tonight and send you my comments.

Overall it looks good.

[jekram]

@sojharo

Couple of things:

Your comment:

"We would not change current conference code, and we would be able to do our own conference on the same URL. For new logic, we would create separate URL e.g. https://api.cloudkibo.com/#/webmeeting/meeting_name Later, we would add plugins into this new logic."

I am fine with cloning the code and having a new URL. The question is how long we would run this parallel?

I also missed one requirement that is co-browsing.

Go ahead and start executing.

[sojharo]

Maybe one week. As soon as we know that our new conference logic (with all plugins and extensions) is stable, we would replace the old one.

On Sat, Jan 16, 2016 at 11:05 AM, Cloudkibo notifications@github.com wrote:

@sojharo https://github.com/sojharo

Couple of things:

Your comment:

"We would not change current conference code, and we would be able to do our own conference on the same URL. For new logic, we would create separate URL e.g. https://api.cloudkibo.com/#/webmeeting/meeting_name Later, we would add plugins into this new logic."

I am fine with cloning the code and having a new URL. The question is how long we would run this parallel?

I also missed one requirement that is co-browsing.

Go ahead and start executing.

— Reply to this email directly or view it on GitHub https://github.com/Cloudkibo/CloudKibo/issues/363#issuecomment-172163348 .

Regards,

Sojharo

[jekram]

Great. Move full speed ahead

[sojharo]

I have worked on it and separated the audio video streams into now separate webrtc connections. It needs more work and is under construction.

[jekram]

Thanks for the update

[sojharo]

I have done my work on new logic of conference. Overall, following is the work summary on this task:

Done:

1. Separate Audio Video connection
2. Separate screen sharing connection
3. Separate data channel connection

Pending Tasks:

1. Expose Rest API to get data of meeting (also put in rest api docs) (priority 3)
2. Webhooks for each company (priority 4)
3. Store conference chat messages in database separated by company (priority 2)
4. Conference recordings and fetching of records (priority 5)
5. Supporting IE, Safari, using plugins and adapter shim (priority 1)

Priority 1 is highest priority.

[jekram]

@sojharo

In regards to Pending Tasks:

Expose Rest API to get data of meeting (also put in rest api docs) (priority 2) Webhooks for each company (priority 3) Store conference chat messages in database separated by company (priority 5) Conference recordings and fetching of records (priority 6) Supporting IE, Safari, using plugins and adapter shim (priority 1) Co-Browsing (priority 4)

[sojharo]

I have worked on this task and used temasys plugin and also used the adapter.js latest version. I have pushed the code for this. Also, I worked to modularize the code as it had become complex to manage the all separate webrtc connections in same file.

[sojharo]

Worked more on modularization of the new conference code. Tested the plugins and they didn't work for IE. I need to look more into it, it maybe issue of IE version. I have started to work on exposing rest api. Inshallah by tomorrow evening, Zarmeen would be able to use rest api to extract company meeting data from Cloudkibo.

[jekram]

Thanks for the update. What plug-ins SighCall is using - can you tell? Also what plhg-ins Mauoz Kahn is recommending?

[sojharo]

I just looked again into SightCall and again I couldn't find where they have stored there plugins on github. Maybe they are not open source. As far as Muaz Khan is concerned, he has not developed his own plugins. But he has written some code on how to use existing webrtc plugins like temasys and webrtc everywhere.

https://github.com/muaz-khan/PluginRTC

The free version of TemaSys doesn't support screen sharing, I just found out. However, there is other plugin which is open source. Going to evaluate the other one now. I am going to finish most of the work in this by Tuesday InshaAllah so that it is almost ready for customer.

[jekram]

Here is the SightCall extension.

https://chrome.google.com/webstore/detail/screen-sharing/obcglfamidbohlfjlcfdajopagnhkgoo?hl=en

We should try to use this extension while we figure out our own extension.

Also can we revers engineer their extension?

[sojharo]

One quick question for this:

Should we assume that for each support session, Microsoft would give us following information:

1. Agent name
2. Agent email 3.Visitor name
3. Visitor email
4. Session id (we generate this randomly on kibosupport for support session, should we give it to Microsoft or they will give us their own id, i.e. support ticket)

I have reverse engineered the extension of SightCall. However, in order to work with extensions we need code of both application logic and extension source code. I will look into the code of extension soon.

Here is the sight call extension source code:

Screen-Sharing_v1.5.zip

[jekram]

@sojharo

Let me explain how their workflow works today at a high level and then we can how we would propose to change it. The final design with them may be different once we engage with them.

Customer

1. They have a widget like us where the customer comes and open an issue.
2. The customer is logged into a portal before the widget is visible. It is more for corporate clients vs. consumers. So when the customer is logged in they already know the company name and the user name.
3. The customer describes the issue on the widget, and at that time they send an automated status on the widget that we have received your request, and we will get back to you.
4. They do not (at least) the demo I saw have live agents getting into chat session right away.
5. They use the widget to send a workflow into their ticketing system.
6. In their case, they use the widget to provide status on the ticket. So think of it that their widget is a persistent chat. So if the user logo off the portal and come back again he will where he left off last time.
7. So they use this widget to provide the status of the ticket. So in the widget first it will say "Thanks we have received your issue, and we will get back to you with status.". Once an engineering is assigned the widget status would get updated ith a message like "Agent has been appointed, and we are working on it"
8. Behind the scene, they have open an issue in their ticketing system. The supervisor would take the ticket is assign it an agent.
9. The agent not only looks at this issue but all the previous issues or any other open issue with this customer.
10. If it is a simple issue that he has been able to resolve then he would close the issue, and the widget will be updated that your issue has been fixed.
11. If the issue is complex. Then the agent the customer on the phone and try to understand better it. In most of the cases, they need to do screen sharing with the customer.
12. They will update the widget with a link to logmein tool. logmein is a remote desktop app (it has screen sharing and remote control)
13. The customer clicks on the link provided in the widget and it will start a download of logmein.
14. Once the download is complete it start the logmein application.
15. To start the screen sharing the agent give the room number he is in.
16. Customer enters the room number and now both agent and customer are doing screen sharing. At this stage, the agent is looking at customer screen and guiding the customer.
17. Even though the agent can take control of the customer machine at this stage. They do not do that. The agent and customer stay on the phone call and are using screen sharing to resolve the problem. Logmein has co-browsing so the agent can draw on customer screen.
18. I do not think agent ever shows his screen. They always work of customer screen.
19. Once a problem is resolved. Both parties logoff the logmein session. In the case of the customer the LogMeIn software is uninstalled by it self.
20. At this stage, the widget is updated that we have successfully closed your issue.
21. Then the widget is updated with a short survey hey they ask questions.

Couple of points:

1. Their use of widget is quite different than most of the widget we see hat are more interactive.
2. Their widget is a communication tool between the customer and their automated system vs. customer and agent.
3. Their whole system is very complicated, and we cannot replace it.
4. What I proposed to them we can replace their LogMeIn with cloudkibo.
5. LogMeIn is not integrated into their system. Also, LogMeIn is manual; the agent gives the room key.
6. There is no API integration with LogMeIn they send the same download instruction to every customer on the widget, and the room number is given on the phone.
7. LogMeIn works with all browser. :(.
8. In our case, we would not have any downloads.
9. In our case, the agent can show his screen
10. So the work is the following:

a) They will call our API to schedule a meeting b) We return them two URLs c) The take te customer URL and post it on the widget d) They get into the meeting e) When the meeting ends they can call our API to retrieve data

I want to keep it very simple, and once we have that we would show it to them, and they can tell us what else they need.

For V1 let's keep it very simple.

Then we would modify to meet their needs

1. They do not (at least) the demo I saw they have agents who would get on these chat sessions right away.

.

[jekram]

In regards to your particular question

Should we assume that for each support session, Microsoft would give us following information:

Agent name Agent email 3.Visitor name Visitor email Session id (we generate this randomly on kibosupport for support session, should we give it to Microsoft or they will give us their own id, i.e. support ticket)

At this stage, I would not screw up our system for Microsoft. If we think these are important fields for other customers also then we should add it. In v2 probably

We should definitely give the session id, so that come back and request meeting data.

[jekram]

@sojharo

Can you summarize what is done and what is pending?

[sojharo]

I worked yesterday on WebRTC everywhere plugin and tried to build the webrtc and plugin from the source. The build process got stuck several times. Maybe, because of I ran on virtual machine. This should be built on windows and mac os.

Here is the summary:

Done:

Separate Audio Video connection Separate screen sharing connection Separate data channel connection Store conference chat messages in database separated by company ()

Pending Tasks:

Expose Rest API to get data of meeting (also put in rest api docs) (under construction) Webhooks for each company (priority 4) Conference recordings and fetching of records (priority 5) Supporting IE, Safari, using plugins and adapter shim (priority 1)

[jekram]

@sojharo

What is the ETA to complete everything excluding recording?

What is the next step building the code? Can you work in lab to get it resolved.

[jekram]

@sojharo

Have you looked at this?

https://www.npmjs.com/package/rtcninja

WebRTC API wrapper to deal with different browsers

[sojharo]

We would use new logic from tomorrow's meeting. By tomorrow evening, I would be done with plugins issue. The plugins are blocking issue in this. Hopefully, rest of the things would be ready by tomorrow meeting.

I have seen this library. New Adapter.js would be better as it is supporting MS Edge too.

[sojharo]

Following is the update on this task:

Done:

Separate Audio Video connection Separate screen sharing connection Separate data channel connection Store conference chat messages in database separated by company Expose Rest API to get data of meeting (also put in rest api docs) (under construction) Webhooks for each company

Pending Tasks:

Supporting IE, Safari, using plugins and adapter shim (it has bugs, priority 1) Conference recordings and fetching of records (priority 5)

The task, except recording, is almost done except few bugs that I am trying to resolve.

I would need to discuss on webhooks in meeting today. In our document, we had discussed that by end of conference we would use company's webhook to send conference data. We also have rest api for company to fetch conference data. It is little ambiguous to know the need to webhook in this case. However, webhook can be just needed to inform the client server that conference has ended and then client would use rest api to fetch conference data.

[jekram]

Good job. Let's wrap up the plug-in and open a separate task for recording.

We should run the old and new logic in parallel for some time.

The intent of Webhook is to inform them that the meeting has ended. Else the sender has no clue when the meeting has ended and would keep polling us.

Also, for recurring meeting like jekram there will be multiple start and multiple end times. How do we handle it?

[jekram]

@sojharo

Please update with status

[sojharo]

Working on plugins now. It would be done today.

[jekram]

Thanks :-)

[jekram]

Q: Is Audio & Video in the new logic on the same channel or separate channel?

[sojharo]

They are on the separate channel so that Edge can be supported. Edge only incorporates audio.

On Sat, Jan 30, 2016 at 3:42 AM, Cloudkibo notifications@github.com wrote:

Q: Is Audio & Video in the new logic on the same channel or separate channel?

— Reply to this email directly or view it on GitHub https://github.com/Cloudkibo/CloudKibo/issues/363#issuecomment-177003150 .

Regards,

Sojharo

[sojharo]

I have been working on chrome and firefox webrtc interoperability and resolved couple of issues like:

1. removeStream was not implemented in firefox (did workaround in code for this)
2. Mandatory fields in offer were removed from firefox (checked the browser and implemented accordingly)

One issue remains in this : Firefox screen sharing extension is corrupt and i have opened issue for this #368

Moreover, for plugins, I got several errors with open source plugin and could not resolve them as they did not use the updated version of adapter.js. The adapter.js given by them was not compatible with the Google's adapter.js version. The adapter.js of temasys is completely compatible with Google's standard adapter.js and they maintain this compatibility.

For now, I removed the opensource plugin and re-added the temasys plugin. On starting the cloudkibo conference on Internet Explorer, it asked me to install the temasys plugin which I installed. Gives better user experience.

However the application later got crashed as Internet Explorer doesn't support HTML5 Web Audio API. We use this to process audio streams and know who is currently speaking. Kindly, let me know should we disable this feature for temporary basis until we figure out how to hide this api code for IE.

This task is now complete except the following things:

1. CloudKibo REST API documentation for company data fetch (I would email after meeting)
2. Fix adapter.js of open source plugin or continue using temasys plugin
3. Recording

[jekram]

@sojharo

1. I do not understand the implication if we disable this feature?

"However the application later got crashed as Internet Explorer doesn't support HTML5 Web Audio API. We use this to process audio streams and know who is currently speaking. Kindly, let me know should we disable this feature for temporary basis until we figure out how to hide this api code for IE."

1. Are you using Temasys plugins with their adapterjs. I thought we had to buy Temasys to get full functionality. Are you commercial version or the basic version?
2. What is your estimate to fix the open source plugins with current adapter.js?
3. Have we reported the issues to the open source plugins?

Thanks

Jawaid

[sojharo]

I have done the following work on this task today:

1. Exposed the URLs in REST API to generate conference URLs for client companies. It gives two rest end points: one to get meeting url for agent and one for visitor
2. I worked on IE issue on this and disabled the "detect speaker" feature on IE only. After this, when I started conference on IE, it gave me error on where I attach WebRTC stream to Video Element. We make the URL object for stream first and then give it to video element. IE doesn't support URL object and gives following error:

I am opening separate issues for :

1. Recording of conference #372
2. Open source plugin WebRTC everywhere should be fixed #373
3. Fix IE and FireFox issues of things (e.g. web audio) not supported #374

We should close this issue now. As it has become very long. I have opened the above sub-tasks into separate tasks now which were blocking this task.

[jekram]

Thanks