Support Apple Safari - Githubissues

mccob commented 7 years ago

Apple Safari/Webkit support is here since Safari 11.

Apple use Google/WebRTC libwebrtc.

See : https://webkit.org/blog/7726/announcing-webrtc-and-media-capture/ https://webkit.org/blog/7627/safari-technology-preview-32/

Available at least on Mac OS.

Safari tech preview is available for actual Mac OS Sierra.

Licode isn't working.

Seems a specific stack is needed.

First error solved, has to use window.RTCPeerConnection

WebkitRTCPeerConnection = window.RTCPeerConnection,

Other errors that must be solved :

[Error] Unhandled Promise Rejection: TypeError: Type error createObjectURL
[Error] TypeError: Error creating RTCPeerConnection initializeRTCPeerConnection
[Error] TypeError: undefined is not an object (evaluating 'stream.pc.processSignalingMessage') (anonymous function)

if you want to help :)

kekkokk commented 7 years ago

I'll probably look into it next week! have a nice day!

mkhahani commented 7 years ago

Finally!! It's a good time for migrating to adapter.js and get rid of all these browser stacks.

kekkokk commented 7 years ago

We might discuss that, is not that straight forward. we have always to differentiate by chrome and firefox for little things (ex. opus nack, the "a=" in front of ices, the filtering out of the end candidate in chrome etc.) so maybe we'll always have the various stacks. but with a refactor of the sdp parser in C++ we might have good results with adapter.

jcague commented 7 years ago

I've been trying this week. I was able to create a videoconference with publishers sending video/audio from Chrome, Firefox and Safari, after solving all the errors mentioned by @mccob. Things to consider when we implement it:

Safari only supports h.264.
h.264 support in Licode is not perfect, we might need to add better support for fmtp lines in the SDP. I found some issue when sending H.264 from Firefox because of that, but I solved it with a quick and dirty hack.
I was able to send video from Safari, and I received and rendered it in Chrome and Firefox.
I didn't receive any data in Safari from publishers in Chrome and Firefox. The API is slightly different and I don't know if things like OfferToReceiveVideo or Transceivers are completely implemented in Safari's current implementation (WebRTC in Safari is still a WIP, but really close to be completed I think, and they're probably thinking of adding native support for WebRTC in iOS 11, I think I saw something about it in their last keynote).

Completely agree with @kekkokk, with might need to discuss adapterjs and/or other solutions. Anyway, thanks @mccob for playing with Safari. Can you try enabling h264 in the rtp_media_config?

jcague commented 7 years ago

FYI: https://github.com/lynckia/licode/compare/master...jcague:poc/safari this is the result of a couple of hours trying to make it work... I'm not proud of it, it's plenty of hacks!! so please don't use it in production, staging or even debugging :joy:

kekkokk commented 7 years ago

@jcague any plan to try edge? I opened the most detailed post i could @ DISCOURSE but I had no response :( in business environments is way more used than safari. Another question I never asked: is licode h264 ready? I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264. Do you think I could switch seamlessly into h264 without problems? or something (ex. externaloutput) will break? I never tried....

menelike commented 7 years ago

@jcague

h.264 support in Licode is not perfect

Are you talking about P2P oder the MCU mode?

Asking because we need h264 support on the MCU, because of

I implemented licode also in iOS and Android but especially in the last one I can feel vp8 more "stressful" than h264.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now* and observed performance problems, we'd love to try h264 since this might be a hardware support related issue.

*We use adapter.js and actually the chrome stack for iOS, works fine for a long time now (but we don't use the latest features like SimulCast, but Opus, Isac etc works fine!)

thehappycoder commented 7 years ago

Is there any ETA on when Licode starts supporting Safari? Even if it's not perfect but good-enough for some limited use cases.

miniruwan commented 6 years ago

Calls

I found the reason why safari doesn't receive media from other browsers.

I didn't receive any data in Safari from publishers in Chrome and Firefox.

MCU doesn't send answer sdp until it receives ICE candidates. But safari doesn't generate and send ICE candidates. https://bugs.webkit.org/show_bug.cgi?id=176157 One way to fix this is a shim in adapter.js (https://github.com/webrtc/adapter/issues/661)

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

Recording

For call recording to work, ExternalOutput probably needs to be changed to follow the h264 specs. Is there any ETA for this?

mkhahani commented 6 years ago

After locally changing adapter js, I could make it work. Now safari receives audio and video from other browsers.

@miniruwan Could you get it to work with Licode? I built EC with the latest adapter.js (v6.3.2) but still could not receive audio/video in Safari.

kekkokk commented 6 years ago

try with this SafariStack

/* global */

import Logger from '../utils/Logger';
import BaseStack from './BaseStack';

const SafariStack = (specInput) => {
  Logger.info('Starting Safari stack');
  const spec = specInput;

  const newIces = [];

  spec.iceServers.forEach((server) => {
    const newServer = {};
    if (server.url) {
      newServer.urls = server.url;
    }
    if (server.username) {
      newServer.username = server.username;
    }
    if (server.credential) {
      newServer.credential = server.credential;
    }
    newIces.push(newServer);
  });

  spec.iceServers = newIces;

  const that = BaseStack(spec);

  return that;
};

export default SafariStack;

mkhahani commented 6 years ago

Thanks for the response but it didn't help :(

Hopefully in p2p mode, Chrome2Safari and Safari2Chrome work well on both MacOS and iOS. I believe it can be fixed to work also in MCU mode.

Even I think Chrome over iOS must be able to receive audio/video after Safari added support for WebRTC since Chrome uses WKWebView component for iOS and it supports RTCPeerConnection now.

menelike commented 6 years ago

added support for WebRTC

Just my two cents...

Well yes, they "support" webRTC but still, VP8 is not supported just to name one of the many drawbacks. Without going into the details on how flawed webRTC on iOS is, as long as you have to rely on mobile browsers between iOS and Android, results are unpredictable. Your best chance will be to stick to h264, but then licode doesn't support h264 within the MCU.

IMHO for a complete working support for all devices including desktop browsers, iOS and Android clients need to be implemented natively. In addition/or instead a transcoding MCU is advised.

kekkokk commented 6 years ago

actually Licode does support for h264. and works well with safari. I actually made lots of changes to be able to, mainly you can see my EDGE stack PR and add the stack I posted few comments above

mkhahani commented 6 years ago

@menelike Thanks for your advice. I'm just trying to find a way to add support for iOS in my app and it's become a nightmare.

We use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc for iOS right now

Actually I was going to write my own hybrid app using cordova+iosrtc plugin, as discussed here, but I figure it out that the provided iOSStack is out of sync with the latest licode changes. Do you have any update on that?

mkhahani commented 6 years ago

@kekkokk Are you saying you could get the mcu to work with Safari+iOS? Then where can I find your PR?

menelike commented 6 years ago

@mkhahani We have to use https://github.com/BasqueVoIPMafia/cordova-plugin-iosrtc as well, and it's a pain. The project is almost dead, one major problem will be, that h264 playback will result in a green screen (h264 is the first choice on iOS because it should reduce the CPU load)

I think all work regarding webRTC has shifted from Cordova to react native (https://github.com/oney/react-native-webrtc), imho Cordova and webRTC will never work, especially not since getUserMedia is not available in WKWebView.

We also switched from licode to https://github.com/meetecho/janus-gateway, but this won't help you in your situation.

kekkokk commented 6 years ago

@mkhahani We actually have one dev only for iOS and we did a native app with the webrtc core. the webrtc core supports vp8 on iphone.

mkhahani commented 6 years ago

@menelike

imho Cordova and webRTC will never work

But it was working in my tests before the latest changes in Erizo Client.

We also switched from licode to https://github.com/meetecho/janus-gateway

Do you really found Janus better than Licode?!

mkhahani commented 6 years ago

@kekkokk

We actually have one dev only for iOS and we did a native app with the webrtc core.

Wow that's a lot of work! What do you mean by the "webrtc core" exactly? Are you using the one provided by webrtc.org? And is that compatible with Licode?

kekkokk commented 6 years ago

yeah sure! that's a ton of works but, actually, is the one used by chrome. so the sdps are quite similar and supports vp8.

mkhahani commented 6 years ago

@kekkokk @menelike What do you think about Temasys WebRTC Plugin which is used by Jitsi to add support for WebRTC in IE and Safari?

kekkokk commented 6 years ago

we are using this only for IE and works quite well (produces sdp very similar to chrome). For the safari side, we found very difficult, now that safari 11 supports natively webrtc, to interoperate with the plugin. It was too difficult to differentiate safari versions and choose to use the plugin or not. ATM we enabled only safari >= 11 in our platform so the call will init in h264 room

ps. doesn't support simulcast ofc

zevarito commented 6 years ago

@mkhahani There is an IOS ObjC client for Licode I've done a while ago, not sure how it works with latest version of Licode since I didn't had chance to work on it lately but I am sure if it doesn't work out-of-the-box it just need some signaling-protocol update almost. PR and maintainers are welcome, Link bellow in case you want to check it out.

https://github.com/zevarito/Licode-ErizoClientIOS

mkhahani commented 6 years ago

Thank you @zevarito, I am aware of your iOS app but I'm trying to avoid writing native since I've done a lot of work on other parts of my app like ui, signaling, etc.

jaydgoss commented 6 years ago

@mkhahani Did you find that Chrome to Safari, iOS to macOS combinations work in p2p mode?

mkhahani commented 6 years ago

Using h264, Windows.Chrome <=> macOS.Safari worked fine in both directions. There was a problem with rendering video on iOS but I didn't spend time to fix it.

pooyarm commented 6 years ago

@mkhahani I just created an ios client with react-native and react-native-webrtc. I suggest use that, the only changes it needs is to change BaseStack to use react-native-webrtc, just with importing them:

import {
  RTCPeerConnection,
  RTCIceCandidate,
  RTCSessionDescription
} from 'react-native-webrtc';

Also you need to change FcStack to use BaseStack. Ask me if you had any problems.

But I have also an issue regarding Safari on mac, it starts but it can't hear and see the other participants (other participants on windows chrome) also the others can't see the safari, I think it's because of the following error in baseStack at processAnswer:

OperationError: Failed to set remote answer sdp: Session error code: ERROR_CONTENT. Session error description: Failed to set remote video description send parameters

can anyone help me around this?

mkhahani commented 6 years ago

@pooyarm Thanks for your suggestion. Then I have to migrate to React which is a lot of work due to the amount of codes in my app.

mkhahani commented 6 years ago

There was a problem with rendering video on iOS but I didn't spend time to fix it.

It seems adding playsinline attribute to the video tag fixes the issue on iOS. Here is a great artice, Guide to WebRTC with Safari.

jcague commented 6 years ago

@mkhahani we're currently adding playsinline within the VideoPlayer so it should work, did you need to add any other attribute?

mkhahani commented 6 years ago

@jcague Very good! No I don't think so.

mkhahani commented 5 years ago

We decided to develop native app as our iOS solution. Some friends suggested React Native but Google Flutter seems also a good choice as a cross platform SDK. My concern is just about how does it deal with WebRTC. Since we may share part of the code publicly and it suppose to work with Licode, I would like to have your thoughts and ideas.

pooyarm commented 5 years ago

We decided to develop native app as our iOS solution. Some friends suggested React Native but Google Flutter seems also a good choice as a cross platform SDK. My concern is just about how does it deal with WebRTC. Since we may share part of the code publicly and it suppose to work with Licode, I would like to have your thoughts and ideas.

@mkhahani If you want to use React Native there is a webrtc library which can be used and licode js client can talk with that. https://github.com/oney/react-native-webrtc I used these two to build conference on react native and this worked with some adjustments. For instance, to have user's stream with Erizo.Stream, first you need to use getUserMedia method from 'react-native-webrtc' library and then create Erizo.Stream object using created stream, here is the code example:

const WebrtcGetUserMedia = (isFront = true, attributes = {}) => {
    return new Promise((success, error) => {
        MediaStreamTrack
        .getSources()
        .then(sourceInfos => {
            let videoSourceId;
            for (let i = 0; i < sourceInfos.length; i++) {
                const sourceInfo = sourceInfos[i];
                if(sourceInfo.kind == "video" && sourceInfo.facing == (isFront ? "front" : "back")) {
                    videoSourceId = sourceInfo.id;
                }
            }
            return getUserMedia({
                audio: true,
                video: {
                    mandatory: {
                        minFrameRate: 30
                    },
                    facingMode: (isFront ? "user" : "environment"),
                    optional: (videoSourceId ? [{sourceId: videoSourceId}] : [])
                }
            });
        })
        .then(stream => {
            var streamObj = Erizo.Stream(undefined, {
                stream,
                video: true,
                audio: true,
                local: true,
                data: true,
                attributes
            });
            success(streamObj);
        })
        .catch((e) => {
            error(e);
        });
    });
}

mccob commented 5 years ago

Have you seen that VP8 support will be in Safari soon ? Available in Safari Preview Release

See :

may Safari / iOS will have also that, see https://bugs.webkit.org/show_bug.cgi?id=189976 mentioning ios

mkhahani commented 5 years ago

Have you seen that VP8 support will be in Safari soon ? Available in Safari Preview Release

I can't believe that Apple has added support for VP8!! It's the power of WebRTC 😎. So better to wait and see what happens.

kekkokk commented 5 years ago

This is a great news for the community! I talked with fippo and seems that VP8 in the developer preview is in a stage that features can't be discoverable, so we should wait a few more weeks to better understand the behaviour, but I mean, it's only matter of time. I never thought Apple would have release VP8. I'll try to give it a try next week!

mccob commented 5 years ago

I have tested with adapter.js and without (with a version of a licode built a few months ago). I have javascript errors. Maybe javascript client stack need some update (or maybe it's already updated on last revision).

kekkokk commented 5 years ago

I tested safari vp8 with our current modified webrtc stack in licode and works like a charm. reacts very well to nack and bandwidth drop. way more stable than current Edge implementation.

mkhahani commented 5 years ago

@kekkokk Interesting!! May I have your stack please?

kekkokk commented 5 years ago

Have you tried with your basestack? it should be work fine. I mean, my base stack (js) is very similar to the project one. If it does not work I suggest you to try to disable the single peerconnection feature and comment this line: https://github.com/lynckia/licode/blob/4ddea91489cc7cbd3b8466a9832156bf4684f642/erizo_controller/erizoClient/src/Stream.js#L38

If it does not work it's maybe something we changed in c++ side but I don't think

mkhahani commented 5 years ago

Does it mean I need to change the following code to use BaseStack?

https://github.com/lynckia/licode/blob/4ddea91489cc7cbd3b8466a9832156bf4684f642/erizo_controller/erizoClient/src/ErizoConnectionManager.js#L35-L37

kekkokk commented 5 years ago

no chrome stack should be ok, it only shim the simulcast

mkhahani commented 5 years ago

I get the following errors when publishing/subscribing with Safari using VP8 while it works fine with H264:

mccob commented 5 years ago

update :

working without any update with last licode release (and Safari Tech Preview). Thank you @kekkokk and everyone. Hope Apple will provide this to iOS soon.

lidedongsn commented 5 years ago

publish from Safari ,sdp "a=sendrecv", Safari send all candidates, and licode acts "All candidates received", but there is not to continue, nothing going on! ICE state not ready. why?

mkhahani commented 5 years ago

@mccob iOS 12 is out and it works using Safari in P2P mode! But I'm still getting the above error in MCU mode. What version of Licode are you using? Any customization?

mccob commented 5 years ago

No relation with VP8, iOS 12 was out before Apple announce VP8 support.

Licode can work with h264, in P2P mode only ? Because I already try in no P2P mode it's not working.

mkhahani commented 5 years ago

I see but I also get the error on Safari Tech Preview. How could you get it to work?

mccob commented 5 years ago

Do you enable vp8 support on preview options ?

lynckia / licode

Support Apple Safari #915

Calls

Recording