Site support Request: Realm.io Videos

[aaronvegh]

Up till a couple weeks ago, Realm hosted their videos on Youtube, but they've now switched to Wistia. Here's an example page: https://realm.io/news/alexis-gallagher-3d-touch-swift/

$ youtube-dl --version
2015.11.27.1

$ youtube-dl -jv "https://realm.io/news/alexis-gallagher-3d-touch-swift/"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-jv', u'https://realm.io/news/alexis-gallagher-3d-touch-swift/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.11.27.1
[debug] Python version 2.7.10 - Darwin-15.0.0-x86_64-i386-64bit
[debug] exe versions: none
[debug] Proxy map: {}
WARNING: Falling back on generic information extractor.
ERROR: Unsupported URL: https://realm.io/news/alexis-gallagher-3d-touch-swift/
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1282, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 248, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=etree.TreeBuilder(element_factory=_element_factory)))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 237, in _XML
    parser.feed(text)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 7, column 145
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 663, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 290, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 1887, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: https://realm.io/news/alexis-gallagher-3d-touch-swift/

[Hrxn]

Well, Wistia should already be supported, according to supportedsites.md

Edit:

Here is a link I found manually: https://embed-ssl.wistia.com/deliveries/b7835fb9475ffcd2c117004ffa9005fd32c8872e.bin

Is that the video you wanted?

[aaronvegh]

I'm not really interested in the video per se; I'm looking to ensure that Youtube-DL can support videos on this site. I think the reason it can't find the video is that Realm is injecting the video tags after the page loads using Javascript. I don't know if Youtube-DL supports loading JS and pulling video data that way, but it should because several sites that I've run into do just that.

[Hrxn]

Which site? Realm.com (Issue title) or Realm.io (Example link)?

Because realm.io doesn't look like a video site to me, how many videos are accessible there?

But you are right otherwise, the video embed isn't there initially, only after clicking that 'Play' symbol on the video..

[aaronvegh]

Yikes, sorry about that. It's realm.io, the mobile database developer. For some reason they publish a series of technical talk videos for iOS developers, so a naive visitor to the site wouldn't know they did such a thing. However, videos like those linked in my original are super-popular among developers. It seems they link primarily from their News page: https://realm.io/news/.

[Hrxn]

Okay, sounds interesting..

There are not that many videos there, I would assume, but it would be nice to get them easily, I agree.. So, if someone has the necessary time and skills to write a new information extractor, please go ahead.

BTW:

Mobile database? So, I can't use that database for a web application, a desktop app, or just as the database for my website backend? :wink:

[keith]

This seems to be related to https://github.com/rg3/youtube-dl/issues/8347. The issue is that the embedded videos on the site are not detected. Example https://realm.io/news/benji-encz-unidirectional-data-flow-swift/

[johnjohndoe]

Here is another example video on realm.io where the download attempt fails. Any news meanwhile?

[johnjohndoe]

Another video link for you to test the implementation with.