Web inspector on windows

[Philip2809]

Trying to setup the web inspector, I have downloaded iTunes from the Microsoft store and tried different versions of the ios-webkit-debug-proxy but port 9222 is always unused and the page on port 9221 is empty (only: "iOS Devices:"). Is this because I am on windows, too new iTunes version or something else? Any idea?

[SheaSmith]

I personally use Windows as well along with the latest iTunes version, so that shouldn't be an issue. I assume you've followed the steps to get the web inspector enabled on the ATV itself?

Additionally, if you download libimobiledevice for Windows (e.g. from https://github.com/L1ghtmann/libimobiledevice/releases/tag/suite-exe-076234c) does idevicesyslog show anything?

[Philip2809]

Yes, I have set all of the files up and the internal mode, I can add sites thanks to that internal mode, and the idevicesyslog only tells me to plug in a device. Do I need to sign into iTunes on either device? I have tried multiple usb cables as well.

edit: do i need to reboot or do something certain to get it to work? edit 2: Thank ,finally got it to connect in the end, needed to reboot my pc and be patient!

[Philip2809]

Now that i gotlogging running i ge: NSURLConnection/CFURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9824) Oct 5 13:48:52 AppleTV[560] : T:[0x2c73000] JS : XHR connection failed: An SSL error has occurred and a secure connection to the server cannot be made. In the syslog when trying to connect to a https site, this also happens with wget or curl, any idea what is causing this?

edit: is it because for every site i add i need to add a .cer in the profiles thing? edit2: also, cant get onExecuteQuery to run, which is needed for the screensaver right?

[SheaSmith]

In the syslog when trying to connect to a https site, this also happens with wget or curl, any idea what is causing this?

Probably either expired SSL certificates (some built-in ATV stuff falls into this category, particularly third party content) or certificate authorities that the ATV doesn't know about (Let's Encrypt for example). For the latter category, you need to install the Lets Encrypt root CA certificate into the ATV's profiles page.

edit: is it because for every site i add i need to add a .cer in the profiles thing?

No, you'll only need to install certificates if you're using a newer CA like Lets Encrypt, or if you're going MITM proxying to get traffic (i.e. what PlexConnect essentially does).

edit2: also, cant get onExecuteQuery to run, which is needed for the screensaver right?

Yes, it's used for the screensaver. Have you added the appropriate screensaver sections to your bag.plist? I've found the following in one of my demo projects:

<key>screensaver</key>
<array>
    <dict>
        <key>preview-image</key>
        <dict>
            <key>720</key>
            <dict>
                <key>image</key>
                <string>http://trailers.apple.com/appletv/us/images/icons/trailerslogo.png</string>
            </dict>
            <key>1080</key>
            <dict>
                <key>image</key>
                <string>http://trailers.apple.com/appletv/us/images/icons/trailerslogo.png</string>
            </dict>
        </dict>
        <key>canonical-name</key>
        <string>ATVFeedScreensaverPhotos</string>
        <key>has-multiple-photo-collections</key>
        <true />
        <key>preferred-order</key>
        <integer>475</integer>
    </dict>
</array>

Have you had any luck with the web inspector?

[Philip2809]

Alright, thank you will try to add the root certificate to my profiles, I have added the following to my bag.plist:

<key>screensaver</key>
    <array>
        <dict>
            <key>preview-image</key>
            <dict>
                <key>720</key>
                <dict>
                    <key>image</key>
                    <string>https://www.w3schools.com/howto/photographer.jpg</string>
                </dict>
                <key>1080</key>
                <dict>
                    <key>image</key>
                    <string>https://www.w3schools.com/howto/photographer.jpg</string>
                </dict>
            </dict>
            <key>canonical-name</key>
            <string>ATVFeedScreensaverPhotos</string>
            <key>has-multiple-photo-collections</key>
            <false />
            <key>photo-collection-identifier</key>
            <string>ph-test</string>
            <key>preferred-order</key>
            <integer>101</integer>
        </dict>
    </array>

And I have confirmed that both onScreensaverPhotosSelectionEntry and onScreensaverPhotosSelectionExit functions run when I select my screen saver collection, however when I do the preview it defaults back to the nature photos and no sign on it ever executing the onExecuteQuery, maybe that is not run immediately, will need to do more digging here.

Yes, I got the web inspector up and running, thank you so much, that will help me a lot. I also made my app appear in the top row, and I wonder, did you ever figure out how to make a custom "top-shelf", or whatever it is called? And what does the "top-shelf-url" do in the plist file?

My last question is the design of the app, I see some people that make the webserver do the work to render the templates and then serve a complete .xml file while others run it via javascript, is any one of them better to do?

[SheaSmith]

What's the contents of your onScreensaverPhotosSelectionEntry function? For reference, I have this in my test app, which I was able to use to create the Typescript types for the onExecuteQuery method.

atv.onScreensaverPhotosSelectionEntry = function () {
    atv.setScreensaverPhotosCollection({
        id: "sample",
        name: "Sample 2",
        type: 'collection' // Must be set to 'collection'
    });
    console.log("screensaver selection entry");
};
atv.onScreensaverPhotosSelectionExit = function () {
    return console.log("Screensaver exit");
};
atv.onExecuteQuery = function (query, callback) {
    query.filters.forEach(function (f) {
        return console.log(f.operation, f.property, f.value);
    });
    console.log(query, query.length, query.shuffle);
    callback.success([{
        id: 'test',
        type: 'photo',
        'assets': [{width: 10, height: 10, src: 'https://avatars.githubusercontent.com/u/3637556'}]
    }]);
    callback.failure("test");
};

did you ever figure out how to make a custom "top-shelf", or whatever it is called? And what does the "top-shelf-url" do in the plist file?

Never did figure either of those out. My suspicion was that it was either an unreleased feature (i.e. from memory Netflix used to have a top shelf defined, despite never appearing in the top row), or was used in an older version of Apple TV Experience/iOS. My suspicion was that they might be built in, but if you're able to check the JS for Apple TV+ it's possible it might have some info (although I strongly suspect it's built-in these days).

For reference, here's the JS file for Apple TV+: https://apps.mzstatic.com/content/df6d5b3d1ab640b28ae04deaa2838946/application.js and then the storefront plist that defines every app: https://itunes.apple.com/WebObjects/MZStore.woa/wa/storeFront (you will need a Apple TV user agent, I found itunesstored/1.0 iOS/8.4.4 AppleTV/7.8 model/AppleTV3,2 build/12H937 (3; dt:12) online and it works).

My last question is the design of the app, I see some people that make the webserver do the work to render the templates and then serve a complete .xml file while others run it via javascript, is any one of them better to do?

It's mostly personal preference. My web application experience is generally SPAs (specifically Angular), so I tended to go down the route (also hence why I made this framework at all, because I wanted a semi-Angular type experience for building these). Other's prefer to render server side.

However, fully JavaScript apps have two limitations - firstly, the search results can only be rendered server side. There is a hack to work around this, but it's not that great. The second is that the performance is definitely a lot worse with a substantial amount of JS. If you're comfortable with both, then predominantly server side (with some client side JS for things like lazy loading for example) is probably the way to go. Apple themselves use a mix (server side for Trailers, client side for Apple TV+).

[Philip2809]

I have tried exactly what you used for the screensaver but is still not working, but now I can load in the flickr js thanks to that user agent, so will try to dig into how to fix it, because I can get the flickr screensaver to work!

I understand about the top shelf, will see if I can find anything in the js. The other apps, the movies, computers etc, is there any way to hide them? Also If I understand correctly, these are .ipa apps?

I understand about this the js stuff, I am also mainly familiar with Angular and React so doing it that way is probably the way I will do, I will probably use your atv.js, awesome thing you have made here! For my backend I will probobly do python because I got that up and running on the apple tv, did you ever get other apps to run on the apple tv?

Again thanks for this awesome project!

[Philip2809]

About the screensaver, using flickr it has a loading wheel after selecting a screensaver, but mine when even using the same code does not! Could it be a problem with webpack?

Edit: tried it with minimal js just like flickr does and still not working. edit 2: Moved on from the screensaver for now, I am wondering what exactly the "onVolatileReload" does and when the callback happens, cant figure it out, can you help me understand? edit 3: If you know which root certificates to add to make it get most of websites today, please tell me!

[SheaSmith]

I understand about the top shelf, will see if I can find anything in the js. The other apps, the movies, computers etc, is there any way to hide them? Also If I understand correctly, these are .ipa apps?

I don't think there's a way to hide them. You can bump them into the next row if you have enough top shelf items though. Not sure about the app format. I know that the actual Apple TV homescreen is a .app, but not sure about movies etc. Probably depends if they show up in the web inspector menu or not.

did you ever get other apps to run on the apple tv?

Not really. I have the beginnings of a few other NZ streaming services, a Plex app, a Youtube one and a Disney+ one, but they're all several years old and I didn't get very far. Often video playback is a bit of an issue if you have to deal with any DRM stuff, although it still seems supported (for now).

Moved on from the screensaver for now, I am wondering what exactly the "onVolatileReload" does and when the callback happens, cant figure it out, can you help me understand?

onVolatileReload when combined with volatile="true" is called when the page is 'exhumed', which is when you navigate back to a particular page from another page. Some more details here which might be helpful: https://github.com/SheaSmith/atv.js/wiki/volatile https://github.com/SheaSmith/atv.js/wiki/onVolatileReload

edit 3: If you know which root certificates to add to make it get most of websites today, please tell me!

The only one I have installed (apart from the HTTP interception proxy I use) is for Lets Encrypt, which I think is this cert: https://letsencrypt.org/certs/isrgrootx1.pem

However if you have other websites not working, then you'll need to find the CA for that website and find the root certificate for it.

tried it with minimal js just like flickr does and still not working.

Hmm, not sure what's going on with that. Do you get any errors in idevicesyslog? That's usually pretty helpful for tracking down JS errors.

[Philip2809]

I see, thank you for all your answers, I have added that encrypt certificate among with others, that is the one for https://letsencrypt.org/ as will if I understand correctly? However when I do wget on "https://letsencrypt.org/" i get: OpenSSL: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version trying other protocols for wget gives similar errors or "Unable to establish SSL connection."

trying to get more info with: openssl s_client -connect letsencrypt.org:443 -showcerts only returns me: 648:error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version:s23_clnt.c:579:

I really would like to make so that the apple-tv itself can do these requests, because I wanna run the "backend" for my app on it, however worst case senario i can always have a proxy to some other device

I will try to look into the screensaver thing more, but moving on from that so I get some other stuff done as well, to me it looks like it never realizes there is a onExecuteQuery function, but will need to try it out more.

edit: by thing ca bundle from mozilla https://curl.se/docs/caextract.html I can connect to https://example.com (which I cant without that ca-bundle) but still not to letsencrypt or some other sites, however adding this as a profile did not change anything on the wget or curl command, i still need to specify it to use that cafile

edit 3: managed to solve my problms with lzma

[SheaSmith]

My guess is your issues with the certificates is because the jailbreak libs are very very old. They probably don't support modern TLS versions. It's probably theoretically possible to compile them yourself, but I wouldn't know where to start personally!

My advice would be to probably not host things directly on the ATV itself. It's pretty resource limited anyway, so hosting a complicated JS app + a backend might be too much for it anyway.

[Philip2809]

I am pretty sure you are right! It looks like most sites need tls 1.3 but curl on the appletv does not have that yet. Will see if I can figure out how to update it otherwise an imageproxy will be needed.

About the performance, you are probobly right about that as well, however tought that because plexconnect did it that way it could be possible.

Again, thanks for all the help and answering all my questions!

[SheaSmith]

No problem. Happy to share this (otherwise useless) knowledge with someone!

[Philip2809]

So I did some testing, the AppleTV app thats running all the code etc, supports TLS 1.2, and can load images etc, etc,. It does not support TLS 1.3 (I hope that if the day comes where TLS 1.2 is deprecated apples releases a fix, but I kinda doubt they will do so). However in root I only have SSLv3.

I also dug a little deeper into the AppleTV binary and found some interesting things. The available topshelfcontrollers seam to be:

BRTopShelfController
MOVTopShelfController
ATVMoviesTopShelfController
TVAPPTopShelfController
TVTopShelfController
ATVTVShowsTopShelfController
MUSICTopShelfController
ATVMusicTopShelfController
COMPUTERSTopShelfController
SETTINGSTopShelfController
ATVRadioTopShelfController

And selecting the TVAPP it becomes the apple tv + one, selecting settings it just shows the apple tv logo, etc etc. When using the TVAPP and selecting a movie it sets a variable on the atv object. atv.topShelfSelection. For example selecting a movie gives me:

{
    "event": "select", // "select" or "play"
    "type": "Movie", // But I guess it can be other things, like "Show"
    "id": "umc.cmc.c3xhu25rw4jxxxzq4oio6snu",
    "url": "https://tv.apple.com/movie/wolfs/umc.cmc.c3xhu25rw4jxxxzq4oio6snu"
}

So as I think where you said somewhere before that the topshelf is built into the AppleTV.app thing. Did you ever looking into building your own .app thing? The kodi thing seams to be a custom built .app and you have also used KodiTopShelfController in the wiki page for adding items to the top row.

I am wondering if there is any way to add a header to a request when the appletv is sending it out? Mainly for when loading images or loading the m3u8 stream in a httpLiveStreamingVideoAsset? I have tried the atv.onGenerateRequest but it does not run for images or streams, and only runs if you are requesting .xml files with the doesJavaScriptLoadRoot set to false.

Edit: To fix the header thing if not possible somehow by code is to intercept the dns maybe? Apply my own header and forward the request, (edit2: this could also be used to fix the tls1.3 problem ones it becomes a problem)

[SheaSmith]

Did you ever looking into building your own .app thing?

Not really. I'm not an iOS dev, so it's not something I looked into too much. I think for me the reality was that I'm not sure how feasible it is to re-use the Apple TV UI, which was the main thing I was going for. There seems to be some examples floating around on Github of projects that are .apps (Kodi being one example), but most of them are built against the Apple TV 2 jailbreak + headers, meaning there might be some UI components missing (e.g. when Apple added Apple TV+ they added a substantial amount of new components).

I am wondering if there is any way to add a header to a request when the appletv is sending it out? Mainly for when loading images or loading the m3u8 stream in a httpLiveStreamingVideoAsset? I have tried the atv.onGenerateRequest but it does not run for images or streams, and only runs if you are requesting .xml files with the doesJavaScriptLoadRoot set to false.

atv.onGenerateRequest is only run for XHR and some XML requests (e.g. search results, or the one you've found), although with XML, there is a headers parameter on loadURL and loadAndSwapURL. So I don't think there's a way to add headers to images or media streams.

Edit: To fix the header thing if not possible somehow by code is to intercept the dns maybe? Apply my own header and forward the request, (edit2: this could also be used to fix the tls1.3 problem ones it becomes a problem)

Yeah that's one option. I personally just used a Cloudflare worker to forward some requests that the Apple TV couldn't process (specifically requesting a Fairplay DRM token where the request body needed to be raw binary data, rather than Base64 encoded)

[Philip2809]

I see, that is a cool way, I guess I also could have those things in a Cloudflare worker. I have played around some more, did you ever get a "up next" view to work on the video player, or the floatingbuttons item? I can't get these to work, the floatingbuttons seam however to block me to do anything else, pause, stop, view the desc etc.

I also found that you can use atv.contextMenu to load a popupmenu when you have held down an item, did you know you can do this?

Again, thanks for the amout of research you have put into this project! Its awesome!

[SheaSmith]

did you ever get a "up next" view to work on the video player

Yes, I was able to get this working. You need a combination of the atv.player.loadRelatedPlayback callback (example: https://github.com/SheaSmith/tvnz-plus-atv-3/blob/091841d169da2f7d201f4017117c1d7fe5b34e2b/src/app.ts#L170) and the upNextPresentationTime and upNextPresentationDuration on the httpLiveStreamingVideoAsset (example: https://github.com/SheaSmith/tvnz-plus-atv-3/blob/master/src/pages/play-show/play-show.xml). I did have an issue with them initially, because I couldn't get them to work correctly. It turns out the issue is that you need to ensure you have the tags in the exact same order as they appear in the XSD as they are defined as a xs:sequence (https://github.com/SheaSmith/atv.js/issues/3#issuecomment-1066408467)

or the floatingbuttons item? I can't get these to work, the floatingbuttons seam however to block me to do anything else, pause, stop, view the desc etc.

I was definitely able to get these to work, but I can't remember the exact behaviour. I suspect it might block the other UI elements (although I would imagine you should still be able to pause using the pause button on the remote).

I also found that you can use atv.contextMenu to load a popupmenu when you have held down an item, did you know you can do this?

Yes I was aware of this. I think I have it in some (unpublished) other projects of mine. It's also present in PlexConnect as well, but it seems like I forgot to add it to my TypeScript types.

[Philip2809]

Ah thanks that explains how it works! thank you! I might try to do the buttons later but not really a dealbreaker.

I am wondering if you have looked into the audioplayer, I am able to play m3u streams in both audio player types but can't get normal audio files to work. Always get this error:

AppleTV[684] <Error>: T:[Main] SYSTEM : Failed to find externalID for asset

have you seen this before, any idea on how to fix it? Edit: Tried to stream via vlc but cant get it to work, when using some radio stream I found I get this error: T:[Main] SYSTEM : Failed to find external asset and channel ID but it still plays, when using my stream it does however say: " Not reporting playback data." It also says that for the files

So turns out I was just stupid and did not serve the files statically, I served them as downloadable octet-streams