It’s been a few years since we got into digging around Sky+HD’s UPnP control protocol, but having recently updated to Sky Q, there are a few changes that break the way we were talking to the box.
Backwards Compatibility
The Sky Q box contains pretty much the same UPnP advertised control system as Sky+HD, and includes a couple of new endpoints (service nodes) to do things that don’t seem too interesting. A “Pairing” service node and a “BasicManagement” service node.
There is one potentially breaking change though. The SkyServe2 service node now provides the service type urn:schemas-nds-com:service:SkyPlay:2 as well as the SkyControl2 service node. If you register a listener, or make any calls against the SkyServe2 node, you won’t get any useful data. This means if you’re using a library like dalhundal/sky-plus-hd, you’ll need to skip over the response from SkyServe2, and just use SkyControl2 like before.
Ultra HD
A couple of notes about UHD broadcasts. They currently broadcast on a channel with the ID 9000 (Hex 2328) (named “Ultra HD”) which isn’t tunable by number, only via EPG prompts when you’re on a station that is simulcast in UHD. Currently, if you query sky’s web services for the channel logo for 9000 you’ll get the Sky Sports 1 logo, so I think it’s safe to assume (for now at least) that if you’re watching UHD on channel 9000, you’re watching whatever is on Sky Sports 1 HD. This is important is you’re writing a logger for what you watch on TV, as there is no listings for 9000 available via tv.sky.com – so you’ll need to pretend you’re watching Hex FA2.
This will also (at least based on test broadcasts) be the case for Sky Sports F1 in Ultra HD. This is currently being broadcast as channel ID 9002 (Hex 232A) (named “Sky Sports F1 UHD”). You can rewrite that channel number to Hex EFB, Sky Sports F1 HD.
For UHD recordings, there’s a new flag on a recorded item simply called ‘uhd’ that will be 1 for UHD content.
The cool new stuff – RESTful and WebSocket interface
The new Sky Q app talks to the Sky Q box solely over a new RESTful interface (with a hint of WebSockets). This is cool, because the XML/SOAP of the UPnP interface was the worst thing ever and I hate SOAP forever.
The RESTful interface lives on port 9006 for the Sky Q box (it still uses UPnP for box IP address discovery). Once it’s in, there are a few endpoints it uses, all using /as/ as a base. There are /status urls that are available for each service, but these are only accessible if you pretend to be a WebSocket.
For example, you can get a full system overview at /as/system/status – but in order to access this URL, you need to send the upgrade headers, so:
curl "http://<sky_box_ip>:9006/as/system/status" -H "Connection: Upgrade" -H "Upgrade: websocket"
You’ll get a bunch of JSON back from that, which includes pretty much all the detail about your box, including a section called entitlements which details your package, and a few other things, such as trials. Apparently I’m a FIELD_TRIAL_1 and a GATEWAYENABLER. (I have no idea what any of those thing are) – There’s some stuff about hdmi which is a bit interesting – it talks about established HDCP connections, and if your display supports 2160p10bit. Also a bunch of things about your viewing card (that you probably don’t ever want to share on the internet)
This acts as a subscription, so any changes to that get pushed to you automatically.
Presumably, if/when they build the REST api out to support controlling the box via the Sky Q app, you’ll also be able to use a WebSocket to get notified of channel changes.
Currently, there are the following non-WS RESTful endpoints used by the Sky Q app:
GET /as/system/information – Returns a JSON array of a bunch of system information.
GET /as/pvr?limit=1000&offset=0 – Return the first 1000 recorded programs on the Sky Q box. This includes pushed on demand content, box sets and recordings.
GET /as/pvr/details/$pvrid – Return full details for an individual recording.
Live Streaming (and recording copies)
All of UPnP and RESTful endpoints expose streaming URLs for live channels and DVR content. They’re served on port 4730. There’s usually a URL for the source quality, and a transcoded version which I assume is used by the iOS and iPad clients when you download programs. These are just .ts muxed video files, but don’t get excited – the tracks are encrypted by whatever the broadcast medium was. For the vast majority of content, this will be Cisco’s (NDS) VideoGuard, the same for any VOD content.
Interestingly, Sky Go recently switched to use a native compiled version of Cisco’s VideoGuard on Mac and Windows because Silverlight was discontinued. This presumably improves Sky’s internal workflow and ingress, as they’re using one super secure encryption everywhere.
Cisco (formally NDS, who Cisco bought) also write all the software for the Sky boxes, speaking of which:
Sky Q EPG
The EPG is interesting. The vast majority of the content displayed is loaded over HTTP. It appears (based on the returned JSON) that Sky have a CMS that they use to curate the UI you see. It’s built around nodes, which load dynamically as you enter each section, with etags and last-modified headers to control caching. The CMS and base loading happens over HTTP from config.ethan.interactive.sky.com.
EPG personalisation is based on the Sky Q box deviceID. Again, unencrypted this is loaded from ethpersonalised.recs.sky.com.
The whole system uses a UUIDs to identify content. Specific media information is then loaded over UDP, which seems weird to me – and obviously makes life much harder to try and understand. I’ve never had to talk to a UDP service before. Anyway, that service runs on oig-e.sky.com.
The majority of thumbnail, images, and (all?) VOD is all powered from an Akamai CDN at cdn.sky.com.
You can manually form the URL to get an image for a DVR recording using the programmeuuid, and osid.
Activity Monitoring
Something that’s a little more concerning is that Sky log virtually every thing you do on the box, by POSTing data to bd01.sky.com, unencrypted.
That data includes some potentially sensitive information, like what you’re viewing, your box serial number and even things like your remote control battery level and serial number. That probably should at least have TLS protection.
Funnily enough, that service appears to often be overloaded, and will return a 503 error after about 5 seconds. The queue for activity monitoring continues though, and it’ll immediately send the next request after that 503. I do wonder if has potential for system instability if the queue gets too big…
Update – 21st March 2017
There are a few more endpoints: /as/services lists all the channels on the EPG, with /as/services/details/SID giving you more detail for a specific channel (but not that interesting). Even less interesting (for now) is /as/apps/status (WebSocket) that just tells you there’s an app called com.bskyb.epgui which is visible.
It looks like data for the EPG is still loaded over satellite – but it does specific program lookups over an EPG service that’s new and lives at awk.epgsky.com – annoyingly it doesn’t return EPG data for the Ultra HD channels, but it’s data includes something called an “eid” which is used to schedule recordings.
You just call /as/pvr/action/bookrecording?eid=<eid> to book it to record it. You’ll be given an internal pvrid that’s unique to your box, and then you can call /as/pvr/action/delete?pvrid=<pvrid> to remove it. You can also use that delete command with the list of PVR recordings you get from /as/pvr/ – This is how the android app works (and hopefully how the iOS app will from the next update)
It looks like none of the UPnP control methods we used on Sky+HD work on Sky Q. You’ll just be told “TRANSPORT IS LOCKED” whenever you try to do change channels, pause or fast forward.