unifiedstreaming / fmp4-ingest

Repository on shared work on developing a fragmented MPEG-4 ingest specification
32 stars 11 forks source link

PostURL ServerAddress #6

Open KevinCJohns opened 6 years ago

KevinCJohns commented 6 years ago

The test indicates that the PostURL ServerAddress can be either a hostname or IP address. Can we restrict to hostnames only? Use of IP Addresses should be discouraged as that creates a number of operational issues.

ghost commented 6 years ago

I disagree - I think that the ServerAddress SHOULD have a hostname, but it should not be mandatory. If operators choose to not have DNS in place in their networks, or can't for whatever reason, that's their call.

unifiedstreaming commented 6 years ago

It is also possible to enter the hostnames to IP Mapping in the etc/hosts table for example on linux when no DNS is present, Kevin could you elaborate further on the problems using the IP creates ? As I agree with James it makes sense to keep both options open to support different scenarios (e.g. private datacentre versus public cloud)

KevinCJohns commented 6 years ago

Maybe we can make supporting Hostnames a MUST at least? The operational complexity with IP addresses is with multi-tentant environments. It makes it easier to give each tenant a different hostname. If a server fails or due to capacity reasons you want to move a stream to a different server, a simple DNS updates makes that happen. You don't need to tell the customer to change their encoder and you don't need to worry about network topologies to move an IP. Further, using hostnames provides more flexibility on the ingest publication point for internal routing. As long as hostname support is a must that should be sufficient.

unifiedstreaming commented 6 years ago

I (Dirk) agree on hostname being a MUST

unifiedstreaming commented 6 years ago

Ok I (Rufael) changed the text according to Kevins recommendation, IP address could still work in some cases but now the text only refers to hostName, this is because a single ip adress can host multiple publishing points often separated by the host name, in case of a single publishing point on a node IP might also work, hostnames can also be mannualy assigned in systems without a DNS

unifiedstreaming commented 6 years ago

let me know if the issue is adressed and can be closed

KevinCJohns commented 6 years ago

I don't think the proposed text is sufficient as there are no normative requirements. I would suggest elaborating as follows:

In this PostURL the HostName is typically the hostname of the media processing entity or publishing point. The Live Encoder MUST support the use of DNS for resolution of hostnames to IP Addresses. The Live Encoder MUST honor the provided Time to Live in the DNS response and re-resolve the media processing entities Hostname according to the TTL. In the case of a long running post, the Live Encoder would only re-resolve the hostname should it need to re-connect due to a failed TCP session. For short running posts, the Live Encoder would re-resolve the media processing entities hostname between requests per the TTL.

The above text does not have to do into this exact section but any suitable section of the document.

unifiedstreaming commented 6 years ago

It is not good to impose too many implementation details and restrict too much the implementation details. The main goal is to have interoperability between the source and receiver based on a bitstream format and communication pattern (protocol). The aim is not to restrict the implementation of both with too many implementation details, I added a sentence that the hostname should be resolved frequently, long running posts are replaced with short running posts in this spec. Together, this should cover your use case of frequently changing hostname/IP mappings.

KevinCJohns commented 6 years ago

I disagree. Interop is not established on vagaries. If we just focus on the bitstream we have not established interoperability. We need to be strict in what is expected and implementations need to be liberal in what is actually received.

This is an area that has caused us much pain on many fronts. We should be very clear on expected behavior and operation. I don't think the proposed text is sufficient. It should explicit say the encoder must honor the TTL and provide guidance on how to deal with multiple A (or AAAA) records in a DNS response (recurse through them till you timeout or successfully connect). We have seen encoders that don't honor TTLs making it hard to move them if we have a server failure and don't try all the IPs we may return in a DNS response (although we don't typically do that for media ingest but that doesn't mean we shouldn't call it out).

unifiedstreaming commented 6 years ago

We understand the pain, but this is an implementation issue from the encoder side (poor implementation), if the encoders resolved domain names well within the TTL this should not happen (and any good implementation would work this way), I can add another line in the spec about this to increase awareness of the issue. But to be clear, this issue is not caused by the spec definition but by poor implementation.