Karolk99
commented
10 months ago Send the whole stream at once on the Engine termination
In this scenario, we would spawn (if needed) the process responsible for sending the stream to S3 on the :add_component event.
This process would monitor the Engine process and on the Engine termination, It would do the sending.
This solution requires some constraints, ex: the stream cannot be removed before the end of postprocessing, so the option persist has to be reconsidered.
On the other hand, It gives us a lot of flexibility with the stream, ex: we can validate and repair the stream if needed before sending it to S3, we can use LL-HLS ( it's important, we can't do it easily in the storage scenario)
@impl true
def handle_call({:add_component, component_type, options}, _from, state) do
options =
Map.merge(
%{engine_pid: state.engine_pid, room_id: state.id},
options
)
with :ok <- check_component_allowed(component_type, state),
{:ok, component} <- Component.new(component_type, options) do
state = put_in(state, [:components, component.id], component)
if component_type == HLS do
on_hls_startup(state.id, component.metadata)
# spawning process responsible for s3
maybe_init_s3_process(state.engine_pid, s3_credentials)
end
:ok = Engine.add_endpoint(state.engine_pid, component.engine_endpoint, id: component.id)
.
.
.
end
codecov[bot]
Acknowledging the stipulations set forth:
This draft is not meant to be reviewed entirely. The purpose of this PR is to show one of two possible solutions that can be fully implemented. In this case, HLS is sent to S3 by
Membrane.HTTPAdaptiveStream.Storageimplementation so every segment and manifest iteration is sent right after creation. Apart from sending it to the S3 bucket each segment is saved on the Jellyfish server. That allows us to serve live HLS simultaneously and fix some problems in postprocessing that may occur during the stream.The big disadvantage is that we cannot push HLS to S3 and simultaneously serve LL-HLS (it is technically possible but It would require a lot of work).