Closed Karolk99 closed 9 months ago
Send the whole stream at once on the Engine termination
In this scenario, we would spawn (if needed) the process responsible for sending the stream to S3 on the :add_component
event.
This process would monitor the Engine process and on the Engine termination, It would do the sending.
This solution requires some constraints, ex: the stream cannot be removed before the end of postprocessing, so the option persist
has to be reconsidered.
On the other hand, It gives us a lot of flexibility with the stream, ex: we can validate and repair the stream if needed before sending it to S3, we can use LL-HLS ( it's important, we can't do it easily in the storage scenario)
@impl true
def handle_call({:add_component, component_type, options}, _from, state) do
options =
Map.merge(
%{engine_pid: state.engine_pid, room_id: state.id},
options
)
with :ok <- check_component_allowed(component_type, state),
{:ok, component} <- Component.new(component_type, options) do
state = put_in(state, [:components, component.id], component)
if component_type == HLS do
on_hls_startup(state.id, component.metadata)
# spawning process responsible for s3
maybe_init_s3_process(state.engine_pid, s3_credentials)
end
:ok = Engine.add_endpoint(state.engine_pid, component.engine_endpoint, id: component.id)
.
.
.
end
Merging #115 (f59e145) into main (cd1d42f) will decrease coverage by
2.58%
. The diff coverage is28.57%
.
@@ Coverage Diff @@
## main #115 +/- ##
==========================================
- Coverage 86.24% 83.66% -2.58%
==========================================
Files 50 51 +1
Lines 923 961 +38
==========================================
+ Hits 796 804 +8
- Misses 127 157 +30
Files | Coverage Δ | |
---|---|---|
lib/jellyfish/component/hls/storage/ll.ex | 76.31% <ø> (ø) |
|
lib/jellyfish/component/hls/storage/regular.ex | 12.50% <ø> (ø) |
|
lib/jellyfish_web/api_spec/component/hls.ex | 100.00% <100.00%> (ø) |
|
lib/jellyfish/component/hls.ex | 85.71% <75.00%> (-8.41%) |
:arrow_down: |
.../jellyfish_web/controllers/component_controller.ex | 90.00% <0.00%> (-10.00%) |
:arrow_down: |
lib/jellyfish/room.ex | 83.44% <55.55%> (-1.77%) |
:arrow_down: |
lib/jellyfish/component/hls/storage/s3.ex | 0.00% <0.00%> (ø) |
Continue to review full report in Codecov by Sentry.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update cd1d42f...f59e145. Read the comment docs.
Acknowledging the stipulations set forth:
This draft is not meant to be reviewed entirely. The purpose of this PR is to show one of two possible solutions that can be fully implemented. In this case, HLS is sent to S3 by
Membrane.HTTPAdaptiveStream.Storage
implementation so every segment and manifest iteration is sent right after creation. Apart from sending it to the S3 bucket each segment is saved on the Jellyfish server. That allows us to serve live HLS simultaneously and fix some problems in postprocessing that may occur during the stream.The big disadvantage is that we cannot push HLS to S3 and simultaneously serve LL-HLS (it is technically possible but It would require a lot of work).