Open roelvz opened 4 years ago
For jsmpeg-vnc to work with the current jsmpeg version, it needs to send out an MPEGTS Stream. FFMpeg has a muxer to create this stream. So, yeah, it's certainly possible.
I want to come back to this project eventually, but I'm buried with "real" work at the moment. Don't hold your breath :/
Just for info, I wrote c++ code (started from jsmpeg-vnc for the idea but it turned out to me completely new code) that works together with the latest version of jsmpeg and also includes audio.
@roelvz Is your version (with sound) published anywhere? I would be interested to check it out!
Just for info, I wrote c++ code (started from jsmpeg-vnc for the idea but it turned out to me completely new code) that works together with the latest version of jsmpeg-vnc and also includes audio.
Hi Roel....by any chance you could share this?
I cannot make the code available, but here are some tips that may be helpful:
libavcodec
and is already done in jsmpeg-vnc. Encoding audio is similar. You'll need to use AV_CODEC_ID_MPEG1VIDEO
for video and AV_CODEC_ID_MP2
for audio with 2 channels. This is what I did back then for the audio:
_audioCodec = avcodec_find_encoder(AV_CODEC_ID_MP2);
_audioEncoder = avcodec_alloc_context3(_audioCodec);
_audioEncoder->sample_rate = outputSampleRate;
_audioEncoder->sample_fmt = AV_SAMPLE_FMT_S16;
_audioEncoder->channel_layout = av_get_default_channel_layout(2);
_audioEncoder->channels = av_get_channel_layout_nb_channels(_audioEncoder->channel_layout);
_audioEncoder->bit_rate = audioBitrate;
_audioEncoder->time_base = AVRational{1, _audioEncoder->sample_rate};
auto result = avcodec_open2(_audioEncoder, _audioCodec, nullptr);
libavformat
for this. Useful functions for initialisation are avformat_alloc_context
, av_guess_format
, avformat_new_stream
, avcodec_parameters_from_context
and avformat_write_header
. For getting the data onto the WebSocket you can use avio_alloc_context
.avcodec_send_frame
and avcodec_receive_packet
for both video and audio.av_write_frame
for both video and audio.swr_alloc
(init) and swr_convert
(real time). This happens before encoding off course (comparable to the scaling for video, which is already present in jsmpeg-vnc).Note: I'm not using this code anymore and it was all build to work with ffmpeg 4.0.2. Some of these functions are probably deprecated or deleted in recent ffmpeg versions.
Hello,
Do you think it's feasible to merge the latest version of jsmpeg into jsmpeg-vnc? I would like to do this because the latest jsmpeg version includes an audio decoder. I first tried with video only, but it doesn't work out of the box. The decoder thinks the received steam is garbage. I guess this is because the c++ part does not send a valid MPEG1-stream (because of a custom header each frame).
I tried to remove the custom header, but this does not seem to be working. Do you think it should be that simple? Or is there anything else I could be missing?
A better solution would be to take the custom header into account in the newest version of jsmpeg. Would this be hard? Could you give some directions on where to start?
For the audio part in the c++ code, I think this is pretty straightforward, but any tips are welcome.
Thank you