search

part-cw / lambdanative

LambdaNative is a cross-platform development environment written in Scheme, supporting Android, iOS, BlackBerry 10, OS X, Linux, Windows, OpenBSD, NetBSD, FreeBSD and OpenWrt.
http://www.lambdanative.org
Other
1.4k stars 86 forks source link

Using ffmpeg libraries for viewing video #132

Open ychakiris opened 8 years ago

ychakiris commented 8 years ago

I am considering using the lambdanative framework/platform for our Montessori lab school infrastructure called the Visible Classroom. It would be used as a desktop and mobile client to view and discuss Visible Classroom data that generated by the classroom learning environments. This would consist of text chatting, live video and archived video. Teachers would be mentored live as they teach. We already do some of this but our tools (whatsapp and ffmpeg, ffplay, ffserver, google docs, etc) are not at all ideal for the job. Our project probably will also see clinical applications to help mentor parents in home care of children with autistic spectrum disorder.

Aside from the fact that I like to hack in lisp, the lambdanative code base is easy to understand, documented, and looks very flexible to accommodate change. The fact that it is not a perfect mobile UI (as far as user experience is concerned) is not important to us at this time.

Currently the only video viewing and archiving software--and I have looked at a lot of alternatives--that approximates what we want is ffmpeg and ffplay. The interface of ffplay is primitive but it works.

I would like to attempt the following:

  1. Make ffmpeg as one of the lambdanative libraries
  2. Create a video module modeled on the code of ffplay
  3. be able to store video files on the device and receive video streams

ffplay currently uses SDL 1.2 as the UI interface to display video. It seems to me that one could use one of the following approaches to moving this into lambdanative:

  1. Incorporate SDL as an alternative in the ./loaders/ directory, take the events as they come in and then feed them into the lambdanative event pipeline as is done for each platform. Tricky code dealing with events will probably be needed.
  2. Extract the code from ffplay that does the decoding and display. Recreate its event loop(s) and threads inside lambdanative.

Any thoughts?

mgorges commented 8 years ago

@ychakiris sounds great! We are also looking at WebRTC integration at the moment which might be complimentary. The SDL piece might a bit more tricky, will discuss with @clpetersen on Tuesday.

ychakiris commented 8 years ago

I also looked at webRTC. For clinical use and for classroom use several things are very important:

  1. The ability to fast forward and backward to see overall "ecology"
  2. Good sound pickup
  3. Ability to annotate and select video segments of clinical or scientific interest
  4. Ability to look at multiple video sources at once at lower frame rates to alert the observer of situations that need attention

On Sunday, September 18, 2016, Matthias Görges notifications@github.com wrote:

@ychakiris https://github.com/ychakiris sounds great! We are also looking at WebRTC integration at the moment which might be complimentary. The SDL piece might a bit more tricky, will discuss with @clpetersen https://github.com/clpetersen on Tuesday.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/part-cw/lambdanative/issues/132#issuecomment-247882189, or mute the thread https://github.com/notifications/unsubscribe-auth/AQHMmncaMoL-snodQiEbchSSWHIbrkCgks5qra4dgaJpZM4J_8RK .

YItzchok Chakiris President Whole Systems Design, Inc. 2639 W. Greenleaf Chicago, Illinois 60645 cell: 773-620-3251

ychakiris commented 8 years ago

Plus

  1. Ability to share video segments, chats and discuss the ones of clinic or scientific interest

On Sunday, September 18, 2016, Yitzchok Chakiris ychakiris@gmail.com wrote:

I also looked at webRTC. For clinical use and for classroom use several things are very important:

  1. The ability to fast forward and backward to see overall "ecology"
  2. Good sound pickup
  3. Ability to annotate and select video segments of clinical or scientific interest
  4. Ability to look at multiple video sources at once at lower frame rates to alert the observer of situations that need attention

On Sunday, September 18, 2016, Matthias Görges <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

@ychakiris https://github.com/ychakiris sounds great! We are also looking at WebRTC integration at the moment which might be complimentary. The SDL piece might a bit more tricky, will discuss with @clpetersen https://github.com/clpetersen on Tuesday.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/part-cw/lambdanative/issues/132#issuecomment-247882189, or mute the thread https://github.com/notifications/unsubscribe-auth/AQHMmncaMoL-snodQiEbchSSWHIbrkCgks5qra4dgaJpZM4J_8RK .

YItzchok Chakiris President Whole Systems Design, Inc. 2639 W. Greenleaf Chicago, Illinois 60645 cell: 773-620-3251

YItzchok Chakiris President Whole Systems Design, Inc. 2639 W. Greenleaf Chicago, Illinois 60645 cell: 773-620-3251