search

awslabs / llrt

LLRT (Low Latency Runtime) is an experimental, lightweight JavaScript runtime designed to address the growing demand for fast and efficient Serverless applications.
Apache License 2.0
8k stars 352 forks source link

WHATWG Streams implementation #544

Open jackkleeman opened 4 weeks ago

jackkleeman commented 4 weeks ago

I'm fairly interested in contributing an implementation of the web stream API. As I see it there are two options:

  1. Use an all JS implementation, taking inspiration from the reference implementation from the working group, using some native APIs where necessary (as I understand from https://github.com/creatorrr/web-streams-polyfill, this will be necessary)
  2. Write a Rust implementation of the streams spec

I would probably lean towards doing the latter, either as a separate crate if it can be abstracted well or simply within this repo. Are there any opinions on how this should be done? Thanks!

richarddavison commented 4 weeks ago

Hi @jackkleeman. Thanks for considering contributing. #2 would almost be a requirement due to performance since there is no JIT in quick JS JavaScript implementation would be very slow. There is already a native rust implementation of a simple and incomplete streams API used by child_process module. The tricky part is that the streams API is very large, complex and contains a lot of cases so a native implementation would probably have to take some shortcuts and be not strictly compliant with Node.

jackkleeman commented 3 weeks ago

ok, i understand that we need a native implementation, and we have some initial work that we can extend, and that we should probably try and do this iteratively adding different parts of the api. will try and kick something off in the coming weeks!

jackkleeman commented 3 weeks ago

i have started work on this. indeed the spec is absolutely huge.... however, i think taking shortcuts will just make things harder tbh than just faithfully transferring the spec function by function (which seems to be what firefox devs did, for example). so thats what i am doing over in https://github.com/jackkleeman/llrt/tree/streams

richarddavison commented 1 week ago

@jackkleeman I took a peek in your repo and you've seen to have had amazing progress in such short time. Fantastic! Anything we can do to help?

jackkleeman commented 1 week ago

@richarddavison thanks! all good for now. im currently working on getting the readable streams web platform tests all working so we can maybe get that into a PR and i can move on to writeable