Support Multipart/x-mixed-replace ?

[wkh237]

143 Not sure if it's possible but worth of trying.

[rares-lupascu]

i think i made some steps towards handling x-mixed-replace in js ... would it help sharing the code?

[wkh237]

Of course !

[rares-lupascu]

ok so this is the code i am using to debug this:

var oReq = new XMLHttpRequest();
    oReq.open("POST", 'http://192.168.1.1/osc/commands/execute', true);
    oReq.setRequestHeader("Content-Type", "multipart/x-mixed-replace; boundary=\"---osclivepreview---\"");
    oReq.onreadystatechange  = function() {
      if (oReq.readyState == 3) {
        var x = oReq.responseText.split("---osclivepreview---");
        console.log(oReq.status);
        console.log(x[x.length - 1]);
      }
    };
    oReq.send(JSON.stringify({
      name: "camera.getLivePreview"
    }));

u see there are 2 console logs there so the output (partial output) looks like this:

200

Content-type: image/jpeg
Content-Length:32387

�����
      



��
@�
!

��
�







  


}
!1AQa"q2���#B��R��$3br� 
%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz�������������������������������������������������������������������������










    

1) biggest pb is that this request does not stop and since this is a continuous stream it does not need to stop so i need to manully stop it somehow 2) second pb ... is my actual image starting from the first � until the end?

[wkh237]

@wowzaaa , thanks for the information ,this is going to be very helpful.

The multipart/x-mix-reaplce type response means each new part ( data between two ---osclivepreview--- boundary ) will replace the old one. That's why the server non-stopping send new part. If you're going to do it in JS context it's very difficult and inefficient, especially for RN.

Basically we can do this in native context and encode every new part in BASE64 so that you can use it this way (this is what I'm planning to do)

RNFetchBlob.fetch(method, url, headers, body)
   .part((partData) => {
     /* do something with BASE64 data */
   })

I'll update my progress in this thread, thank you 👍

[rares-lupascu]

i do not want to be rude but one of the needs of my app is to work in windows phones too ... and since u r talking about native code ... this mean no windows right? :)

[wkh237]

Unfortunately, yes. This library depends on lot of native implementations and we are not yet supporting Windows.

[rares-lupascu]

ok at least i will know where i stand :) ... i will now wait for news from you ... like a little kid on Christmas' eve :P

[rares-lupascu]

one last favor before i will resume waiting :D how do i convert the response part with binary data (or whatever it is ... the one i pasted on top) to base64? i cannot use normal javascript from what i see

i want to test this because i will need to show my clients some results and i cannot ask you when could this be ready to test ... :)

[wkh237]

React Native is running in a pure Javascript environment some APIs are not implemented, for example Web APIs, and the atob btoa you've mentioned. However you can use some polyfills, in my case I use base-64 to do this.

[rares-lupascu]

then i really do not know what to encode :(

• do i encode the content + Content-Type + Content-Length or i need to strip them out?
• if i do base64.encode(unescape(encodeURIComponent(x[x.length - 1]))) i get a base64 string but if i attach it to data:image/jpeg;base64,encoded_text the image i get is broken
• if i do base64.encode only ... it tells me the string contains characters outside Latin1

[wkh237]

A multipart response just structured like this

# HTTP response status Code
HTTP/1.1 200 OK 
# HTTP response headers
X-Powered-By: Express  
Content-Type: multipart/x-mixed-replace; boundary="---osclivepreview---";
Date: Fri, 30 Sep 2016 01:10:38 GMT
Connection: keep-alive
Transfer-Encoding: chunked

# HTTP response body
# this is multipart boundary
---osclivepreview--- 
# the below is part data headers end with \r\n
Content-Type: image/jpeg
Content-Length: 19153

# part data will append after an empty line
8c65

����JFIF��Lavc57.24.103��C

# lot of binary data

# part data end with another boundary
---osclivepreview---

...

Basically the image data is inside part body, so you have to stripe out boundary and part headers. But the part data you pasted above have been encoded into UTF8 string and it's broken, even if you encode it into BASE64 string it's still broken 😞

[wkh237]

I'm able to make it working on iPhone6 right now 🎉

keep work on Android part ..

[rares-lupascu]

nice .... :) from what i understand u need to utf8.decode it first? if it helps i can console.log some frames from my one shot camera so you can try out any combination to see what gets the image data out :)

there is one thing that puzzles me ... it is in this link: https://jeff.mcfadden.io/articles/handling-motion-jpeg-streams-on-ios/

they say u receive chunks of data ... that do not necessarily mean a frame ... so u can receive parts of a frame and might need to listen to multiple chunks before u actually get the full frame

[wkh237]

The IOS part is done, because IOS SDK has very good support for x-mixed-replace data. I'm now struggling with Android, looks like OkHttp does not support this feature, perhaps I have to do it on my own 💦

[rares-lupascu]

maybe this helps https://supportforums.blackberry.com/t5/Java-Development/I-have-a-question-about-httpConnection-and-InputStream-on/m-p/648315#M133290

https://supportforums.blackberry.com/t5/Java-Development/How-to-read-a-continous-multipart-data-stream-over-network-in/td-p/640991

[rares-lupascu]

for Android maybe this can help https://github.com/ArseniKavalchuk/multipart-x-mixed-replace-java-player

[rares-lupascu]

hey @wkh237 any news? can i help in any way? :)

[wkh237]

@wowzaaa , I'm little busy these days, still working on Android parser ..

[rares-lupascu]

@wkh237 got it ... take care

[rares-lupascu]

hey ... do not want to be rude or anything ... should i consider this lead dead and move on? :)

[wkh237]

@wowzaaa , sorry for replying late. I still can't find a solid solution on Android, and I'm quite busy these days. However I'll keep work on it once I have time.

[rares-lupascu]

hey ... is there anything i can do? should i learn android and try to do it? i would prefer for this to be part of ur library and not make a new one

[wkh237]

@wowzaaa , I've tried to implement a body parser on Android but the performance has very big problem still trying to find a better solution.

On the other hand, I'm quite busy these days, as such I can just fix those bugs which take less time. If you can make a PR that would be grateful ! 👍

[rares-lupascu]

man i really tried making it work but i lack a lot of android knowledge to solve it now

so u r my only hope right now :(

[rares-lupascu]

i recently found this ... i really think this can help us do it

http://stackoverflow.com/questions/3205191/android-and-mjpeg/3305276#3305276

[zzq1023]

@wowzaaa How do I show this picture stream in react-native project?