search

dryark / stf_ios_support

Central repo to connect and document components/repos needed for IOS stf support
Other
153 stars 65 forks source link

Video rotation is not handled #120

Open KostyaSha opened 1 year ago

KostyaSha commented 1 year ago

Just in case somebody already solved and can share while i'm debugging this thing. When ios runs it has 0' orientation in portrait mode, as soon as it runs landscape app image in browser is rotated and looks correct, but clicks are messed. Also ui doesn't highlight landscape mode and info->orientation is not changed. That happens because of missing device.change -> rotation event. As soon as it added rotate button highlights and work fine, but image is double rotated and still miss-clicked. I guess there is nuance on how at all images should be streamed into browser with websocket. avfoundation changes width and height according to layout, i guess android send always the same size, but with the note of rotation. As experiment i enforced ivf_pull to produce sticky image layout (rotated with 270' when width > height and swapped width-height values). But clicks are still 90' degree wrong.

I also don't see any updates from backend into UI about scale. @nanoscopic do you remember in what state did you end this part? Did it work ever? What is clickscale? Maybe you can share some knowledge on how STF web processes this part?

nanoscopic commented 1 year ago

This was not coded to be handled. It is TODO.

Handling for it needs to be written in.

Clickscale, to my memory, is the scaling of coordinates for clicking, versus the scale to turn the raw video into coordinates for on screen display. I agree "clickscale" is really confusing.