Option to present ReDroid as connected via USB (to make IME usable with scrcpy)

[schrmh]

Is your feature request related to a problem? Please describe. The usual way to connect to a ReDroid container is via scrcpy. But it seems like there is no way currently to send characters from a IME over tcpip to ReDroid; for passing through input from a IME, --hid-keyboard can be passed to scrcpy and then it makes use of USB HID over AOAv2 which requires a USB connection as it seems.

Describe the solution you'd like ReDroid could have a parameter that lets it emulate a USB connected Android device.

Describe alternatives you've considered

• I opened an issue at https://github.com/Genymobile/scrcpy/issues/3913 but I feel like it might be out of scope for them.
• I'm open for suggestions from other people here (especially since a lot of people speaking asian languages use ReDroid... maybe I'm just missing something that could easily be used?)
• "Fcitx5 for Android" but it is not ready yet (I use layouts that don't exist there)

Additional context fcitx5 is used on my Arch Linux machine the ReDroid container runs on.

[zhouziyang]

For non-ASCII character input, you may need install an IME in redroid container.

redroid is designed to running on server side (can boot many android instances), and scrcpy is just handy tool. Typically, users should implement their own streaming solutions (WebRTC etc.).

[schrmh]

For non-ASCII character input, you may need install an IME in redroid container.

Yeah, the problem is that the best thing here that I know about is the "Fcitx5 for Android" app which allows to switch IME's fast (e.g. on ctrl+shift key press) but well, this is what it currently supports:

So basically only chinese IME's (but even missing e.g. RIME) + english. And well, on my computer* I not only use fcitx5 engines but I make use of fcitx5 modules as well (im-emoji-picker for fast emoji input), which the Android app doesn't seem to support at all.

Also that would still be completely separate from fcitx5 on my computer (so I could still only send (extended) ASCII from that to the device when using scrcpy), which is a slight bummer.

Typically, users should implement their own streaming solutions (WebRTC etc.).

Yeah, I guess things can be done outside of ReDroid (or scrcpy) in that case, but I lean towards that it's unlikely that something will show up for that case (and I'm likely not able to develop a solution I can live with within the next few years)...

The approach with the emulated USB device would be nice since that device could also potentially be shared over the network by using USB/IP.

(* my desktop computer on which ReDroid runs on as well but I might as connect from a separate phone running a mobile Linux distro in the future that has a keyboard attached...; biggest thing holding me back is cost of mobile data but necessary hardware is there)

[zhouziyang]

After install some 3rd party IME (Sogou chinese input), I can input non-ASCII characters via my hardware keyboard. scrcpy 2.0 used here.

BTW, You can share redroid instances (adb shell) with just IP / Port pair (possible with authentication enabled). No complicated USBIP involved.

[schrmh]

Yes, IME per se work (they receive ASCII over scrcpy in that case and suggest characters in some other script e.g. Chinese — which means my keyboard layout outside the container can't be in the same asian script).

However, I would have to use and find very different IME's to those I am used to. And that is only one part.
One other part is switching between them... does Android even allow that by using a simple shortcut when a physical keyboard is connected?
And even then, what I did not mention before, it might still be sup-optimal due to lacking grouping support of IME's/software keyboards.
For example, depending on who I write with, I switch between groups on my computer so that I don't have to cycle trough IME's for a language I don't need during that time:
DE/CN group (that I don't use that often at the moment): DE/JP group (I use Mozc for Hiragana and Kanji; Skk for Katakana and Anthy for full-width ASCII input): DE group (only containing a newer standard layout for my native language that includes some extra symbols; I use that when I write with people in german or english only)

All of this so far could be handled by the Android app "Fcitx5 for Android", which is what it says, but it is not ready yet (as you can see on the screenshot from the previous post which only lists chinese IME's and English keyboard).

Now, to be fair, I don't use DE/CN group that often so lacking grouping support might be okay by itself if we only looked at IME's... at least for a few year until I'm better in the Chinese language and want to use it more often.
But there is also the problem to find good keyboards that include emojis (and preferably kaomojis) and allow fast typing of them. For the regular fcitx5, there are addons and one of that is the im-emoji-picker engine: When I press this shortcut (Ctrl+Alt+.), a window opens in which I can simply type the name of any emojii (+ a few kaomojis):

So yeah, while it is possible to use several soft keyboard IME's on Android, I would need several and one of those is likely one for emojis only since most keyboards don't have a good emoji search. And compared to that, it is just immensely faster to switch keyboards to a wanted language and to type emoji on a GNU/Linux device using fcitx5 than on a Android device using several different keyboards and no kind of grouping.

And even if there was just the most amazing IME software ever for Android, it would still be a — really small compared to the previous mentioned stuff — bummer that I would need to switch to a non-asian layout on my computer every time I want to send something to scrcpy 2.0. For example, here when I have Mozc IME enabled and type a, which then generates Hiragana あ (u+3042) on my computer, I get "[server] WARN: Could not inject char u+3042" as output from scrcpy:

For the USB/IP part: That was just a mention of a technology that could be used if Redroid exposed the containers as USB connected Android devices. If that part could be easier done then that would be amazing.

[zhouziyang]

I think you cannot inject non-ASCII characters into redroid via scrcpy directly (only ASCII / some control keys can be injected, and processed by IME in redroid). You should find an amazing Android IME (possible react on some shortcut keys to change keyboard layout …) to suit your needs.

BTW, it must be approved by end user if want to change IME in android; It's behavior of the stock Andorid (well, feasible to customize).