castle-engine / castle-engine

Cross-platform (desktop, mobile, console) 3D and 2D game engine. Powerful visual editor. Support for glTF, X3D, Spine and more. Fast clean code using modern Pascal. Free and open-source.
https://castle-engine.io/
Other
973 stars 128 forks source link

Cannot type in TCastleEdit on Android #399

Closed AndreMikulec closed 1 year ago

AndreMikulec commented 2 years ago

Hi,

I create a directory called castle-engine_examples_component_gallery. I made it a local Git repository.

I copied the contents of castle-engine\examples\component_gallery into castle-engine_examples_component_gallery.

I created a Github repository called castle-engine_examples_component_gallery.

I uploaded the directory contents into Github. https://github.com/AndreMikulec/castle-engine_examples_component_gallery

Using the build.yml and Github Actions, I built the Android file cge-component-gallery-0.1-android-debug.apk.

I downloaded the Android artifict https://github.com/AndreMikulec/castle-engine_examples_component_gallery/suites/7473736524/artifacts/306370501 into BlueStacks 5 and into my Amazon Fire HD 10 (2021) [still running Android 9]

On Android, in the Component Demo option, Edit(TCastleEdit, TCastleIntegerEdit, TCastleFloatEdit), I can not type anything into the Edit boxes. My typing does not work in neither my BlueStacks 5 nor my Amazon Fire HD 10 (2021). (All other Demos works correct.)

On Windows, I can type into Edit(TCastleEdi, TCastleIntegerEdit, TCastleFloatEdit).

On Android, I tried to type into Edit(TCastleEdit, TCastleIntegerEdit, TCastleFloatEdit). However, I could not.

What should I do next, to be able to type into Android TCastleEdit boxes?

Attached is the file cge-component-gallery-0.1-android-debug.apk in the .zip file.

Thanks, Andre Mikulec android-build (2).zip

michaliskambi commented 2 years ago

Thank you for the report!

You have encountered a known missing feature in Castle Game Engine. Indeed TCastleEdit and friends do not yet work on mobile (Android and iOS). They do not show and handle "on-screen keyboard" necessary for input on mobile.

We actually have 2 missing features related to it. You can do some manual work to workaround most of them in a particular application... but one bit (AD 2.B below) really requires a fix on CGE side.

If you're curious, I describe the issues below (and the current workarounds). But basically I'll try to handle it next week, at the vey least the "Components Gallery" should handle the edit boxes out-of-the-box (ideally, without any temporary hacks :) ). Unless someone else wants to tackle this, following my link in AD 2.B below, let me know :)

Missing features:

  1. Edit boxes don't immediately get focus

  2. Edit boxes don't show and handle the "on-screen keyboard" on mobile, even when they get focus.

For now there are 2 workarounds, basically you need to do some things manually:

AD 1 - To make the desired edit box have focus, set Container.ForceCaptureInput to the desired TCastleEdit instance.

Make sure to later set it back to nil when the edit box is not longer visible, e.g. the state showing it stops. You can also (you have to do it manually for now) handle keys like Tab, Shift+Tab to allow user to switch Container.ForceCaptureInput to the next/previous control that can receive key input.

Probably the simplest example of doing it now is https://github.com/castle-engine/not-quake example, see how https://github.com/castle-engine/not-quake/blob/master/client/code/gamestateinputchat.pas does StateContainer.ForceCaptureInput := EditChatMessage; at start and StateContainer.ForceCaptureInput := nil; at stop.

AD 2 - To show the on-screen keyboard when edit box is focused,

A. set AutoOnScreenKeyboard on TCastleEdit to true. It is not true by default, because merely setting it doesn't yet make the edit work -- you will see the on-screen keyboard, but typing there will not do anything.

B. the most important missing bit -- we right now have no way to actually receive the edit input.

We should implement it on Java side (likely in tools/build-tool/data/android/integrated/app/src/main/java/net/sourceforge/castleengine/ServiceMiscellaneous.java, https://github.com/castle-engine/castle-engine/blob/master/tools/build-tool/data/android/integrated/app/src/main/java/net/sourceforge/castleengine/ServiceMiscellaneous.java , that handles InputMethodManager now).

We should follow https://developer.android.com/training/keyboard-input and in particular https://developer.android.com/training/keyboard-input/style#java -- we need to react to user pressing e.g. "Send" by generating a message like "on-screen-keyboard-send" with the input contents. TCastleEdit, if it is focused now, should handle this message and effectively set its own Text to the new content.

AndreMikulec commented 2 years ago

My Game Player needs to "type in" his/her name to keep track of who has the High Score.

AD 1

Yes.

handle keys like Tab, Shift+Tab to allow user to 
switch Container.ForceCaptureInput to the next/previous control that can receive key input.

Yes. Also, I was from the Microsoft Visual Basic 6 world.
I remember the TabIndex property.

Probably the simplest example of doing it now is 
https://github.com/castle-engine/not-quake 
example, 
see how https://github.com/castle-engine/not-quake/blob/master/client/code/gamestateinputchat.pas 
does 
StateContainer.ForceCaptureInput := EditChatMessage; at start 
and 
StateContainer.ForceCaptureInput := nil; at stop.

O.K. I will take a look.

A. set AutoOnScreenKeyboard on TCastleEdit to true. 
It is not true by default, 
because merely setting it doesn't yet make the edit work 
-- you will see the on-screen keyboard, but typing there will not do anything.

O.K. I see in the Castle Editor

[ ] AutoOnScreenKeyboard [ ] (False)

Also, I noticed . . .

castlecontrols_edit.inc

procedure TCastleEdit.SetFocused(const Value: boolean);
begin
  inherited;

  if FAutoOnScreenKeyboard then
    Messaging.Send(['change-keyboard-state', TMessaging.BoolToStr(Value)])
end;
. . .
we right now have no way to actually receive the edit input.
. . .
Unless someone else wants to tackle this, following my link in AD 2.B below, let me know :)

Most of this coding of "Pascal bridge"\Java\Android is [currently] beyond my immediate skills. (Hopefully, in the near future, I can improve.) However, I think can contribute immediately by searching/reading the Web.
Maybe I will find something useful.

AndreMikulec commented 2 years ago

This is what I have so far.

I think that the gist of the idea is to do:

  1. Replace Button by EditText
  2. Set its(EditText) parent properties: TextView_inputType = text
  3. Set its parent properties: TextView_imeOptions = actionSend
  4. Replace setOnClickListener by setOnEditorActionListener - see BELOW - the middle example class
  5. sendMessage() - see the furthest example at the bottom - see BELOW - the very end

https://developer.android.com/reference/android/widget/EditText (I HOPE THIS EXISTS) import android.widget.EditText;

IME_ACTION_SEND
Added in API level 3

public static final int IME_ACTION_SEND
Bits of IME_MASK_ACTION: the action key performs a "send" operation, 
delivering the text to its target. 
This is typically used when composing a message in IM or SMS where sending is immediate.

Constant Value: 4 (0x00000004)

https://developer.android.com/reference/android/view/inputmethod/EditorInfo#IME_ACTION_SEND

Must be one or more (separated by '|') of the following constant values.

Constant    Value   Description
actionSend  4     The action key performs a "send" operation, delivering the text to its target. 
                  This is typically used when composing a message. 
                  Corresponds to EditorInfo.IME_ACTION_SEND.

https://developer.android.com/reference/android/widget/TextView#attr_android:imeOptions

TextView.BufferType EDITABLE https://developer.android.com/reference/android/widget/TextView.BufferType.html

EditText extends TextView; TextView.BufferType.EDITABLE How To Set Text In An EditText https://stackoverflow.com/questions/4590957/how-to-set-text-in-an-edittext

ImeOptions(
    singleLine: Boolean,
    capitalization: KeyboardCapitalization,
    autoCorrect: Boolean,
    keyboardType: KeyboardType,
    imeAction: ImeAction
)

https://developer.android.com/reference/kotlin/androidx/compose/ui/text/input/ImeOptions

What is Android's icicle parameter? SHORT ANSWER: JUST "USE" IT https://stackoverflow.com/questions/919153/what-is-androids-icicle-parameter

public class MyActivity extends Activity {
     protected void onCreate(Bundle icicle) {
         super.onCreate(icicle);

         final Button button = new Button(this);
         button.setText("Press me!");
         setContentView(button);

         button.setOnClickListener(new View.OnClickListener() {
             public void onClick(View v) {
                 // Perform action on click
             }
         });
     }
 }

How to create GUI in Android instead of using XML? https://stackoverflow.com/questions/4801631/how-to-create-gui-in-android-instead-of-using-xml

EditText myEditText = new EditText(context); // Pass it an Activity or Context

Generating Edit Text Programmatically in Android https://stackoverflow.com/questions/12985825/generating-edit-text-programmatically-in-android

If you look at the XML attributes in the docs, 
it lists the corresponding method you can call in your java code for each attribute

Setting attributes of an EditText added dynamically in Android https://stackoverflow.com/questions/4205775/setting-attributes-of-an-edittext-added-dynamically-in-android

TextViewMultiLineBackgroundState_state_multiline
public static final int TextViewMultiLineBackgroundState_state_multiline

State identifier indicating a TextView has a multi-line layout.

May be a boolean value, such as "true" or "false".

Constant Value: 0 (0x00000000)
TextView_imeOptions
public static final int TextView_imeOptions

Additional features you can enable in an IME associated with an editor 
to improve the integration with your application. 
The constants here correspond to those defined by EditorInfo.imeOptions.

Must be one or more (separated by '|') of the following constant values.

Constant    Value   Description
actionSend  4       The action key performs a "send" operation, delivering the text to its target. 
                    This is typically used when composing a message. 
                    Corresponds to EditorInfo.IME_ACTION_SEND.
TextView_inputType

public static final int TextView_inputType

The type of data being placed in a text field, 
used to help an input method decide how to let the user enter text. 
The constants here correspond to those defined by InputType. 
Generally you can select a single value, 
though some can be combined together as indicated. 
Setting this attribute to anything besides none also implies that the text is editable.

Must be one or more (separated by '|') of the following constant values.
Constant      Value Description

number        2     A numeric only field. Corresponds to 
                    InputType.TYPE_CLASS_NUMBER | InputType.TYPE_NUMBER_VARIATION_NORMAL.

numberDecimal   2002    Can be combined with number and its other options to allow a decimal 
                    (fractional) number. 
                    Corresponds to InputType.TYPE_CLASS_NUMBER | InputType.TYPE_NUMBER_FLAG_DECIMAL.

text          1      Just plain old text. 
                     Corresponds to InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_NORMAL.

textImeMultiLine 40001 Can be combined with text and its variations 
                       to indicate that though the regular text view 
                       should not be multiple lines, the IME should provide multiple lines if it can. 
                       Corresponds to InputType.TYPE_TEXT_FLAG_IME_MULTI_LINE.

textLongMessage  51   Text that is the content of a long message. 
                      Corresponds to InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_LONG_MESSAGE.

textMultiLine   20001  Can be combined with text and its variations to allow multiple lines of text 
                     in the field. 
                     If this flag is not set, the text field will be constrained to a single line. 
                     Corresponds to InputType.TYPE_TEXT_FLAG_MULTI_LINE. 
                     Note: If this flag is not set and the text field doesn't have max length limit, 
                     the framework automatically set maximum length of the characters to 5000 
                     for the performance reasons.

textPassword     81  Text that is a password. 
                     Corresponds to InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_PASSWORD.

textVisiblePassword 91  Text that is a password that should be visible. 
                        Corresponds to InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_VISIBLE_PASSWORD.
TextView_singleLine
public static final int TextView_singleLine

Constrains the text to a single horizontally scrolling line 
instead of letting it wrap onto multiple lines, 
and advances focus instead of inserting a newline when you press the enter key. 

The default value is false (multi-line wrapped text mode) for non-editable text, 
but 
if you specify any value for inputType, the default is true (single-line input field mode).

May be a boolean value, such as "true" or "false".

Constant Value: 32 (0x00000020)

TextView https://developer.android.com/reference/android/R.styleable?hl=zh-cn#TextView HAD BEEN REDIRECTED FROM TextView Attributes FROM EditText https://developer.android.com/reference/android/widget/EditText.html?hl=zh-CN

Specify the input method action

To specify the keyboard action button, 
use the android:imeOptions attribute with an action value such as "actionSend"

You can then listen for presses on the action button

<EditText
    android:id="@+id/search"
    android:layout_width="fill_parent"
    android:layout_height="wrap_content"
    android:hint="@string/search_hint"
    android:inputType="text"
    android:imeOptions="actionSend" />

listen for presses on the action button by defining a TextView.OnEditorActionListener for the EditText element

EditText editText = (EditText) findViewById(R.id.search);
editText.setOnEditorActionListener(new OnEditorActionListener() {
    @Override
    public boolean onEditorAction(TextView v, int actionId, KeyEvent event) {
        boolean handled = false;
        if (actionId == EditorInfo.IME_ACTION_SEND) {
            sendMessage();
            handled = true;
        }
        return handled;
    }
});

https://developer.android.com/training/keyboard-input/style#java

AndreMikulec commented 2 years ago

I also found this information.

. . . want an action to occur once the user has finished typing text with the Soft Keyboard.

<EditText
   android:inputType="text"
   android:singleLine="true"
   android:imeOptions="actionDone"
   android:id="@+id/etValue"
   android:layout_width="wrap_content"
   android:layout_height="wrap_content">
   <requestFocus />
</EditText>

In particular, singleLine and imeOptions are required for the Done button to display. Now, we can hook into a editor listener for when the done button is pressed with:

etValue.setOnEditorActionListener(new OnEditorActionListener() {
    @Override
    public boolean onEditorAction(TextView v, int actionId, KeyEvent event) {
        if (actionId == EditorInfo.IME_ACTION_DONE) {
                        String text = v.getText().toString();
            Toast.makeText(MainActivity.this, text, Toast.LENGTH_SHORT).show();
            return true;
        }
        return false;
    }
})

. . .

etValue.setOnEditorActionListener(OnEditorActionListener { v, actionId, event ->
  if (actionId == EditorInfo.IME_ACTION_DONE) {
    val text = v.text.toString()
    Toast.makeText(this@MainActivity, text, Toast.LENGTH_SHORT).show()
    return@OnEditorActionListener true
  }
  false
})

This is often great whenever a user needs to type text and then explicitly have an action performed when they are finished. There are many imeOptions for different situations.

EditText Common Listeners https://guides.codepath.com/android/Basic-Event-Listeners#edittext-common-listeners

AndreMikulec commented 2 years ago
IME_ACTION_SEND
Bits of IME_MASK_ACTION: the action key performs a "send" operation, 
delivering the text to its target.

https://developer.android.com/reference/android/view/inputmethod/EditorInfo

or in Java:

myEditText.setImeOptions(EditorInfo.IME_ACTION_DONE);
myEditText.setImeOptions(EditorInfo.IME_ACTION_DONE)

Working with the Soft Keyboard https://guides.codepath.com/android/Working-with-the-Soft-Keyboard Google search "android soft keyboard service send data"

AndreMikulec commented 2 years ago

This is completely different. I have not yet have time to investigate.

I have now run into the problem that 
I can't update the text field where the input from the keyboard should go. 
I tried creating a static InputConnection, 
in my service class, then updating it from the Activity, but nothing happens.

So I guess here is my question: I was able to find lots of info about 
how to send data from a Service to an Activity, 
but nothing about sending data from an Activity 
to a Service(specifically an input method service). 
Does anyone know how to do this?

Answer

Use a Binder for that.

This tutorial about music player service 
is a good sample http://www.helloandroid.com/tutorials/musicdroid-audio-player-part-ii

Especially this line

mpInterface.addSongPlaylist(file.getName());

Comment: can we use broadcast to send message from activity to service as well??

Send message from Activity to Service - Android https://stackoverflow.com/questions/8169749/send-message-from-activity-to-service-android Google search "android soft keyboard service send data"

michaliskambi commented 2 years ago

Thank you for the research.

I think part of it will be useful, part not so much (because our TCastleEdit is a not "mapped" to native Android UI control, in general our Castle Game Engine is a set of custom-drawn OpenGL controls -- so you can style them perfectly to fit the "mood" of your game, but it also means we need to do a bit of extra work to integrate with necessarily-native functionality like on-screen keyboard on Android and iOS).

I'll try to tackle it properly this week, stay tuned :)

AndreMikulec commented 2 years ago

I'll try to tackle it properly this week, stay tuned :)

O.K. This is my last post for a while.

I looked over the source code in many mobile game engines in here.

The best 22 mobile game engines and development platforms https://appradar.com/blog/mobile-game-engines-development-platforms

I only looked at the code of the open source mobile game engines. (I obviously can not look at the code of the proprietary mobile game engines.) Many "many" open source game mobile engines have no 'editbox with editable input ability' at all. True.

Only two open source mobile game engines seem to have an 'editbox with editable input ability'

Engines that have a 'editbox with editable input ability'

1. This GDevelop (1.) probably just sends its code to Cordova. So (1.) probably will not work in the Castle Game Engine.

javascript https://github.com/4ian/GDevelop/blob/master/Extensions/TextInput/JsExtension.js npm, electron https://github.com/4ian/GDevelop/blob/master/appveyor.yml Cordova, Electron https://github.com/4ian/GDevelop/tree/master/GDJS/Runtime

2. This Cocos (2.) looks very very daunting. If somone can figure this out (2.), this may work. This is probably too much work.

javascript, typescript https://docs.cocos.com/creator/3.5/api/en/class/EditBox https://docs.cocos.com/creator/3.5/api/en/class/EditBox?id=EditBox https://github.com/cocos/cocos-engine/blob/6a29cb9fdb558a4ef99fe8aa7de47448527b0d7a/platforms/minigame/common/engine/Editbox.js https://github.com/cocos/cocos-engine/blob/6a29cb9fdb558a4ef99fe8aa7de47448527b0d7a/platforms/runtime/common/engine/EditBox.js https://github.com/cocos/cocos-engine/blob/6a29cb9fdb558a4ef99fe8aa7de47448527b0d7a/native/cocos/platform/ohos/libcocos/src/main/java/com/cocos/lib/CocosEditBoxAbility.java https://github.com/cocos/cocos-engine/blob/00fca7a77cde1ec21504a4a500c99fa09aefeb19/native/cocos/ui/edit-box/EditBox.h https://github.com/cocos/cocos-engine/blob/fbd92ab01ffc4a40c42bb5342872ee0443432284/platforms/minigame/platforms/xiaomi/wrapper/engine/Editbox.js

if CC_USE_EDITBOX

https://github.com/cocos/cocos-engine/blob/00fca7a77cde1ec21504a4a500c99fa09aefeb19/native/cocos/bindings/manual/jsb_global.cpp https://docs.cocos.com/creator/api/en/class/EditBox?id=_editBoxEditingDidBegan

Pre-Feature Request - about Touch

Possibly, this "issue" could be transformed into a feature request. My feature request follows:

The situation may be "easier" just to write it all in Pascal.

What I mean is the following: see 1, 2, 3, 4, 5.

Note, I originally started 2,3,4, and 5 with "On a mobile platform, . . ." but I omitted these phrases. The reasons are the following:

On some Windows 10 computers, some laptops specifically, touchscreens exist. Also, external touchscreen monitors that "can and do" attach to Windows 10 computers also exist. (I own two monitors. My second monitor is this:
beetronics 10 inch touchscreen https://www.beetronics.com/10-inch-touchscreen?gclid=Cj0KCQjwof6WBhD4ARIsAOi65aiBLsI2IU8UD5zY1bCAQV3S-QqE7OVHYJDPR5V3w_YoQUgVSYNpAdkaAqb_EALw_wcB )

Feature Request

  1. Probably a "tabIndex" or "tabOrder" property or something similar is needed. Expanding on what you said:
    
    AD 1 - To make the desired edit box have focus, 
    set Container.ForceCaptureInput to the desired TCastleEdit instance.

Make sure to later set it back to nil when the edit box is not longer visible, e.g. the state showing it stops. You can also (you have to do it manually for now) handle keys like Tab, Shift+Tab to allow user to switch Container.
ForceCaptureInput to the next/previous control that can receive key input.



Pre 2.-5.

On Android, I just attach a keyboard through hardware of a 
USB2Go (OTG) device or through software of Bluetooth.
For 2,3,4,5 I just refer to "attached" as either USB2Go or Bluetooth.

2.
If a person has an attached keyboard 
then just detect keyboard events (or just always detect all keyboard events),
then make the behaviour like a desktop: A person can press `tab` to move from
one focusable object (e.g. 'non-disabled editbox with editable input ability') to another.

3.
If a person has an attached mouse 
(In Pascal code, one must detect the presence of a mouse)
[by same hardware that attaches a keyboard], and the person hovers
the mouse over the 'non-disabled editbox with editable input ability', 
then change the cursor to an I-bar cursor image or some other cursor image.
(The behaviour is like a desktop.)

4.
If a person has an attached mouse and an attached keyboard
(In Pascal code, one must detect a left-mouse-click vs a 1-point-touch)
and mouse-clicks down into the 'non-disabled editbox with editable input ability'
then in that editbox, a blinking bar  or blinking cursor 
or something blinking appears.
(The behaviour is like a desktop)

5.
If a person "does not have an attached mouse and 
does not have an attached keyboard (both must be detected)"
OR the person, for any reason, 1-point-touches down into 
the 'non-disabled editbox with editable input ability', 
then a *Pascal-written keyboard appears*.
(The behaviour is like a[n Android] mobile device.)

### Reasons for the *Pascal-written keyboard*

1. Keep control.  One only needs that "java" to communicate back and forth to Big Brother Google.
   In this issue/feature-request - No communication is needed to Big Brother Google.   
2. Developers and contributors know Pascal
3. Keep control of the Both inputs on the Canvas Pascal 'form' and  Canvas Pascal 'keyboard form'
4. Avoid current problems and "feature creep" problems from Big Brothers, Google and Apple. 

   Apple 1984 Super Bowl Commercial Introducing Macintosh Computer (HD)
   https://www.youtube.com/watch?v=2zfqw8nhUwA

5. One code base(Pascal) and not [more of] three(Java, C?, C++?, Objective-C, whatever?).

### Thanks

That was just my 2 cents.

My two cents
`American idiomatic expression`
`a very small amount . . . the user of the phrase, showing politeness and humility`
https://en.wikipedia.org/wiki/My_two_cents
michaliskambi commented 2 years ago

As for using hardware keyboard connected to a mobile device (by cable or Bluetooth) -- sure, this is also something we should handle, and then the keyboard on mobile can work similar as on desktop. This one should be easy.

As for writing our own on-screen keyboard in Pascal: I wondered about this, but I think

  1. it would be a big task, much bigger than integrating with system on-screen keyboard. E.g. because of localization (imagine Chinese, Greek, cyryllic-using languages), autocomplete (with dictionaries for languages, smart prediction), extra input methods (gestures to type with swipe on various keyboards, not to mention input by dictating to microphone). We don't want to write this all on CGE side in Pascal :)

  2. And in the end some users would just want to use the on-screen keyboard native to their phone anyway (or installed on their phone, on both Android and iOS you can install custom on-screen keyboard applications, like GBoard on iOS or Hacker's Keyboard on Android. and use them through all applications). Even if we would manage to implement fully-functional on-screen keyboard in Pascal, people could still miss the familiarity of their default on-screen keyboard.

It's tempting of course to implement our own on-screen keyboard, for reasons you mention. We would have it available on all systems (at least Android and iOS) in a uniform fashion, we would not need to care about integration with system on-screen keyboard in Java (Android) or Objective-C (iOS). But in exchange, we would add a big task and it would take us months before we have a fully-functional keyboard in Pascal.

AndreMikulec commented 2 years ago

You said:

We should follow https://developer.android.com/training/keyboard-input 

Actually, these are not the immediate the problem.

The first three categories:

1. Specify the input method type
2. Handle input method visibility
3. Support keyboard navigation

Of the last category (4.), this is not this problem. (4.) is a setting up a custom "non-default" Android behavior. (I am not interested in this).

4. Handle keyboard actions

The default Android behavior is to wait until the user is finished typing his/her many words, then presses the "action" key.

This Stackoverflow Q&A goes over many ways to grab a key. . .

Android - Get keyboard key press
https://stackoverflow.com/questions/11273243/android-get-keyboard-key-press

Simon Choi (fame - Pull Shark - https://github.com/tr3esoftware)) of LAMW: Lazarus Android Module Wizard https://github.com/jmpessoa/lazandroidmodulewizard did this Get keyboard key press (at 71 seconds in). Free Pascal 2.7.1 and Android Native Control Simon Choi https://youtu.be/RqVQOx6mZIQ?t=73

You said:

and in particular https://developer.android.com/training/keyboard-input/style#java 
-- we need to 
react to user pressing e.g. "Send" by generating a message like "on-screen-keyboard-send" 
with the input contents. TCastleEdit, if it is focused now, 
should handle this message and effectively set its own Text to the new content.

In particualar:

"Send" by generating a message like "on-screen-keyboard-send" with the input contents. TCastleEdit

like. It can not be done. See below.

(because our TCastleEdit is a not "mapped" to native Android UI control, 
in general our Castle Game Engine is a set of custom-drawn OpenGL controls 
-- so you can style them perfectly to fit the "mood" of your game, 
but it also means we need to do a bit of extra work 
to integrate with necessarily-native functionality like on-screen keyboard on Android and iOS).

TCastleEdit is a not "mapped" to native Android UI control Not exactly.

TCastleEdit needs to "be exactly" the native Android UI control EditText (or one of its subclasses: AutoCompleteTextView, ExtractEditText, or MultiAutoCompleteTextView) (and have Android focus).

Here is some information:

Android - Get keyboard key press https://stackoverflow.com/questions/11273243/android-get-keyboard-key-press

I want to catch the press of any key of the softkeyboard. I don't want a EditView
. . .
Why it's so complicate take a simple key event from a keyboard ?
. . .
asked Jun 30, 2012 at 10:06
Bemipefe
. . .
Accepted Solution:

UPD: unfortunately you can't handle soft keyboard events [and not use EditView]
(see Handle single key events), 
unless you develop your own custom keyboard 
(follow the link to learn how Creating input method).

Create an input method https://developer.android.com/guide/topics/text/creating-input-method.html

this class has callbacks for providing 
your IME's UI [means Android SoftKeyboard], 
handling user input, and delivering text to the field that currently has focus.

Applications should extend this class [InputConnection]
rather than implementing the base interface *InputConnection*.

view/inputmethod/BaseInputConnection https://developer.android.com/reference/android/view/inputmethod/BaseInputConnection

Public constructors
BaseInputConnection(View targetView, boolean fullEditor)
  . . .
  https://developer.android.com/reference/android/view/View
    . . . 
    Known direct subclasses - TextView
      TextView
      https://developer.android.com/reference/android/widget/TextView
        . . .
        Known direct subclasses - EditText
          EditText
          https://developer.android.com/reference/android/widget/EditText

Here is the final article. The gist is that the Android SoftKeyboard needs to know: android.view.inputmethod.EditorInfo structure.

On-screen Input Methods 21 April 2009 https://android-developers.googleblog.com/2009/04/updating-applications-for-on-screen.html

View APIs, the most important of these being onCreateInputConnection() 
which creates a new InputConnection for an IME 
(and fills in an android.view.inputmethod.EditorInfo structure with your 
input type, IME options, and other data)

Looking at Pascal /w Native Android Controls - ver 0.17 Native Android Controls for Pascal Simon Choi http://andcontrol.blogspot.com/2013/08/pascal-w-native-android-controls-ver-017.html

Simon Choi, had to create the Native Android Control "jEdit" in Pascal and use "jni" calls to talk to Java. This is the immediate problem.

We should follow https://developer.android.com/training/keyboard-input These non-immediate problems, may seem easy, after the the immediate problem is solved.

I do not think that I am smarter than Simon Choi (or as patient as Simon Choi).

I think that I will "dream" of an experimental Pascal-only [Android like Soft]Keyboard.

Maybe I will call the keyboard "FluffyKeyboard".

michaliskambi commented 2 years ago

Thanks for the analysis and links.

At this point I just need to find time and start experimenting and then see the optimal approach to CGE -- then we can enjoy a hopefully good solution and see where it puts us:)

I'm not ruling out any solution at this point, except implementing our own keyboard from scratch in Pascal (as I think it would be a big work to make it full-featured for users, so it's better to integrate with native Android/iOS on-screen keyboards in whatever way is possible).

Sorry for not having time to do it this week, even though I promised it. I still have it high on my TODO list, we want our examples and TCastleEdit to "just work" on mobile of course:)

and3md commented 1 year ago

@AndreMikulec Hi, I've been experimenting with android software keyboard support in CastleEdit for a few days, currently sending keycodes to the engine window works (WIP):

https://user-images.githubusercontent.com/18555708/203281214-aefcef22-6a5d-4bbf-b15d-265699ad6c62.mp4

Implemented as service named keyboard.

Work in progress so still few things to do: