New helper for adb service.

[UrielCh]

Hi,

I'm starting implementing services functions for a adbkit fork project. In the beginning, I just want to access the phone number of a phone using ADB.

but I want a clean implementation.

so I start to implement a service mapping code.

Maybe we can share some code (I'm using typescript, in my version).

Now I can list services, get the android version, the download the corrects aidl files from android git.

About the Parcel response, I extract all data as a buffer, convert big-endian to little-endian, and then read the response as UTF-16-LE (nodeJS do not support big-endian UTF-16).

TODO:

• Extract calls name + parameter from aidl files.
• Generate the helper code.
• Parse Parcel response.

Question: How can I get the android minor revision. (for now I will check if all minor version are the same)

I found some interesting code here

[T-vK]

android-svc is basically just a wrapper around Android's internal service command. service call by itself is pretty useless unless you want to manually dig through the aidl source code.

I'm not sure if we can actually share the same code because Bash and TypeScript are very different languages. I guess you could call android-svc from nodejs, but I'm not sure if that's what you're looking for. Feel free to copy / port anything you need from this project.

About getting the android minor version, check the GetAndroidVersion method: https://github.com/T-vK/android-svc/blob/0d25669f34252bcafba1462cb56bf185d0165542/android-svc-lib.sh#L220

I'm calling getprop ro.build.version.release to get the major version. If you just run getprop, you get a big list of all the properties. Maybe ro.build.expect.firmware would do the job, but I don't know.

About your TODO, take a look at:

• https://github.com/T-vK/android-svc/blob/0d25669f34252bcafba1462cb56bf185d0165542/android-svc-lib.sh#L174
• https://github.com/T-vK/android-svc/blob/0d25669f34252bcafba1462cb56bf185d0165542/android-svc-lib.sh#L49
• https://github.com/T-vK/android-svc/blob/0d25669f34252bcafba1462cb56bf185d0165542/android-svc-lib.sh#L315

[UrielCh]

thx for your point of view, I do not plan to integrate a bash script into a typescript, I prefer sharing some regexp.

First strange issue.

I'm trying to parse aidl file with a java parser.

ex: https://cs.android.com/android/platform/superproject/+/master:frameworks/base/core/java/android/hardware/display/IDisplayManager.aidl

But my parser fail to read this line: void registerCallback(in IDisplayManagerCallback callback); is this file is a java file? I had never seen an in keyword... that not a java file.... In your code in ConvertDataType() you support some strange type as in double, in long, 'in boolean' ...

so it's look like your parser is:

    echo "${l_serviceSource}" | sed -e '1,/interface/d' \ # crope top of the file
    -e '/^$/d' \  # drop empty line
    -e 's/^[[:space:]]*//' \  # merge multiple space
    -e '/^[^a-zA-Z]/d' \ # drop comment I would like to keep them
    -e '/^[^;]*$/{$!N}' \ # I do not understand this one
    -e '$d' \ # never seen a $d in a sed
    -e 's/\(^\|\n\)[[:space:]]*\|\([[:space:]]\)\{2,\}/\2/g' | \   # collapse mutiline declaration
    sed -e ':x;N;s/\([^;]\)\n/\1/;bx' | # do not need more filter in my implementation
    sed -e "s/ ,/, /g" | # do not need more filter in my implementation
    sed -E "s/,([^[:space:]])/, \1/g" # do not need more filter in my implementation

I will replace the code I do not understand by a replace ,\r\n by a , that may work like that.

[T-vK]

Yeah, Aidl is not Java, keep that in mind. Sharing the same regex is probably not possible if you use standard JS regex.

-e '/^[^;]*$/{$!N}' # remove leading spaces of methods that are split into multiple lines
-e '$d' # delete the last line
-e 's/\(^\|\n\)[[:space:]]*\|\([[:space:]]\)\{2,\}/\2/g' # ensures that *most* long method signatures don't go across multiple lines 
sed -e ':x;N;s/\([^;]\)\n/\1/;bx' # ensures that long method signatures don't go across multiple lines (see queryIntentActivityOptions in IPackageManager.aidl)
sed -e "s/ ,/, /g" # ensures there are no spaces before any commas
sed -E "s/,([^[:space:]])/, \1/g" # ensures there is a space behind every comma 

The regex is certainly not perfect and can be simplified quite a bit in TS I'm sure.

Try testing against IPackageManager.aidl: https://raw.githubusercontent.com/LineageOS/android_frameworks_base/7fc95f204527ee079c5891d56c969668f0b35a0b/core/java/android/content/pm/IPackageManager.aidl

I really would have loved to keep the comments, but sed makes this very difficult. Maybe someone comes up with a solution eventually.

Maybe you can try using the aidl that comes with the Android SDK:

$ ./aidl --help
usage:
./aidl --lang={java|cpp|ndk} [OPTION]... INPUT...
   Generate Java or C++ files for AIDL file(s).

./aidl --preprocess OUTPUT INPUT...
   Create an AIDL file having declarations of AIDL file(s).

./aidl --dumpapi --out=DIR INPUT...
   Dump API signature of AIDL file(s) to DIR.

./aidl --checkapi OLD_DIR NEW_DIR
   Checkes whether API dump NEW_DIR is backwards compatible extension 
   of the API dump OLD_DIR.

./aidl [OPTION]... INPUT [OUTPUT]
   Generate a Java file for an AIDL file.

OPTION:
  -I DIR, --include=DIR
          Use DIR as a search path for import statements.
  -m FILE, --import=FILE
          Import FILE directly without searching in the search paths.
  -p FILE, --preprocessed=FILE
          Include FILE which is created by --preprocess.
  -d FILE, --dep=FILE
          Generate dependency file as FILE. Don't use this when
          there are multiple input files. Use -a then.
  -o DIR, --out=DIR
          Use DIR as the base output directory for generated files.
  -h DIR, --header_out=DIR
          Generate C++ headers under DIR.
  -a
          Generate dependency file next to the output file with the
          name based on the input file.
  -b
          Trigger fail when trying to compile a parcelable.
  --ninja
          Generate dependency file in a format ninja understands.
  --structured
          Whether this interface is defined exclusively in AIDL.
          It is therefore a candidate for stabilization.
  --stability=<level>
          The stability requirement of this interface.
  -t, --trace
          Include tracing code for systrace. Note that if either
          the client or service code is not auto-generated by this
          tool, that part will not be traced.
  --transaction_names
          Generate transaction names.
  --apimapping
          Generates a mapping of declared aidl method signatures to
          the original line number. e.g.: 
              If line 39 of foo/bar/IFoo.aidl contains:              void doFoo(int bar, String baz);
              Then the result would be:
              foo.bar.Baz|doFoo|int,String,|void
              foo/bar/IFoo.aidl:39
  -v VER, --version=VER
          Set the version of the interface and parcelable to VER.
          VER must be an interger greater than 0.
  --hash=HASH
          Set the interface hash to HASH.
  --log
          Information about the transaction, e.g., method name, argument
          values, execution time, etc., is provided via callback.
  --parcelable-to-string
          Generates an implementation of toString() for Java parcelables,
          and ostream& operator << for C++ parcelables.
  --help
          Show this help.

INPUT:
  An AIDL file.

OUTPUT:
  Path to the generated Java or C++ source file. This is ignored when
  -o or --out is specified or the number of the input files are
  more than one.
  For Java, if omitted, Java source file is generated at the same
  place as the input AIDL file,

HEADER_DIR:
  Path to where C++ headers are generated.

Not sure where the source code for this tool is, but it might be this: https://android.googlesource.com/platform/system/tools/aidl/

[UrielCh]

Calling an external binary is a performance killer, and the aidl file looks not so complicated.

or maybe I can convert some aidl file, and store them in my package.

thx for the tips.

[T-vK]

I'm not sure how you're planning on calling the actual service methods. I guess you could try to wrap the service cli tool into a C++ addon. This would probably be the most efficient way, but also orders of magnitudes more complicated than just calling the binary.

I think the aidl cli tool is written in C++ as well.

But this is just a suggestion.

One more thing: In theory you could convert all the aidl files into Java files once after installing and from then on use your Java parser.

[UrielCh]

Hi,

Do you know how to list files from a git branch with an HTTP request so that I can list all aidl files?

for now, I have found the following URL:

• https://android.googlesource.com/platform/frameworks/base/+refs/tags/?format=text list all tags
• https://android.googlesource.com/platform/frameworks/base/+/${tag}/core/java/${pkg}.aidl?format=TEXT download a file as BASE64

I do not know if those URLs are standard git URLs or google git specific.

I found that this git view is hosted by a gitiles.

---

it looks like it is not possible: https://github.com/google/gitiles/blob/master/java/com/google/gitiles/GitilesView.java

I need to find a way to list aidl files in android gits ...

[T-vK]

I do it using the GitHub API. Fortunately Google is mirroring the repos to GitHub so you don't have to use googlesource.com.

[UrielCh]

New day, a new solution.

I will preload all existing aidl files with a bash script.

Can I fork and PR your project to add some doc? I like bash scripts, but they are hard to read once they became big without comment.

[UrielCh]

Tell me what you think about my firsts comments suggestion #5

I do not understand the purpose of AndroidShell if g_shellType is not set to adb.

It's look's like you attend to use the code from the phone itself, but is the su mandatory? If I call your script from a PC, I must add the --adb options all the time.

Maybe there is an easy way to avoid the mandatory --adb parameter if the shell looks not to be run in an android device.

[T-vK]

Of course you can add comments, that would be great. :) I'll take a look at your PR and make some comments there.

AndroidShell is an abstraction layer that allows me to share the same functions for usage on Andorid directly via Termux and for usage via adb. When running on Termux directly on the device su is necessary, when running remotely adb is necessary.

I use android-svc almost exclusively on the device directly, thus I made that the default. You can always define an alias or a wrapper script to make the --adb option the default.

But I would advice to always use --adb=<device-id> over --adb, so that you can't accidentally destroy another adb-enabled device on the network or accidentally mess with your emulator.

Edit: Another option I might add in the future would be ssh.

[UrielCh]

Maybe calling service once can be used to determine if you need to call su or adb.

When I launched it for the first time, it was strange to see a su call on my Linux host.

[T-vK]

But service might be something else entirely on a Linux system https://linux.die.net/man/8/service

And checking if adb exists on the system wouldn't work either because adb could be installed directly on the Android device to access another Android device.

The safest way would be to make the --adb flag mandatory and allow setting it to --adb=no if you don't want to use adb. Then we could simply make it exit with an error if the --adb flag is missing.

But this would break backwards-compatibility. So I would prefer improving the Readme to state more clearly when to use --adb.

[UrielCh]

Service exists, but the default help message is totally different.

[T-vK]

Yes, but the help message can always change and calling the service binary probably requires root privileges on some systems and people might wanna use it on systems where root access isn't allowed for security reasons. It also sounds risky to call an executable without knowing it. There may very well be cases where service is a custom alias that ignores the --help flag and just calls a unknown bash script.

Instead of actually calling the service executable, I'd say it would make more sense simply scan it for a certain signature/string. The problem with that however is that you have to know the location of service which would require root privileges again, at least on Android.

The most likely solution I would consider at the moment would probably be checking for certain environment variables. For example TERMUX_VERSION which should only exist if executed on Android inside of Termux. The disadvantage of this approach of course would be that for every new terminal emulator that should be supported an additional check would have to be implemented. But there may be other environment variables that should exist on any Android terminal.

[agnostic-apollo]

Simply checking /system/build.prop would work, if its accessible, then we are on android, even within an adb shell. I am currently rewriting the script, and have included such support, it should "hopefully" be done in the next few days.

Another case is that adb wireless support is present on android, specially with now android-tools official package that has adb, so termux users who don't have root, may use that to make service calls.

There are basically 5 cases.

• From linux.
• From termux with root.
• From termux with adb.
• From outside termux, like with tasker Run Shell.
• From /data/local/tmp after pushing script with adb and running it inside adb shell.

[UrielCh]

Finally, I choose a perfect soliution.

Since you already depend on GitHub to access android sources. The simplest way is to pre-rebuild all the needed data mapping and push them to a GitHub repo.

You will just have to download a single file from this repo, to get all mapping to solve your speed issue.

I will try to maintain up-to-date git content with some GitHub Actions; if not, I will write access to people who can update its content if I forgot to do so. (A crontab can do the job, but it relay on a single PC, and so don't last forever)

[agnostic-apollo]

I have thought about that as well, that is doable, but not all aidl files of services are available on android github repo. Pulling them from elsewhere is possible. But even that solution has problems due to ro.build.version.release 11.0 not always being compatible with android version revision like 11.0.0_r35, also vendor specific incompatibility and missing vendor services. Best way is to have an app like termux-api to get all service interface methods with reflection on interface classes returned by service list and caching them.

Speed is not an issue with any solution, currently local caching is completely broken, I have fixed that with the rewrite as well (well almost done).

[UrielCh]

Wait and see.

[T-vK]

@UrielCh That would work indeed, but every time you want to add support for a new ROM, you'd have to update android-svc or at least your custom repo. Also, I'm not sure if users would like it if android-svc would be downloading files from a source other than the official one the ROM developers provide. I think we should give users the option to provide any git repo URL they want eventually. Even non-GitHub ones, to support ROMs that aren't hosted on GitHub. In that case however a very slow git clone might be required.

@agnostic-apollo

But even that solution has problems due to ro.build.version.release 11.0 not always being compatible with android version revision like 11.0.0_r35, also vendor specific incompatibility and missing vendor services.

True, especially with closed-source ROMs this is a big issue I haven't found a solution for yet.

Best way is to have an app like termux-api to get all service interface methods with reflection on interface classes returned by service list and caching them.

Ideally it wouldn't even require having to install a separate app, especially considering that I'd like it to be compatible with adb as well. But I suppose an app like this would be the only way to do it without requiring root? I don't really know anything about reflection, but it would be pretty amazing if we could somehow make this work without needing access to the aidl files.

Another thing I've been wondering about for a long time is a way to call the services directly (without going through the service binary). Maybe from an app like termux-api or maybe by writing a small dedicated Java program. I mean calling the services directly with Java could finally allow us to easily instantiate complex objects/datatypes and pass them to the service methods.

Speed is not an issue with any solution, currently local caching is completely broken, I have fixed that with the rewrite as well (well almost done).

Sorry about that, I wasn't aware it was broken. Glad to hear you wrote a fix. :)

[UrielCh]

I plan to group and compare all aidl files of each android version; if all revisions have the same aidl file signature, they will be merged and be available. If not, an extra parameter will be required to choose the proper mapping.

In conflict, all methods with an id smaller than the first change should also be available without giving the android revision number.

JSON files should work fine (you already using jq binary so, you will be able to use them)