moleus / domru

domru domofon client
MIT License
7 stars 4 forks source link

BREAKING: New application implementation #13

Closed moleus closed 5 months ago

moleus commented 6 months ago

What's new

Breaking changes:

fixes: https://github.com/ad/domru/issues/13

twocolors commented 5 months ago

i test docker

domru:devel-new-implementation

and see error when use login/password (in web - not console)

failed to login with password: auth password request: failed to decode response. Body: '<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <title>..</title> <meta http-equiv="refresh" content="10;URL=/ciez2a" /> </head> <body> <script type="text/javascript"> // // !function(t,i){"object"==typeof exports&&"undefined"!=typeof module?i(exports):"function"==typeof define&&define.amd?define(["exports"],i):i(t.t={})}(this,function(t){"use strict";var e="0123456789abcdefghijklmnopqrstuvwxyz";function a(t){return e.charAt(t)}function i(t,i){return t&i}function u(t,i){return t|i}function r(t,i){return t^i}function n(t,

and two slash in link https://myhome.proptech.ru/auth/v2/auth//password

moleus commented 5 months ago

Oh, thx. I haven't checked login/password authentication yet. Will take a look in a week or so

moleus commented 5 months ago

html is returned by Varity server. It's a bot protection software. User guide - https://disk.yandex.ru/i/RVPkZX1EOVGSeA

Currently I don't know how to overcome it. When I send request using curl or go, it always returns html page instead of json.

As far as I understand this software checks headers, SSL handshake etc. Body is not processed.

curl -X POST 'https://myhome.proptech.ru/auth/v2/auth/{account_id}/password' \
-H 'User-Agent: Google sdkgphone64x8664 | Android 14 | erth | 8.9.2 (8090200) |  | null | 10c99d90-9899-4a25-926f-067b34bc4a7f | null' \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Connection: Keep-Alive' \
-H 'Accept-Encoding: gzip'

If Variti checks pass then the request is sent to nginx server which returns json with credentials

twocolors commented 5 months ago

i will try sniffing first req for password use https://requestly.com/blog/modify-http-traffic-from-android-apps/

moleus commented 5 months ago

i will try sniffing first req for password use https://requestly.com/blog/modify-http-traffic-from-android-apps/

I have already sniffed traffic using mitmproxy from domru application running emulator. I suspect that the Variti software is looking at TLS handshake and blocks based on TLS Fingerprint.

Currently, I'm trying to modify Client Hello using utls library to make Hello Message from go look like android's one

Golang's ClientHello has a very unique fingerprint, which especially sticks out on mobile clients, where Golang is not too popular yet. Some members of anti-censorship community are concerned that their tools could be trivially blocked based on ClientHello with relatively small collateral damage. There are multiple solutions to this issue.

Here is the client hello from android application: image

moleus commented 5 months ago

Here is the POST request to get token: image image image

twocolors commented 5 months ago

I don't think this is the main problem.

authorization via SMS works and everything else works

but I could be wrong...

moleus commented 5 months ago

I don't think this is the main problem.

Maybe, you are right. I can't prove that TLS fingerprint is checked (but i've managed to construct custom Client Hello which is the same as an Android's one).

I've found that there is an issue with headers order: if I change the order of headers in mitmproxy's replay feature, I get the html page from Variti, if I move headers back - I get response from nginx and valid tokens.

It's a common practice to detect bots and block them based on headers order. So I've noticed that Go sends request with headers ordered differently then Android does...

Currently, there is no easy way to order headers in http request (issue), but I'll try to find a solution or a workaround

moleus commented 5 months ago

I'll try fhttp to send ordered headers

moleus commented 5 months ago

@twocolors Hi! Finally, I've managed to defeat bot protection and random errors. Also, updated README.

May I ask you to check that it works in your setup?

twocolors commented 5 months ago

warning on run

dietpi@kubik3:~$ sudo docker compose logs -f --tail=150 domru 
domru  | 2024/06/02 16:32:21 Credentials file is not set
domru  | Usage of /ko-app/domru:
domru  |       --credentials string   credentials file path (i.e: /usr/domofon/credentials.yaml
domru  |       --login int            dom.ru login or phone (i.e: 71231234567)
domru  |       --operator int         operator id
domru  |       --port int             listen port (default 18000)
domru  |       --refresh string       dom.ru refresh token
domru  |       --token string         dom.ru token
domru  | 2024/06/02 16:32:21 Operator id is not set. Set your value
domru  | Usage of /ko-app/domru:
domru  |       --credentials string   credentials file path (i.e: /usr/domofon/credentials.yaml
domru  |       --login int            dom.ru login or phone (i.e: 71231234567)
domru  |       --operator int         operator id
domru  |       --port int             listen port (default 18000)
domru  |       --refresh string       dom.ru refresh token
domru  |       --token string         dom.ru token
domru  | 2024/06/02 16:32:21 Listening on %!s(int=18000)

and my password not work for connect

moleus commented 5 months ago

this is because of old image. Please, run docker-compose pull and check that image hash matches the latest version

twocolors commented 5 months ago

oh is my mistake , on error login | password

Снимок экрана 2024-06-02 в 16 49 23

and when is good

failed to save credentials: open accounts.json: permission denied

my docker

  domru:
    container_name: "domru"
    hostname: "domru"
    image: "ghcr.io/moleus/domru:devel-new-implementation"
    environment:
      - TZ=Europe/Moscow
    volumes:
      - "/home/dietpi/.domru:/share/domofon"
    restart: unless-stopped
    networks:
      - stalker
    ports:
      - 18000:18000
dietpi@kubik3:~$ pwd
/home/dietpi
dietpi@kubik3:~$ ll .domru/
total 12K
drwxr-xr-x  2 root   root   4.0K Jun  2 16:50 ./
-rwxrwxrwx  1 root   root      0 Jun  2 16:50 accounts.json*
drwxr-xr-x 12 dietpi dietpi 4.0K Jun  2 16:48 ../
-rwxrwxrwx  1 root   root    163 Jun  2 16:36 account.json*
dietpi@kubik3:~$ 
moleus commented 5 months ago

failed to save credentials: open accounts.json: permission denied

oh, I see. The image is built using ko-build and it uses chainguard/static as a base image where default user has ID=65532, so it can't write in directory where root is owner.

There are a few possible solutions:

  1. use root in container: user: 0:0 in docker compose
  2. change permissions of volume mount on host: chown -R 65532:65532 ~/.domru
twocolors commented 5 months ago

failed to save credentials: open accounts.json: permission denied my dir

dietpi@kubik3:~$ ll
total 104K
drwxr-xr-x 12 dietpi dietpi 4.0K Jun  2 22:33 ./
-rw-r--r--  1 dietpi dietpi 4.8K Jun  2 22:33 docker-compose.yml
drwxr-xr-x  6 dietpi dietpi 4.0K Jun  2 17:00 .vaultwarden/
drwxrwxrwx  2  65532  65532 4.0K Jun  2 16:50 .domru/
-rw-------  1 dietpi dietpi  25K May 31 15:36 .bash_history
drwxr-xr-x  8 dietpi dietpi 4.0K May 28 15:20 .node-red/
drwxr-xr-x  3 dietpi dietpi 4.0K May  8 22:04 .homepage/
drwxr-xr-x  4 root   root   4.0K May  3 13:59 .android/
drwx------  3 dietpi dietpi 4.0K Apr 27 15:57 .config/
-rw-r--r--  1 dietpi dietpi 1.4K Apr 24 14:19 .multibackup
drwxr-xr-x  3 dietpi dietpi 4.0K Jan 16 09:53 .esphome/
drwxr-xr-x  4 root   root   4.0K Dec 31 00:44 .scrypted/
drwxr-xr-x  2 dietpi dietpi 4.0K Nov 27  2023 .go2rtc/
-rw-r--r--  1 dietpi dietpi  571 Nov 27  2023 .bashrc
-rw-r--r--  1 dietpi dietpi  436 Nov 27  2023 tunnel-nbzsv7.service
drwxr-xr-x  2 dietpi dietpi 4.0K Nov 27  2023 .ssh/
drwxr-xr-x  3 root   root   4.0K Nov 24  2023 ../
-rw-r--r--  1 dietpi dietpi  161 Nov 24  2023 .profile
-rw-r--r--  1 dietpi dietpi  220 Apr 24  2023 .bash_logout
dietpi@kubik3:~$ ll .domru/
total 12K
drwxr-xr-x 12 dietpi dietpi 4.0K Jun  2 22:33 ../
-rwxrwxrwx  1  65532  65532  163 Jun  2 17:07 account.json*
drwxrwxrwx  2  65532  65532 4.0K Jun  2 16:50 ./
-rwxrwxrwx  1  65532  65532    0 Jun  2 16:50 accounts.json*
dietpi@kubik3:~$ 

this 0.1.6-dev.0 work good and have all permit to rw account.json

twocolors commented 5 months ago

user: 0:0 this help , but running as root is a bad idea

moleus commented 5 months ago
    volumes:
      - "/home/dietpi/.domru:/share/domofon"

I think that you also have to specify workdir or credentials path because default workdir is / and user can't write to it.

P.S: added /share/domofon/accounts.json as a default path for credentials file. Not it should work out of the box ;)

twocolors commented 5 months ago

@moleus oh if you can find me in https://t.me/alice_in_red_wonderland