env_parsing.py treated as a directory, then failing.

[jselan]

Hi, I receive this error when running app.py: [CRITICAL] The .env file cannot be created [[Errno 20] Not a directory: '/home/jusko/huawei/libs/env_parsing.py/../../.env']

I opened reddit thread. https://www.reddit.com/r/learnpython/comments/123l1mj/issues_with_installing_python_app_from_github/

I've been redirected here. One of the replies was: " Digging into the code a bit, it is treating env_parsing.py as a directory and failing.

This line specifically seems to be causing the problem. As for a solution, I'd create an Issue on GitHub to see if the creator can assist further. This seems like something they should know about.

https://github.com/yoratoni/huawei-router-sms-forwarding/blob/aa5278703dde065fa4d2c3cfd9b3e8af2c256709/src/libs/env_parsing.py#L114 "

[yoratoni]

Hi! Sorry about that, I never really updated the code, I made that when I was a beginner so yeah, a lot of mistakes ^^

I'm gonna update the whole thing today and i'll come back to you to see if everything works fine.

[jselan]

Great! No need to be sorry :-) I will test as soon as you make changes...

[yoratoni]

So, I updated the whole code, I tested it with Python 3.11 and it should work.

[jselan]

It is not working for me :(

jusko@pihole:~/huawei $ python app.py Traceback (most recent call last): File "/home/jusko/huawei/app.py", line 1, in from libs import HuaweiWrapper, EnvParsing, AppHistory File "/home/jusko/huawei/libs/init.py", line 3, in from libs.env_parsing import EnvParsing File "/home/jusko/huawei/libs/env_parsing.py", line 11, in class EnvParsing: File "/home/jusko/huawei/libs/env_parsing.py", line 131, in EnvParsing def get_formatted_env() -> dict[str, str | int | list[str]]: TypeError: unsupported operand type(s) for |: 'type' and 'type'

This is my enviroment:

jusko@pihole:~/huawei $ python --version Python 3.9.2

jusko@pihole:~/huawei/ $ pip list Package Version

certifi 2020.6.20 chardet 4.0.0 colorama 0.4.6 colorzero 1.1 distro 1.5.0 dotenv 0.0.5 gpiozero 1.6.2 huawei-lte-api 1.6.11 idna 2.10 numpy 1.19.5 picamera2 0.3.9 pidng 4.0.9 piexif 1.1.3 Pillow 8.1.2 pip 20.3.4 pycryptodomex 3.9.7 pyostra 0.1.4 python-apt 2.2.1 python-dotenv 1.0.0 python-prctl 1.7 requests 2.25.1 RPi.GPIO 0.7.0 setuptools 52.0.0 simplejpeg 1.6.4 six 1.16.0 spidev 3.5 ssh-import-id 5.10 toml 0.10.1 urllib3 1.26.5 v4l2-python3 0.3.2 wheel 0.34.2 xmltodict 0.12.0

jusko@pihole:~/huawei $ pwd /home/jusko/huawei

jusko@pihole:~/huawei $ ls -la -rw-r--r-- 1 jusko jusko 1047 Mar 29 08:18 app.py -rw-r--r-- 1 jusko jusko 1066 Mar 29 08:23 .env drwxr-xr-x 4 jusko jusko 4096 Mar 29 08:30 libs

jusko@pihole:~/huawei/libs $ pwd /home/jusko/huawei/libs

jusko@pihole:~/huawei/libs $ ls -la -rw-r--r-- 1 jusko jusko 3451 Mar 29 08:27 app_history.py -rw-r--r-- 1 jusko jusko 6509 Mar 29 08:27 env_parsing.py -rw-r--r-- 1 jusko jusko 12952 Mar 29 08:28 huawei_wrapper.py -rw-r--r-- 1 jusko jusko 126 Mar 29 08:27 init.py drwxr-xr-x 3 jusko jusko 4096 Mar 29 08:30 logger drwxr-xr-x 2 jusko jusko 4096 Mar 29 08:30 pycache

jusko@pihole:~/huawei/libs/logger $ pwd /home/jusko/huawei/libs/logger

jusko@pihole:~/huawei/libs/logger $ ls -la -rw-r--r-- 1 jusko jusko 1164 Mar 29 08:29 init.py drwxr-xr-x 2 jusko jusko 4096 Mar 29 08:30 pycache -rw-r--r-- 1 jusko jusko 9560 Mar 29 08:29 pyprint.py

[yoratoni]

Oh I see, you're using Python 3.9 which does not support the new "|" operand, I updated the code with "Union" instead, could you try and tell me if it works pls.

[jselan]

Great job! Works as expected 👍

jusko@pihole:~/huawei $ python app.py [INFO] Trying to connect to the router.. [INFO] Successfully connected to the router [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found.. [DATA] (2023-03-30 09:03:33) New SMS received from +386xxxxxxxx [SUCCESS] SMS correctly forwarded to +386xxxxxxxx [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found.. [INFO] No new SMS found..

[yoratoni]

Perfect! Don't hesitate if you have any suggestion!

[jselan]

Well, I have an idea, but not sure if it this Huawei API provides that option. Is it possible for me to send SMS back to my LTE router which would then forward this SMS to specific number, or let's say to a number from which the SMS was originally sent??

Explanation:

1. LTE router I have is Huawei B535-232 and it is my only internet router at home.
2. My provider limits internet speed after 20GB daily use, and I receive SMS from provider that speed is reduced.
3. But the data package is not limited. I just have to send SMS to specific number with specific word = NADALJUJ (in Slovene language this means CONTINUE) and then I get additional 20GB. And I can send unlimited SMSes...
4. ATM I will be at least notified on my phone when the speed is reduced - which is great!
5. But I will still need to login to my router and send SMS back to the provider from Huawei WebUI.

I am sorry to write herein. I am not sure where to give a "feature-request".

[yoratoni]

The thing that I could do is to add a better config system with a YAML file instead of the .env file which is not the best practice to configure from. It will allow me to make a new config dict where I'll add something like an auto-reply system, it detects when a specific message is received and replies something else to a specified number.

It would be like:

autoReply: {
    "message": "YOUR_INTERNET_PROVIDER_MESSAGE",  # If the message is fix, or I could find another way like the number to detect from
    "replyTo": "+386_xxxxxxxx_",
    "replyMsg": "NADALJUJ"
}

I just need to know if you prefer to detect all messages from a specific number, or detect a specific message, because I don't know if your internet provider sends a message where the content is fix (like, no date etc..).

[jselan]

I think the best would be to detect messages from a specific number.

This is the SMS that my provider sends from their number 7070: "Obvescamo vas, da ste presegli celotno kolicino dnevnega prenosa podatkov po najvisji razpolozljivi hitrosti v vasem narocniskem paketu, zato je hitrost zmanjsana na 64/64 kbps. Za nadaljnjo uporabo podatkovne storitve pri najvisji razpolozljivi hitrosti, ki ne predstavlja dodatnih stroskov, posljite SMS s kljucno besedo NADALJUJ na stevilko 7070. stevilo podaljsanj ni omejeno. Vas Telemach."

This is direct translation (just for you understanding): "We inform you that you have exceeded the total amount of daily data transfer at the highest available speed in your subscription package, so the speed is reduced to 64/64 kbps. To continue using the data service at the highest available speed, which does not represent additional costs, send an SMS with the keyword CONTINUE to the number 7070. The number of extensions is not limited. Your Telemach."

Print screen of my router SMS page

I do not mind if this is achieavable via YAML 😋

[yoratoni]

So, I updated the whole code again:

• I changed the old logger that I made a long time ago, that was totally inefficient, to a logging library used by everyone
• I also replaced the .env config by the YAML one.
• Updated the README
• Added your replier feature!

You can test it and tell me if everything works fine!

Don't hesitate to add a star to my repo if you like my work, it's already so cool that someone actually uses that code lol

[jselan]

Hi,

I have tested the app... and I have some issues.

Python libraries were updated via "pip install -r requirements.txt"

jusko@pihole:/$ pip list argcomplete 2.0.0 certifi 2022.12.7 chardet 4.0.0 charset-normalizer 3.1.0 click 8.1.3 colorama 0.4.6 colorzero 1.1 distro 1.5.0 docopt 0.6.2 dotenv 0.0.5 Flask 2.2.3 gpiozero 1.6.2 huawei-lte-api 1.6.11 idna 3.4 importlib-metadata 6.1.0 itsdangerous 2.1.2 Jinja2 3.1.2 MarkupSafe 2.1.2 numpy 1.19.5 packaging 21.3 picamera2 0.3.9 pidng 4.0.9 piexif 1.1.3 Pillow 8.1.2 pip 20.3.4 pipreqs 0.4.11 pipx 1.1.0 pycryptodomex 3.17 pyostra 0.1.4 pyparsing 3.0.9 python-apt 2.2.1 python-dotenv 1.0.0 python-prctl 1.7 PyYAML 6.0 requests 2.28.2 RPi.GPIO 0.7.0 setuptools 52.0.0 simplejpeg 1.6.4 six 1.16.0 spidev 3.5 ssh-import-id 5.10 toml 0.10.1 urllib3 1.26.15 userpath 1.8.0 v4l2-python3 0.3.2 Werkzeug 2.2.3 wheel 0.34.2 xmltodict 0.13.0 yarg 0.1.9 zipp 3.15.0

Folder situation: jusko@pihole:/$ pwd /home/jusko/lte_sms jusko@pihole: /$ ls -la total 24 drwxr-xr-x 3 jusko jusko 4096 Apr 2 09:32 . drwxr-xr-x 7 jusko jusko 4096 Apr 2 08:32 .. -rw-r--r-- 1 jusko jusko 1199 Apr 2 08:56 app.py -rw-r--r-- 1 jusko jusko 1794 Apr 2 09:18 config.yaml drwxr-xr-x 4 jusko jusko 4096 Apr 2 09:19 libs -rw-r--r-- 1 jusko jusko 418 Apr 2 08:50 requirements.txt

jusko@pihole:/$ pwd /home/jusko/lte_sms/libs jusko@pihole:/$ ls -la total 48 drwxr-xr-x 4 jusko jusko 4096 Apr 2 09:19 . drwxr-xr-x 3 jusko jusko 4096 Apr 2 09:32 .. -rw-r--r-- 1 jusko jusko 3224 Apr 2 09:00 app_history.py -rw-r--r-- 1 jusko jusko 6699 Apr 2 09:01 config_parser.py -rw-r--r-- 1 jusko jusko 14141 Apr 2 09:01 huawei_wrapper.py -rw-r--r-- 1 jusko jusko 38 Apr 2 08:59 init.py drwxr-xr-x 3 jusko jusko 4096 Apr 2 09:19 logger drwxr-xr-x 2 jusko jusko 4096 Apr 2 09:19 pycache

If I run as "normal" user I get CRITICAL error (yaml not found) jusko@pihole:/ $ python app.py [2023-04-02 09:42:32] [CRITICAL] The config.yaml file cannot be found, please, check that you renamed the file 'example.yaml' to 'config.yaml'

If I run sudo I get 'no module yaml': jusko@pihole:/$ sudo python app.py Traceback (most recent call last): File "/home/jusko/lte_sms/app.py", line 1, in from libs.huawei_wrapper import HuaweiWrapper File "/home/jusko/lte_sms/libs/huawei_wrapper.py", line 7, in from libs.config_parser import ConfigParser File "/home/jusko/lte_sms/libs/config_parser.py", line 4, in import yaml ModuleNotFoundError: No module named 'yaml'

[yoratoni]

For the normal user, it could be because the script don't have the right to access the config file, so you could just modify the rights maybe ? For the YAML error, it's strange that it happens only when run as sudo...

The module is normally named PyYAML and you have it: https://bobbyhadz.com/blog/python-no-module-named-yaml

So, it's on your side, if you can't find a way to solve it, I could try to add a dynamic import with maybe another module ? I know that a guy made an alternative to the original YAML module.

[jselan]

I was tshooting and I can't solve it. I now get same results with sudo as well. Problem with sudo before was the pip packages were installed just for user jusko not for sudo. After I ran sudo pip install -r requirements.txt and then sudo python app.py I now also get: [CRITICAL] The config.yaml file cannot be found, please, check that you renamed the file 'example.yaml' to 'config.yaml'

---

Also the permissions for yaml file are correct: jusko@pihole:~/lte_sms $ ls -l -rw-r--r-- 1 jusko jusko 1199 Apr 2 08:56 app.py -rw-r--r-- 1 jusko jusko 1794 Apr 2 09:18 config.yaml → jusko has read/write permissions drwxr-xr-x 4 jusko jusko 4096 Apr 3 22:55 libs -rw-r--r-- 1 jusko jusko 418 Apr 2 08:50 requirements.txt

Additional tshoot:

jusko@pihole:~/lte_sms $ pip show pyyaml Name: PyYAML Version: 6.0 Summary: YAML parser and emitter for Python Home-page: https://pyyaml.org/ Author: Kirill Simonov Author-email: xi@resolvent.net License: MIT Location: /home/jusko/.local/lib/python3.9/site-packages Requires: Required-by:

---

I temporarily added _print("yaml_path:", yamlpath) before the _if os.path.exists(yamlpath) line in _configparser.py in 'load_yaml' method to see if it matches the expected path.

jusko@pihole:~/lte_sms $ python app.py yaml_path: config.dev.yaml [2023-04-03 22:52:49] [CRITICAL] The config.yaml file cannot be found, please, check that you renamed the file 'example.yaml' to 'config.yaml'

---

This is all my punny brain is capable off... I am not a sw developer and I am quite struggling with all if this :sweat_smile:

[yoratoni]

Oops, I think I know why lol, I forgot to turn off development mode before pushing to GitHub..

Just committed, you can try now!

For your information config = ConfigParser.get_config(True) at the beginning of the app.py file try to search for config.dev.yaml instead of config.yaml, I was using that for development and I forgot to remove the True lol

[jselan]

We are pretty close, but not there yet 😄

jusko@pihole:~/lte_sms $ python app.py [2023-04-04 18:59:23] [CRITICAL] The phone number must start with a + [7070]

The thing is that there are "special" numbers, e.g. 911 in USA. For these numbers we cannot use + sign. It is actually just the number without country code in any format (+386 / 00386)

I am actually sending SMSes back to number 7070 (exactly as written)

[yoratoni]

Okayyyy, now it should work !

[jselan]

This is what I see now

jusko@pihole:~/sms_lte $ python app.py Traceback (most recent call last): File "/home/jusko/sms_lte/app.py", line 11, in AppHistory.load_history() File "/home/jusko/sms_lte/libs/app_history.py", line 92, in load_history AppHistory.save_history() File "/home/jusko/sms_lte/libs/app_history.py", line 67, in save_history with open(AppHistory.HISTORY_PATH, "w+") as history_file: FileNotFoundError: [Errno 2] No such file or directory: 'logs/history.json'

[yoratoni]

So, it creates the history file but not the /logs directory, so, if the directory is missing, it returns an error.

I added a function that creates the directory in case it is missing, and a fallback that simply disables the history if there's a permission error.

Sorry for all the bugs lol

[jselan]

Application now runs! 👍🏼 Router also forwards the SMS to a specified number in yaml file. But the router doesn't reply to a filtered text.

See prtscr: https://ibb.co/3mKHh6y

[yoratoni]

I tried to reproduce it but I don't have that issue on my side, what the does the code prints inside the console ?

Normally, it prints this:

[2023-04-05 16:03:16] [INFO] New SMS received from +336XXXXXXXX
[2023-04-05 16:03:16] [INFO] SMS correctly sent to +336XXXXXXXX

I tried with the same message and a second filter to see if there is any issue, but I don't have any.

[yoratoni]

The SMS forwarding system and the replier system uses the same send_sms() function, if the forwarding works, it certainly means that the filter is the problem.

I'm gonna update the code with a bunch of temporary print statements to see what's going on.

[jselan]

This is what the console prints when I run app.py

jusko@pihole:~/sms_lte $ python app.py [2023-04-05 19:55:20] [INFO] Loading config.yaml [2023-04-05 19:55:20] [INFO] Trying to connect to the router.. [2023-04-05 19:55:20] [INFO] Successfully connected to the router [2023-04-05 19:55:20] [INFO] New SMS received from +386xxxxxxxx [2023-04-05 19:55:20] [INFO] SMS correctly sent to +386xxxxxxxx [2023-04-05 19:55:22] [INFO] No new SMS found.. [2023-04-05 19:55:22] [INFO] No new SMS found.. [2023-04-05 19:55:22] [INFO] No new SMS found.. ^C [2023-04-05 19:55:35] [INFO] Successfully disconnected from the router

This is fine. Router forwards SMS, but it doesn't reply... :man_shrugging:

[yoratoni]

Is the code updated with the latest push i've made ? It should print a bunch of other things

[jselan]

I ran the updated code:

jusko@pihole:~/sms_lte $ python app.py
[2023-04-07 08:29:30] [INFO] Loading config.yaml
[2023-04-07 08:29:31] [INFO] Trying to connect to the router..
[2023-04-07 08:29:31] [INFO] Successfully connected to the router
[2023-04-07 08:29:33] [INFO] No new SMS found..
[2023-04-07 08:29:35] [INFO] New SMS received from +3863xxxxxxx
[2023-04-07 08:29:35] [INFO] SMS correctly sent to +3865xxxxxxx
SMS Replier: +3865xxxxxxx
Repliers: {'7070': [{'filter': 'posljite SMS s kljucno besedo NADALJUJ na stevilko 7070', 'reply': 'NADALJUJ'}], '+3863xxxxxxx': [{'filter': 'test123', 'reply': 'test 123 dela'}], '+3865xxxxxxx': [{'filter': 'test456', 'reply': 'test 456 dela'}]}
Replier: [{'filter': 'test456', 'reply': 'test 456 dela'}]
SMS Content: test456
Message Filter: test456
[2023-04-07 08:29:35] [ERROR] SMS cannot be sent to +3865xxxxxxx
113004: Unknown
Is the SMS properly sent: False
[2023-04-07 08:29:37] [INFO] No new SMS found..
[2023-04-07 08:29:39] [ERROR] Last SMS received by the router cannot be returned
125003: Wrong Session Token
[2023-04-07 08:29:41] [ERROR] Last SMS received by the router cannot be returned
100003: No rights (needs login)

This is yaml config:

# The parameters of the router that you're using.
router:
  # Router IP address (generally 192.168.8.1)
  ip_address: "192.168.8.1"

  # The phone number of the router.
  phone_number: "+386 7x xxx xxx"

  # Account details (the same one used to identify yourself on the local Huawei router website).
  username: "xxxxxx"
  password: "xxxxxxxxxxx"

  # Delay between iteration of the loop, checks if a SMS has been received (in seconds).
  loop: 2

# (For the forwarders only) Allows to link a phone number to a contact name.
contacts:
  - phone_number: ""
    name: ""
# - phone_number: ""
#   name: ""

# A forwarder allows the SMS received by the router to be sent to a phone number.
# A whitelist of international phone numbers can be added, if empty, the whitelist is disabled.
forwarders:
  - phone_number: "+386 5x xxx xxx"
    whitelist: []
# - phone_number: ""
#   whitelist: []

# A replier allows to reply something to a phone number in the case of a received filter message.
# The filter should be the first words to appear inside the message that you want to reply to.
repliers:
  - phone_number: "7070"
    messages:
      - filter: "posljite SMS s kljucno besedo NADALJUJ na stevilko 7070"
        reply: "NADALJUJ"
  - phone_number: "+386 3x xxx xxx"
    messages:
      - filter: "test123"
        reply: "test 123 dela"
  - phone_number: "+386 5x xxx xxx"
    messages:
      - filter: "test456"
        reply: "test 456 dela"

[yoratoni]

So, the issue seems to come from the API, I added a bit more code for error-handling, could you retry ? It seems like the API disconnects from your router when it tries to send an SMS from the replier code.

I checked their repository but it seems like there's no info about this error code 113004: Unknown

[jselan]

I ran the updated code and it is pretty much the same.

jusko@pihole:~/sms_lte $ python app.py
[2023-04-07 16:06:03] [INFO] Loading config.yaml
[2023-04-07 16:06:03] [INFO] Trying to connect to the router..
[2023-04-07 16:06:03] [INFO] Successfully connected to the router
[2023-04-07 16:06:07] [INFO] No new SMS found..
[2023-04-07 16:06:09] [INFO] New SMS received from +3865xxxxxxx
[2023-04-07 16:06:09] [INFO] SMS correctly sent to +38651xxxxxxx
[2023-04-07 16:06:09] [ERROR] SMS cannot be sent to +3865xxxxxxx
113004: Unknown
API response: UNKNOWN
[2023-04-07 16:06:11] [INFO] No new SMS found..
[2023-04-07 16:06:13] [ERROR] Last SMS received by the router cannot be returned
125003: Wrong Session Token
[2023-04-07 16:06:16] [INFO] Trying to connect to the router..
[2023-04-07 16:06:16] [INFO] Successfully connected to the router
[2023-04-07 16:06:28] [INFO] No new SMS found..
[2023-04-07 16:06:29] [INFO] Successfully disconnected from the router
jusko@pihole:~/sms_lte $

[yoratoni]

So, yeah, it seems like the problem comes from the Huawei API then, it's a bit strange because I don't have that particular issue myself, at least, I added some code that restarts the bot if it happens.

It's even more strange because the forwarder works and sends the SMS with the same internal function, but not the replier, maybe because the two requests are too close to each other, idk, I'm gonna try something.

[yoratoni]

On my side, it just sends both SMS one after the other..

I suppose that, yeah, the issue could be from the fact that it requests to send two SMS too quickly ? Try removing the forwarder maybe but I doubt that it solves the issue.

It's pretty hard to debug that when I can't reproduce the issue myself.

[jselan]

Well actually that is it! :rocket: After I removed forwarder, the replier sent SMS back! I will be a bit in the dark, but hey I will monitor for few days and I'll see how it goes...

jusko@pihole:~/sms_lte $ python app.py
[2023-04-07 17:12:28] [INFO] Loading config.yaml
[2023-04-07 17:12:28] [INFO] Trying to connect to the router..
[2023-04-07 17:12:29] [INFO] Successfully connected to the router
[2023-04-07 17:12:29] [INFO] No new SMS found..
[2023-04-07 17:12:31] [INFO] No new SMS found..
[2023-04-07 17:12:33] [INFO] No new SMS found..
[2023-04-07 17:12:35] [INFO] New SMS received from +38651380875
[2023-04-07 17:12:35] [INFO] SMS correctly sent to +38651380875
[2023-04-07 17:12:37] [INFO] No new SMS found..
[2023-04-07 17:12:39] [INFO] No new SMS found..

[yoratoni]

I added a 1 second delay between the two functions so it should work now!

[jselan]

Yeah boi ! It works :grinning: Thanks a lot :thumbsup:

jusko@pihole:~/sms_lte $ python app.py
[2023-04-07 19:46:01] [INFO] Loading config.yaml
[2023-04-07 19:46:02] [INFO] Trying to connect to the router..
[2023-04-07 19:46:02] [INFO] Successfully connected to the router
[2023-04-07 19:46:02] [INFO] No new SMS found..
[2023-04-07 19:46:05] [INFO] No new SMS found..
[2023-04-07 19:46:08] [INFO] No new SMS found..
[2023-04-07 19:46:11] [INFO] New SMS received from +3863xxxxxxx
[2023-04-07 19:46:11] [INFO] SMS correctly sent to +3865xxxxxxx
[2023-04-07 19:46:12] [INFO] SMS correctly sent to +3863xxxxxxx
[2023-04-07 19:46:14] [INFO] No new SMS found..
[2023-04-07 19:46:17] [INFO] No new SMS found..