Testing out on RPi, not allowed to connect

[fribse]

Testing out in headless

My LAN is set up with 10.11.12.0/24 The RPi is at 10.11.12.238 My WS is at 10.11.12.40 When I connect I get 10.11.12.40 - - [03/Mar/2022 13:14:20] code 403, message The IP of the device you're making the request from (10.11.12.40) has to be in the list of accepted IP addresses in order to use the NeewerLite-Python HTTP Server, any outside addresses will generate this Forbidden error. To use this device with NeewerLite-Python, add its IP address (or range of IP addresses) to the list of acceptable IPs I tried modifying line 144 acceptable_HTTP_IPs = ["10.11.12.0/24"] or acceptable_HTTP_IPs = ["10.11.12.0"] or acceptable_HTTP_IPs = ["10.11.12.*"] I still get the same message 😢

I tried doing it directly from the RPi via web to 127.0.0.1 and that worked fine. So please enlighten me :-D

[taburineagle]

Ha, yes - this one's easy to explain. Although since you're on a Pi, I see I need a non-GUI solution to this, so that'll need to be upcoming... ha, good to know! - There's a list of "allowed IP addresses" in the preferences that allow specific IP ranges through. By default, the ranges are 127.0.0.1 (which is why the Pi itself works) as well as IP ranges 192.168.*.* and 10.0.0.* - to add 10.11.12.0/24 to the allowed IP addresses, launch the GUI, go to Global preferences, and under "Acceptable IPs to use for the HTTP Server:", click on the text field underneath, make a new line (hit Enter) and type "10.11.12" - this is the wildcard for 10.11.12.*, then scroll to the bottom of the global preferences and click "Save Global Preferences" to add it to the list.

Since you're on a Pi (and theoretically mean you don't have access to the GUI from there), add this line to NeewerLite-Python.prefs inside the light_prefs folder (the latest release 0.11 moves the main preferences file inside the light_prefs folder - if you're running an older version, just stick it in the main NeewerLite-Python folder) to accomplish the same thing -

acceptable_HTTP_IPs=127.0.0.1;192.168;10.0.0;10.11.12

[fribse]

Hmm, so where is the light_prefs folder, under /opt/NeewerLite-Python ? I tried creating that, and then a NeewerLite-Python.prefs inside that one and adding the line, still no joy

I tried starting it in the GUI, and I got told that I need PySide2 But when doing a pip install PySide2 I got

ERROR: Could not find a version that satisfies the requirement PySide2
ERROR: No matching distribution found for PySide2

It looks like there are real issues for PySide2 on Raspberry OS.

[taburineagle]

Interesting - yeah, I tried both pip and pip3 for PySide2 and got the same result. To be honest, I never even considered running the GUI on a Raspberry Pi (I've always thought of that platform as a headless CLI system, even though there is a GUI system available for Raspberry Pi OS) - so not having a way to set the preferences without using a GUI didn't come to mind until this issue was raised, so definitely duly noted on my end! I need a CLI preferences system.

Yeah, if you're running the installation that maxime-roy suggested, then the preferences file should be located here: /opt/NeewerLite-Python/light_prefs/NeewerLite-Python.prefs

But that's only if you're running the latest version (0.11) - she suggested moving the prefs files into one folder for better Docker integration, that's why I made that change for this version - all the versions before the latest ones would store it here:

/opt/NeewerLite-Python/NeewerLite-Python.prefs

If you still want to try (not the suggested way to do it, but then again, I still need to work on the CLI-setting of preferences, so it's a stopgap measure for now) hard-coding that setting instead of going the NeewerLite-Python.prefs route, change line 2899 (you can't directly change line 144, as it always gets re-written on launch, line 144 is just initializing the variable itself, then the loadPrefsFile() function called after launch actually sets that variable to the list of IPs) from

prefsParser.add_argument("--acceptableIPs", default=["127.0.0.1", "192.168", "10.0.0"])

to

prefsParser.add_argument("--acceptableIPs", default=["127.0.0.1", "192.168", "10.0.0", "10.11.12"])

or if you'd rather,

prefsParser.add_argument("--acceptableIPs", default=["127.0.0.1", "10.11.12"])

that would give you local (Raspberry Pi) access, and access from 10.11.12.*

[fribse]

I did follow the systemd instructions, so I should be on 0.11, it doesn't really say clearly in the log or the code, but the package is called 0.11 :-)

The structure looks like:

pi@neewerlite:/opt/NeewerLite-Python/light_prefs $ ls
NeewerLite-Python.prefs
pi@neewerlite:/opt/NeewerLite-Python/light_prefs $ cat NeewerLite-Python.prefs
acceptable_HTTP_IPs=127.0.0.1;192.168;10.0.0;10.11.12
pi@neewerlite:/opt/NeewerLite-Python/light_prefs $

But accessing it from 10.11.12.40

pi@neewerlite:/opt/NeewerLite-Python/light_prefs $ python3 /opt/NeewerLite-Python/NeewerLite-Python.py --http
[08:54:58] - Loading global preferences from file...
[08:54:58] - Starting program with command-line arguments
[08:54:58] - Starting the HTTP Server on Port 8080...
[08:54:58] - -------------------------------------------------------------------------------------
10.11.12.40 - - [07/Mar/2022 08:55:05] code 403, message The IP of the device you're making the request from (10.11.12.40) has to be in the list of accepted IP addresses in order to use the NeewerLite-Python HTTP Server, any outside addresses will generate this Forbidden error.  To use this device with NeewerLite-Python, add its IP address (or range of IP addresses) to the list of acceptable IPs
10.11.12.40 - - [07/Mar/2022 08:55:05] "GET / HTTP/1.1" 403 -
10.11.12.40 - - [07/Mar/2022 08:55:05] code 404, message Not Found
10.11.12.40 - - [07/Mar/2022 08:55:05] "GET /favicon.ico HTTP/1.1" 404 -

Changing the code makes it work, so somehow it doesn't really read the prefs file. I did check that the ownership is pi:pi. If I accesss the root / it gives me a lot of 302's and the webpage shows an error that the redirects are wrong. If I add list to the doAction? it shows the page correctly.

I just tried copying the 'presets' from the windows box to the RPi, and that shows up, so I guess that part works, and the rights are correct.

[fribse]

Hmm, the pc prefs might not be directly compatible :-o

[09:16:47] - Searching for new lights
[09:16:52] - Found new light! [NEEWER-RGB480] MAC Address: CB:72:58:58:26:64 RSSI: -70 dBm
[09:16:52] - A custom preferences file was found for CB:72:58:58:26:64!
[09:16:52] - Found new light! [NEEWER-SNL660] MAC Address: DF:88:56:D0:72:A8 RSSI: -78 dBm
[09:16:52] - A custom preferences file was found for DF:88:56:D0:72:A8!
Exception in thread htmlProcessThread:
Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/NeewerLite-Python/NeewerLite-Python.py", line 2543, in processHTMLCommands
    loop.run_until_complete(findDevices()) # find the lights available to control
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/opt/NeewerLite-Python/NeewerLite-Python.py", line 1954, in findDevices
    customPrefs = getCustomLightPrefs(currentScan[a].address, currentScan[a].name)
  File "/opt/NeewerLite-Python/NeewerLite-Python.py", line 1975, in getCustomLightPrefs
    customPrefs = fileToOpen.read().split("|")
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd8 in position 0: invalid continuation byte

I've removed the 'custom naming' files for now, just for testing, they have danish letters in them. Without them, it can run...

[fribse]

So I've gotten it set up as a service. WiFi and BT doesn't really play well together on the RPi, serious connection issues, and lower signal strenght compared to running on cabled net. When it's running as a service, where is the logfile? I can get it flowing by using journalctl -f -u neewerlite-python but is it stored somewhere in /var? Secondly, it looks like the 'link' doesn't really work. If I do a 'scan for new lights' it links to the ones that it failed the first time, but the link doesn't do anything. It also lacks some sort of 'busy' while it scans, so that you shouldn't be able to ask it to discover while it's discovering.

[taburineagle]

Ha, quite a few things - sorry it's been such a pain in the ass! Pi testing has been a bit behind the GUI and CLI versions... The Pi was my intended end, but the GUI eventually took precedence, and I admittedly haven't completely worked all of the kinks out with the Pi-specific version of it.

it doesn't really say clearly in the log or the code, but the package is called 0.11 :-)

It does in a few places (the GUI has it, and the HTTP server does show it on the footer), but the code itself doesn't prominently, so... good point :) I'll add that to the top!

Changing the code makes it work, so somehow it doesn't really read the prefs file.

OK, that one's weird, because it does on the GUI (or it has in my testing), but I have to admit I haven't tested much on the Pi-side, so I'll need to take a look at that one. It should work fine on the Pi, and it does see the file (as it says Loading global preferences from file... in the log you posted), but it looks like it's not properly setting the variables from that.

it gives me a lot of 302's and the webpage shows an error that the redirects are wrong.

Also weird, I haven't seen any 302 or redirection errors myself - I'll take a look into that one as well.

UPDATE: OK, this one is definitely in the Pi version specifically (although I want to test this on Ubuntu as well), as the Windows version (even when explicitly told not to accept 127.0.0.1) still only gives me 403 errors and no 302 errors.

Hmm, the pc prefs might not be directly compatible :-o

That's definitely a file encoding issue - let me do some accented letter testing, it's possible I'm not reading the stream in as UTF-8 (which is why the Danish characters trip it up). I have run into that a few times when switching files from Windows to Linux, especially if their formatting is UTF-8.

When it's running as a service, where is the logfile?

The service settings as written send it to /dev/null, as the neewerlite-python.service service file itself has this line in it: StandardOutput=null

Secondly, it looks like the 'link' doesn't really work.

Will take a look at this one too!

It also lacks some sort of 'busy' while it scans, so that you shouldn't be able to ask it to discover while it's discovering.

This one it does actually have - when you try to run a command a 2nd time, the console (of course, you can't see that when running it as a service), says The HTTP Server requested an action, but we're already working on one. Please wait... - the HTML renderer will show the page again, but the 2nd/3rd/nth discover() call won't go through until the last one is done.

[fribse]

Eh? Pain in the ass, no, don't take it like that, there is no complaining from me, I'm just happy for all the great work put into this! I just wanted to report back what I've encountered so far, no stress at all, and with the minor code modifications you so kindly pointed out, it's running smoothly. My youtube studio lights are now completely automatic, so when I press the button on the stream deck, it lowers the blind, turns off all normal lights, set's the matrix display to say that I'm recording, and turns on the studio lights to the stored prefs, it's very nice :-) And it's still beta, so much better state than expected. It's very nice to know that you've made semaphores around the 'scan', it would be terrible to have it disturb it. But it would benefit if the webpage somehow reflected 'working' somewhere ;-)

[taburineagle]

Ha, that's good to hear 😊 Not taken any bad way - I definitely appreciate all the reports you've given me! Hehe, and honestly, testing can be a pain in the ass sometimes, but that's a good thing because it helps improve things. I would like to add a more indicative status display on the HTML page, most likely Javascript-based, so I can change a line or two of status reports, and the page itself didn't need to be refreshed as much. Ha, all things I definitely want to get into when I get a chance to!

[fribse]

Btw. the correct LAN zone for 10 is 10.0.0.0/8 :-)

[taburineagle]

Got ya! I wasn't sure if all of 10.0.0.0/8 was all private intranet or not (my ISP "rent-a-router" kind of system only uses 10.0.0.*), although now that I look that up, I see that it is... I will change that in the main code, then, when I get a chance to - I'm working on a freelance project that's taxing most of my time at the moment 😊

[taburineagle]

OK, @fribse - I still haven't had a chance to crack the code open again since the last release (freelance project is done and delivered, so I do have more time now), but I understand what happened with the accented characters. The Ø character in hex (for Øvre venstre and Øvre højre) doesn't decode properly in UTF-8, but they do in ISO-8859-1 encoding - so I need better codec handling for the custom names. What confuses me however is why the GUI has no issue reading those files, but the HTTP server does, using the same text encoding, unless the GUI just takes the data as it is, and the HTTP server parses it differently. More investigative work to be done...

[taburineagle]

Ha, well, I was a bit right, but not 100% - the UTF-8 encoding does handle that properly, but I had to explicitly define it in the HTTP server's rendering of the page. I will test that out more in the coming days...

[fribse]

Well Ø and ø are in UTF-8 and UTF-16, so it should decode properly, ahh, just saw your second update, that makes way more sense, apache is not good with uft-8 :-( Yes, it is rather strange when it's the same code showing the text

[taburineagle]

I think I have it sorted out - I had to add the HTML META tag to the renderer to force it to render everything in UTF-8 - <META HTTP-EQUIV='Content-Type' CONTENT='text/html;charset=UTF-8'>

Now it seems to work fine with the accented characters. Ha, I added an ß for good measure as well, just to test multiple characters out -

Ha, and yes... I am testing it out with my watch. Because I don't have any lights with me at the moment... 😆 But the lights should work the same way. I want to test it out more over the weekend.

[fribse]

I just tried my new bluetooth long range dongle on the RPi3b. I have disabled the on-board. With lsusb I see:

Bus 001 Device 004: ID 2550:8761 Realtek Bluetooth Radio
Bus 001 Device 005: ID 0424:7800 Microchip Technology, Inc. (formerly SMSC)
Bus 001 Device 003: ID 0424:2514 Microchip Technology, Inc. (formerly SMSC) USB 2.0 Hub
Bus 001 Device 002: ID 0424:2514 Microchip Technology, Inc. (formerly SMSC) USB 2.0 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

So the bluetooth is there. hciconfig gives me:

hci0:   Type: Primary  Bus: USB
        BD Address: 00:E0:4C:21:D2:33  ACL MTU: 1021:6  SCO MTU: 255:12
        UP RUNNING
        RX bytes:804 acl:0 sco:0 events:63 errors:0
        TX bytes:2755 acl:0 sco:0 commands:63 errors:0

Which I guess is also ok. But the neewerlite-python doesn't catch it, any idea on how to make it do that?

[taburineagle]

Ha, I saw the original comment you left before this last one (I think you deleted it), but... I forgot to bring the lights with me to test, hence the watch 🤣 Hmm... let me see here. I know how to use multiple wifi access points (I do that with curl, you can specify which interface it uses), but I'm not as familiar with using multiple Bluetooth devices.

[taburineagle]

I found this link after searching though several different ones that weren't nearly as helpful - hopefully this has some usable tips for switching interfaces. I haven't had a chance to try anything in this link out personally, but it does sound like it's on the right track -

https://www.pcsuggest.com/linux-bluetooth-setup-hcitool-bluez/

[fribse]

I'll close this for now, it's working ok with the built-in BT receiver in the RPi, so I think I'll save this for later... :-)