New OS for RPI4/5 causing Problems

decodeais commented 5 months ago

I wanted to set up RasperiPy4 again, as I use it as a true multifunctional device (Print spooler, web server, etc.). However, with the latest version of the OS (Bookworm), some issues arose.

Activating the VNC server is no longer as straightforward. "raspi-config" must be called , and then "WayVNC" needs to be switched to the X backend. After that, RealVNC can be activated as usual by enabling the "VNC" option in the "Interface Options". Additionally, the "wayvnc" backend had to be uninstalled, as it caused annoying delays during boot. The old instruction in the manual is no longer entirely accurate and could be confusing.
Whenever "pip3" commands were executed in the installation script, the option "--break-system-packages" had to be added.
Now that the new Python Version 3.11 is installed, the WHL for wxPython no longer fits. The reference in the manual to MarcDobler's page was useless as the link was dead (it should be removed from the manual).

Nevertheless, I managed to generate a suitable one:

sudo apt-get install libgtk-3-dev
sudo apt-get install libpng-dev libjpeg-dev libtiff-dev
pip install -U --no-binary :all: wxPython

The generated file "wxPython-4.2.1-cp311-cp311-linux_aarch64.whl" works, The but the scales on the performance and speed indicators have disappeared.

These were the main issues. Initially, I would appreciate a suggestion on where to search for the missing scales. However, this is only a "Design" problem.

WouterJD commented 5 months ago

Yeah Yeah; "Python is a WORA (Write Once, Run Anywhere) language."; everybody may judge how true the statement is. Sorry for the inconvenience, anyway.

Please suggest what changes help you forward, apart from the missing whl.

WouterJD commented 5 months ago

The reference in the manual to MarcDobler's page was useless as the link was dead (it should be removed from the manual).

Hmmm... Did a quick check but don't find; please what page?

decodeais commented 5 months ago

Sorry, the address of marcDobler.com appeared in the message from the instalation script "4_InstallWxPython.sh".

decodeais commented 5 months ago

The missing scales look a little strange, and i would like to know, wether the generated whl in issue #440 has the same lost. It has the same wx revision number.

WouterJD commented 5 months ago

I do not have an RPI4/5 so cannot test, but perhaps others can...

decodeais commented 5 months ago

Will you add the missing Parameter "--break-system-packages"? Or will this brake the installation for the others?

WouterJD commented 5 months ago

I'm ok to do so, but in what line please

decodeais commented 5 months ago

every pip3 command needs it.

decodeais commented 5 months ago

I see two ways:

May be, thats it is easier to use the virtual envirement to determine a special Python Version. This would fix the error messages from the Script.
The second idea would be, somebody makes a minimum installation and compress it to the minimum image size (~3GB). One for Raspi0 and Raspi3/4/5.

decodeais commented 5 months ago

There is a strange behavior when I close the program. the terminal windows stays open and does not close.

22:31:31,016: FortiusAnt applies the MotorBrake power curve
22:31:35,484: Tacx2Dongle; Pedal Stroke Analysis active
22:31:35,484: ---------- BLE-devices are activated ----------
22:31:37,486: ---------- BLE-devices are deactivated ----------

Does this effect only appear with "bookworm"? I think I will look into the source code. May be you can give me a hint.

WouterJD commented 5 months ago

It happens on Windows as well. Not reproducible.

Perhaps a coding error waiting for threads to complete. I don't know

decodeais commented 5 months ago

For me the waiting the thread problem seems be solved. I could not find the reason. I fixed the problem by "brutal force". I look for the process id from main. At the end I kill it. In FortiusAnt.py (731 )I get the PID:

#-----------------------------------------------------------------------------------
# Main program; when restart is required due to new parameters we will loop
#-----------------------------------------------------------------------------------
if __name__ == "__main__":
    multiprocessing.freeze_support()
    global RestartApplication, clv
    Kill.pid = os.getpid()
    RestartApplication = False

In FortiusAntGui.py (1502) I kill it:

if self.CloseButtonPressed == True:
            self.CloseButtonPressed = False     # Otherwise self.Close() is blocked
            self.Close()
            os.kill(Kill.pid, 9)

What do you think, will it have bad side effects?

decodeais commented 4 months ago

I found the reason for the disappeared Scales ( it was a white eagle on white ground problem). I found the reason in the speedmeter library. When I added the typecast to the color changing variables the blue background came back and the white scaling appeared again. (speedmeter.py, line 843)

                #currCol = (r1 + rf, g1 + gf, b1 + bf)
                currCol = (r1 + int(rf), g1 +int(gf), b1 +int(bf))

                dc.SetBrush(wx.Brush(currCol))

I have no experience how to handle this kind of problem. Should somebody write an issue on github for speedmeter.py ?

WouterJD commented 4 months ago

Hi @decodeais good work.

You have greatly re-engineered to the location where the problem is created, let's see how to help.

Some questions (I assume you can answer):

In what function, the "currCol=" statement appears,
the r1, rf, g1, gf and b1, bf may be provided by FortiusAnt; can you tell me in what function-call?

If you tell me this, it avoids work for myself. I never downloaded the speedmeter code untill now.

WouterJD commented 4 months ago

os.kill(Kill.pid, 9)

I will look into it; in fact it suggests there is some flow-error and it circumvents the problem creatively; but still no beauty. Kind a busy these days, but I will pick up.

decodeais commented 4 months ago

I tried to make a pull request for the speedometer software( I have no expearence with this things) . I found that there are other people withe the same problems with the intern typecast errors on the other instrumentts, but nothing seems to happen. Please keep relaxed, for me it works. I dont't want to make stress, I want to help.

WouterJD commented 4 months ago

I assume it can be considered a calling error, do if you help me forward, we can solve in our code

decodeais commented 4 months ago

It is an internal problem in the speedmeter software. The only workaround you can do, is to change the color of the scale from white to black. it looks much better than the white on white.

WouterJD commented 4 months ago

Ok, please suggest code change

decodeais commented 4 months ago

For the Workaround: In FortiusAntGUI.py you can change the SetTicksColour from White to Black. But it looks not good.

More elegant: (speedmeter.py, line 843)

            #currCol = (r1 + rf, g1 + gf, b1 + bf)
            currCol = (r1 + int(rf), g1 +int(gf), b1 +int(bf))
            dc.SetBrush(wx.Brush(currCol))

WouterJD commented 4 months ago

2.. More elegant:

I cannot change this code (other than locally) so I assume I cannot help.

decodeais commented 4 months ago

I made my first pull request (I never done before). The problem was accepted and verified. I hope they will change it as soon as possible. In my repository is a patched version. If you no how to install it from there, you can test it. When I checked the problem, I searched the file on my hard disk and patched the line. But do you know that this is not only a Raspbian problem? It affects all OS. The problem comes from the changes in Python3.10 upwards. The behavior between integer an float in calculations has changed.

WouterJD commented 4 months ago

The behavior between integer an float in calculations has changed.

I my opinion a fundamental design of Python error not to implement strict data typing...

decodeais commented 4 months ago

You don't have to declare the Type, but you have to find out to what it has mutated. I agree.

decodeais commented 4 months ago

I'am afraid that this problem caused some displacements in the GUI. I had to move some Buttons. You should think about fixing the Python version. On the Raspberry it would be possible to fix the OS too. I told you, I would prefer to make a image. We Would have to define: OS version ,Preinstalled apps, Passwords ... Then we could make a installation and compress it like the Kodi or Elec images.
The last problem would be to find a Server, but I think we don`t need much space.

WouterJD commented 4 months ago

Well, not helping the subject but sure fun to read https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

I did not get to the end of this stunning text 😉

decodeais commented 4 months ago

This is what it looks like when a mathematician tries to code

WouterJD commented 4 months ago

Does this effect only appear with "bookworm"? I think I will look into the source code. May be you can give me a hint.

I created this problem with the introduction of the GearboxOverlay window; forgot to close the window. So my version is now OK again; please check whether it OK on your system now.

Issue was caused 2024-01-24 implementing #456, source module FortiusAntGui.py

A basic way to test the user-interface is to run ..\pythoncode\FortiusAntGui.py and check whether it properly ends.

decodeais commented 4 months ago

I did not see a gearbox window when I stopped on Linux.
This would be a better solution to fix the bug, but kill worked too😉. I can test next week.

WouterJD commented 4 months ago

The GearboxOverlay is always created but hidden. It's shown when FortiusAnt is Not on top And gearbox is used

decodeais commented 4 months ago

Good news from wxpyrhon, the speedometer bug ist fixed. I have seen it in the wxpyrhon repository. Then I can both Bugfixes next week.

decodeais commented 4 months ago

I succeeded in compiling the wheels for Raspian and Linux: wxPython-4.2.2a1-cp311-cp311-linux_x86_64.whl wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl I installed them and the scales and numbers of the instruments are visible again. But where can we save such wheels? It would be stupid to compile it allways new. They seem to have a problem there at the wxPython Snapshot Server. There is also an issue open concerning this problem wxWidgets/Phoenix/issues/2530.

WouterJD commented 4 months ago

I will create a folder in FortiusAnt to upload. Any suggestions for a location for "deliverables" where no version control is required?

decodeais commented 4 months ago

It was a hard work, so I hope it realy works. When I used these Wheels thy worked. But I know there are sometimes side effects tat it is only an illusion.

This is my default for <3.10: wxPython-4.2.0-cp39-cp39-linux_armv7l.whl

This was my first test for Raspbian Bookworm with Python3.11, it has still the Speedmeter bug: wxPython-4.2.1-cp311-cp311-linux_aarch64.whl

This is unofficial pre release with bug fix: wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl

Here are the wheels for normal Linux with Python >3.09 wxPython-4.2.2a1-cp310-cp310-linux_x86_64.whl wxPython-4.2.2a1-cp311-cp311-linux_x86_64.whl

I hope somebody can use it and tell me if there is something wrong.

Here is the script i used to generate these wheels. It has to be tested on some fresh systems. Nevertheless its a good starting point for own generated wheels:

#!/bin/bash
# export PATH="$HOME/.local/bin:$PATH"

export DOXYGEN=/usr/bin/doxygen # for Raspian we must use the internal dxygen
# Update package lists
sudo apt-get update
sudo apt install doxygen
# Install required packagesexport PATH="$HOME/.local/bin:$PATH"
sudo apt install -y dpkg-dev build-essential python3-dev freeglut3-dev libgl1-mesa-dev \
libglu1-mesa-dev libgstreamer-plugins-base1.0-dev libgtk-3-dev libjpeg-dev libnotify-dev \
libpng-dev libsdl2-dev libsm-dev libtiff-dev libwebkit2gtk-4.0-dev libxtst-dev \
libgstreamer1.0-dev libjpeg-dev libtiff-dev libsm-dev

# Create directory for building
mkdir -p ~/WheelBuild311

# Change directory to the build directory
cd ~/WheelBuild311

# Create and activate virtual environment
python3.11 -m venv WheelBuilder311_py
source ~/WheelBuild311/WheelBuilder311_py/bin/activate

# Upgrade pip and install required Python packages
pip install -U pip six wheel setuptools sip requests

# Clone Phoenix repository
git clone https://github.com/wxWidgets/Phoenix.git --branch master
cd Phoenix
git submodule init
git submodule update
git submodule update --init --recursive

# Build Phoenix
python build.py dox etg --nodoc sip build bdist_wheel
#python build.py build bdist_wheel

May be it would be necessary to use a virtual machine for linux. A fresh Raspbian should not be a problem.

WouterJD commented 4 months ago

But where can we save such wheels? It would be stupid to compile it allways new.

From the beginning - I was 100% new to python, github and visual studio code - I created the folder structure, including the .exe and .pdf files. Later I learned, github is intended for source-files only and large deliverables should be stored elsewhere; not that github blocks in any way.

The .exe / .pdf files litter the repository (all previous versions are still available) but I see no straight-forward solution.

If anyone reads this and has suggestions - feel free to suggest

For the files you have created, the following approach is good for me:

We write a support page in the wiki with the links
Store the files in the Supportfiles directory.

If you write a pull request with the wheels at the suggested location, we can proceed like that and keep the files available for others. Please give the files a clear name...

And again: If anyone reads this and has suggestions - feel free to suggest

decodeais commented 4 months ago

First you can copy the link, I let them there. Everybody who has this link can use the file. I the link in your installation script is of same kind, I think.

decodeais commented 4 months ago

I think the reason for the problem is that the chief from this project was unable to work for this problem for two years. They restarted the project, made the changes for python 3.11, the tool to generate the wheels does not work for linux and windows at this moment. This will change, I hope.

WouterJD commented 4 months ago

First you can copy the link, I let them there. Everybody who has this link can use the file. I the link in your installation script is of same kind, I think.

Fair enough, easy stuff: https://github.com/WouterJD/FortiusANT/wiki#special-files,

Accessible for everybody who needs.

decodeais commented 4 months ago

Please be strong i have some more question:

I always wondering why pint the cadence meter a darker blue as background color, shouldn't it be "colorTacxFortius".
```
self.Revs.SetFirstGradientColour(wx.BLUE)
```
Did you mention that Linux changed from X11 to Wayland? In my first test I switched the desktop system to X11. The reason was that VNC only works with X11 on Rasbian. On X11 I have seen only little displacements of the buttons. With Wayland the behavier is worse. The meters are cut by the window ( left and bottom) and the gear window is complete confused (stacked). Do you have an idea what is going wrong, wher do you calculate the size of the window?
The install of 3_InstallNodeJs.sh fails, do we really need it?

The good news:

The wheel works in a fresh installation too. I changed the script for Python3.11:

wxPYTHON=wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl
GDRIVE=https://drive.google.com/file/d/1pPrAQLd3plFiuZYifh1HSn_Kr5Iy2EVr/view?usp=sharing
gdown --id 1pPrAQLd3plFiuZYifh1HSn_Kr5Iy2EVr --output $wxPYTHON

I think I understand the virtual envirement a little ( I learned it fighting with the wheels). It is a good way to solve the pip installation problem with the --break-system-packages option.

decodeais commented 4 months ago

I fixed the size problem for Wayland desktop, with the addition of a constant value to the size. I know its not the right way, but it is dificult to find out your strategy of your window size calculation.

self.SetSize (BitmapX + BitmapW + 40, BitmapY + BitmapH+60)
self.panel.SetSize (BitmapX + BitmapW + 40, BitmapY + BitmapH+60)

For the gearbox, I had to change the size too.

frameX = 60      #2 * Margin +     pCranckset.Size[0] + 15
frameY = 200    #3 * Margin + 2 * pCranckset.Size[1] + 39

Now it looks quite normal again.

decodeais commented 4 months ago

Next Problem but fixed. The old autostart can not work. These are working: Autostart_On.sh

#!/bin/bash 
echo "[autostart]" >> ~/.config/wayfire.ini
echo "a0 = lxterminal --working-directory=/home/pi/FortiusANT/raspberry -e \$HOME/FortiusANT/raspberry/FortiusAnt.sh" >> ~/.config/wayfire.ini

Autostart_Off.sh

#!/bin/bash 
sed -i '/\[autostart\]/,/^$/d' ~/.config/wayfire.ini

decodeais commented 4 months ago

I have put my changes to github: https://github.com/decodeais/FortiusANT/tree/Wayland I changed the installation to virtual enviroment for Python. Every thing for FortiusAnts Python is hosted in the FortiusAnt directory. I if the Python version of the Raspi changes,the version for FortiusAnt stays 3.11 . The installation can be started by this script: ./FortiusANT/raspberry/FortiusAntInstall.sh When its ready you can start it in "Others" of the Raspi startmenue. I think I have fixed all problems for me. Next days I will test a fresh installation.

WouterJD commented 4 months ago

but it is dificult to find out your strategy of your window size calculation.

I understand that. I think it's been experimental what looks goed. The +40 and +60 have to with the border sizes, which I did not find anywhere. The constants cause all controls to be same/related shape.

decodeais commented 4 months ago

I took this image 1.1GB Raspbian Bookworm and installed it with my installation script. It worked. I hope somebody can check it. My problem is that the first video output from my Raspi is broken and the Raspi gets incredible hot. I will have to buy a new one.

zapiens commented 4 months ago

Hi, I can check it out. I just got a RP4 and am going to try to set it up.

`Installing collected packages: six, numpy, wxPython

Successfully installed numpy-1.26.4 six-1.16.0 wxPython-4.2.2a1 ......... Reading state information... Done E: Unable to locate package wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl E: Couldn't find any package by glob 'wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl' E: Couldn't find any package by regex 'wxPython-4.2.2a1-cp311-cp311-linux_aarch64.whl' bash: stop.sh: No such file or directory `

decodeais commented 4 months ago

It looks for me that it can not find the stop.sh script. I get this error messages too. When I tested it this morning renamed the FortiusAnt Directory to have a backup. When i started the new installation it stopped at the same position but the reason was a full sd-card. I erased the backup and then everything worked ( i got the same error message but "no stop.sh not found"). if you want to check the wxPython installation :

#!/bin/bash
cd /home/pi/FortiusANT/raspberry
source FortAntEnv/bin/activate #start the virtual envirement
pip list
./stop.sh

Please check the execute permission for stop.sh, it was lost on my Raspi. Remember all Python installations are only valid when you started the virtual enviroment in the terminal:

#!/bin/bash
cd /home/pi/FortiusANT/raspberry
source FortAntEnv/bin/activate #start the virtual enviroment

decodeais commented 4 months ago

I found the reason for the error messages there where 2 installations in the script. One from the download and one stock install with apt-get. The "stop.sh" has now its permission to run.

zapiens commented 4 months ago

I will do a complete new install from a clean sd card.

Do you have have your install script on the Wayland branch?

I noted you mix /home/pi with $HOME the latter is preferable I think as not to force people having the pi user.

decodeais commented 4 months ago

I know, but I have changed so much in this Wayland Branch. My aim is to get a good installation as soon as possible. Next step for me will be to shrink the complete
Sd-card and put it into an image ready to use like the media player images. Compressing last very long, so I must be sure. You can start the script with the parameter n. Then the breaks between the installation blocks should be disabled. Only the stops from the apt-get commands will come, at the first time.

zapiens commented 4 months ago

I saw you made a Zapiens commit. I branched off at that commit to make the changes I thought of but I could not push. I forked the repo instead to try a complete hands free installation.

WouterJD / FortiusANT

New OS for RPI4/5 causing Problems #457