DarkFenX
commented
2 years ago Pinging @wereii, maybe you will understand what's going on, since you have much better expertise on it
Closed DarkFenX closed 2 years ago
DarkFenX
commented
2 years ago Pinging @wereii, maybe you will understand what's going on, since you have much better expertise on it
DarkFenX
commented
2 years ago According to build log difference (between last available working version v2.43.0 and first available broken v2.43.0+11, following python packages changed:
Following packages were built on the older build, and cached versions were used on the newer one:
Order of installation also looked different, but it's too early to look into it so far.
Difference in debian packages installed, the older build:
[00:02:22] Setting up libdbus-1-dev:amd64 (1.12.2-1ubuntu1.2) ...The newer, with a warning when setting it up:
[00:02:41] Setting up libdbus-1-dev:amd64 (1.12.2-1ubuntu1.3) ...
[00:02:42] Setting up dbus (1.12.2-1ubuntu1.3) ...
[00:02:40] Setting up libdbus-1-3:amd64 (1.12.2-1ubuntu1.3) ...
...
[00:02:42] Setting up dbus (1.12.2-1ubuntu1.3) ...
[00:02:42] A reboot is required to replace the running dbus-daemon.
[00:02:42] Please reboot the system when convenient.Seems like it's not python package versions. I tried to install the same versions as the ones which were used in the last known working build on appimage_sandbox branch, using these changes:
iff --git a/.appveyor.yml b/.appveyor.yml
index 4d749aa5c..e96eda454 100644
--- a/.appveyor.yml
+++ b/.appveyor.yml
@@ -14,8 +14,8 @@ for:
PYTHON_APPIMAGE: python3.7.13-cp37-cp37m-manylinux2014_x86_64.AppImage
DEPLOY_DIR: AppDir/opt/pyfa
# APPVEYOR_SSH_BLOCK: true
- cache:
- - /home/appveyor/.cache/pip -> requirements.txt
+# cache:
+# - /home/appveyor/.cache/pip -> requirements.txt
init:
- sh: curl -sflL 'https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-ssh.sh' | bash -e -
install:
@@ -36,7 +36,8 @@ for:
- sh: mkdir -p $DEPLOY_DIR
# run install pyfa packages and any other requirements
- - sh: AppDir/usr/bin/python -s -m pip install -U pip setuptools==41.6.0 wheel pathlib2
+ - sh: AppDir/usr/bin/python -s -m pip install pip==22.0.4 setuptools==41.6.0 wheel pathlib2
+ - sh: AppDir/usr/bin/python -s -m pip install requests==2.27.1 Pillow==9.1.0 kiwisolver==1.4.2 certifi==2021.10.8
- sh: AppDir/usr/bin/python -s -m pip install -r ../requirements.txt
# Run scripts to prep pyfa data and build database
But, new build still has the issue.
brandonholland
commented
2 years ago This is the issue I discovered way before release and so far was unable to fix it, so putting it here. Last version where it worked fine was previous one, v2.43.0 - I downloaded it and it launches just fine. Images starting from at least fe84cce are broken (this commit is the oldest build in AppVeyor, since artifact retention policy removes builds older than 1 month).
Upon launching AppImage build, it seems to launch pyfa.py in an infinite recursion loop. For example, here is
ps auxfor v2.43.0:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND dfx 7923 0.0 0.0 8888 5056 pts/1 S 14:21 0:00 /bin/bash -i /tmp/.mount_pyfa-v9x9nrO/AppRun dfx 7926 6.6 0.0 12148 2396 ? Ssl 14:21 0:01 ./pyfa-v2.43.0-linux.AppImage dfx 7931 0.0 0.0 7512 3056 pts/1 S+ 14:21 0:00 /bin/bash /tmp/.mount_pyfa-v9x9nrO/usr/bin/pyfa dfx 7932 25.4 0.8 1619288 271748 pts/1 Sl+ 14:21 0:05 /tmp/.mount_pyfa-v9x9nrO/usr/bin/python3.7 -s /tmp/.mount_pyfa-v9x9nrO/opt/pyfa/pyfa.pyAnd here is
ps auxfor a broken build (just a few first cycles):USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND dfx 84688 0.5 0.0 8888 4924 pts/3 S 04:50 0:00 /bin/bash -i /tmp/.mount_pyfa-vKkF4uG/AppRun dfx 84691 1.5 0.0 11576 1540 ? Ssl 04:50 0:00 ./pyfa-v2.44.0+1-ga95d69623-linux.AppImage dfx 84696 0.0 0.0 7512 3152 pts/3 S 04:50 0:00 /bin/bash /tmp/.mount_pyfa-vKkF4uG/usr/bin/pyfa dfx 84697 0.5 0.0 8888 5068 pts/3 S 04:50 0:00 /bin/bash -i /tmp/.mount_pyfa-vKkF4uG/usr/bin/python3.7 -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.py dfx 84700 0.0 0.0 7512 3152 pts/3 S 04:50 0:00 /bin/bash /tmp/.mount_pyfa-vKkF4uG/usr/bin/pyfa -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.py dfx 84701 0.5 0.0 8888 5044 pts/3 S 04:50 0:00 /bin/bash -i /tmp/.mount_pyfa-vKkF4uG/usr/bin/python3.7 -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.py -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.py dfx 84704 0.0 0.0 7512 3084 pts/3 S 04:50 0:00 /bin/bash /tmp/.mount_pyfa-vKkF4uG/usr/bin/pyfa -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.py -s /tmp/.mount_pyfa-vKkF4uG/opt/pyfa/pyfa.pyThere were no relevant code changes, so I guess environment changed. v2.43.0 build log, v2.44.0 build log.
So far I don't have any knowledge on how AppImage is built and how to debug it, so will keep posting data to this ticket, and maybe someone will help.
This doesn't affect all builds for Linux does it? I should still be able to build without AppImage? So far, I keep getting stuck on trying to build the wxPython wheel or find one.
I'm on Debian bookworm/testing (because it makes EVE playable on medium settings with mesa 22 on my chromebook)
I've tried virtual environments with default (3.10), 3.9 using apt, and python 3.7(13) manual install, but it keeps erroring out about a symlink. Reading online was saying try changing setuptools version to an older version (I used the version that came with 3.7 confirmed in venv but didnt really change it)
copying symlink wx/libwx_baseu-3.0.so -> build/lib.linux-x86_64-3.7/wx/libwx_baseu-3.0.so
error: [Errno 2] No such file or directory: 'build/lib.linux-x86_64-3.7/wx/libwx_baseu-3.0.so'
[end of output]With 3.7 installed, I can download an ubuntu version whl that seems to install, but am getting errors about .so files
ImportError: /lib/x86_64-linux-gnu/libatk-bridge-2.0.so.0: undefined symbol: atk_component_scroll_to_point
wereii
commented
2 years ago So, I am looking into this. Though because the build of appimage happens in CI only it's hard to try changes locally :) and that Makefile of mine is old by now, not 1:1 to the CI build.
wereii
commented
2 years ago Here is the problem:
I have unpacked the AppImage (./pyfa-appimage_hacks-linux.AppImage --appimage-extract) and somehow the dist_assets/linux/AppRun ends up in the file usr/bin/python3.7, probably replacing the python3.7 binary all together.
Currently I have no clue how but the base python AppImage repository had some activity so maybe the author has changed how the AppRun (entrypoint) is handled.
DarkFenX
commented
2 years ago Seems like wxpython takes a lot of build time on appveryor:
[00:03:52] Building wheels for collected packages: wxPython, logbook, future
[00:59:46] Created wheel for wxPython: filename=wxPython-4.0.6-cp37-cp37m-linux_x86_64.whl size=135262001 sha256=4b82e7d5dab15926f3e6f8bb3dddcb8c803dd50bdf22868b4d6d6d9a749a395fSo can't build wxpython. I will try asking to increase build time on appveyor.
edit: requested it here, will see what's the response would be.
DarkFenX
commented
2 years ago Enabling use of binary builds did not work:
Traceback (most recent call last):
File "/tmp/.mount_pyfa-vyYxgxD/opt/pyfa/pyfa.py", line 27, in <module>
import config
File "/tmp/.mount_pyfa-vyYxgxD/opt/pyfa/config.py", line 4, in <module>
import wx
File "/tmp/.mount_pyfa-vyYxgxD/opt/python3.7/lib/python3.7/site-packages/wx/__init__.py", line 17, in <module>
from wx.core import *
File "/tmp/.mount_pyfa-vyYxgxD/opt/python3.7/lib/python3.7/site-packages/wx/core.py", line 12, in <module>
from ._core import *
ImportError: libjpeg.so.8: cannot open shared object file: No such file or directoryYeah, alternative would be to pre-build wxpy on your own, either locally and upload it somewhere or in CI with self-hosted runner.
DarkFenX
commented
2 years ago Well, I found quite a dirty hack, which will do for now. We have pip cache kept between sessions, so I created separate branch appimg_wx_build where I stripped Ubuntu build of anything but wxpython build (so that it manages to build it just in time, then cache it before it times out). So I launch it to cache build results, then launch master build, and it works just fine.
This is the issue I discovered way before release and so far was unable to fix it, so putting it here. Last version where it worked fine was previous one, v2.43.0 - I downloaded it and it launches just fine. Images starting from at least fe84cce19b0571291f1d82833130e0d6faa503e1 are broken (this commit is the oldest build in AppVeyor, since artifact retention policy removes builds older than 1 month).
Upon launching AppImage build, it seems to launch pyfa.py in an infinite recursion loop. For example, here is
ps auxfor v2.43.0:And here is
ps auxfor a broken build (just a few first cycles):There were no relevant code changes, so I guess environment changed. v2.43.0 build log, v2.44.0 build log.
So far I don't have any knowledge on how AppImage is built and how to debug it, so will keep posting data to this ticket, and maybe someone will help.