FreeCAD / FreeCAD

This is the official source code of FreeCAD, a free and opensource multiplatform 3D parametric modeler.
https://www.freecad.org
Other
19.38k stars 4.01k forks source link

[Problem] Addon Manager fails to display module README.md #8055

Closed g3grau closed 1 year ago

g3grau commented 1 year ago

Is there an existing issue for this?

Version

0.21 (Development)

Full version info

Builds tested: 
FreeCAD_weekly-builds-0-2022-12-15-conda-Linux-x86_64-py310.AppImage
and FreeCAD_0.20-29177-2022-06-15-conda-Linux-x86_64-py39.AppImage
System: openSuse Tumbleweed (and a comparision to openSuse 12.3)

[code]
OS: openSUSE Tumbleweed (KDE//usr/share/xsessions/plasma5)
Word size of FreeCAD: 64-bit
Version: 0.21.0.31328 (Git) AppImage
Build type: Release
Branch: master
Hash: 9e702c016bcc6318ea35a69437241ae67f92524d
Python 3.10.8, Qt 5.15.4, Coin 4.0.0, Vtk 9.1.0, OCC 7.6.3
Locale: C/Default (C)
Installed mods: 
  * Silk 1.0.0
  * sheetmetal 0.2.59
  * kicadStepUpMod 10.16.5
  * CfdOF 1.20.2
  * Assembly4 0.12.4
  * fasteners 0.4.23
  * offline-documentation 1.0.0-alpha
  * MeshRemodel 1.8919.0
  * LCInterlocking
  * 3D_Printing_Tools
  * A2plus 0.4.60d
  * 3DfindIT 1.2.0
  * EM
  * ExplodedAssembly
  * ThreadProfile 1.84.0
  * Lithophane
  * Manipulator 1.4.9
  * Assembly3 0.11.4
  * CurvedShapes 1.0.4
  * Curves 0.6.1
  * frame 0.1.0
  * FEM_FrontISTR 0.1.0
  * fcgear 1.0.0
  * OpticsWorkbench 1.0.9
  * CommandPanel
[/code]

Subproject(s) affected?

None

Problem description

From 0.19 to 0.21 weekly build I face the same issue that the display manager reports "Could not load README data from URL". This happens on any addon I select. For some the message differs slightly, e.g. "Could not load README data from URL data:text/html;charset=UTF-8,%3Chtml%3E%3Cbody%3E%3Cp%3ECould not load README data from URL %3C%2Fp%3E%3C%2Fbody%3E%3C%2Fhtml%3E". The addon manager properly shows the list of all addons with their short info, and from the currently selected addon I can also install the addon. It's just that it doesn't want to show the description.

Sometimes I get an exception in the command line (I can't tell why it's not very repeatable): Again, this is not specific to a particular addon!

Failed to clone https://github.com/mwganson/ThreadProfile into /home/gg/.local/share/FreeCAD/Mod/ThreadProfile using gitTraceback (most recent call last): File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_git.py", line 368, in _synchronous_call_git proc = subprocess.run( File "/tmp/.mount_freecaUk7Urb/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/git', 'clone', '--recurse-submodules', 'https://github.com/mwganson/ThreadProfile', '/home/gg/.local/share/FreeCAD/Mod/ThreadProfile']' returned non-zero exit status 127.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 107, in run self.run_git(target_dir) File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 141, in run_git self.run_git_update(clonedir) File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 151, in run_git_update self.git_manager.repair(self.repo.url, clonedir) File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_git.py", line 232, in repair raise e File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_git.py", line 224, in repair self.clone(remote, local_path) File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_git.py", line 79, in clone self._synchronous_call_git(final_args) File "/tmp/.mount_freecaUk7Urb/usr/Mod/AddonManager/addonmanager_git.py", line 375, in _synchronous_call_git raise GitFailed(str(e)) from e addonmanager_git.GitFailed: Command '['/usr/bin/git', 'clone', '--recurse-submodules', 'https://github.com/mwganson/ThreadProfile', '/home/gg/.local/share/FreeCAD/Mod/ThreadProfile']' returned non-zero exit status 127.

Anything else?

For completeness I want to mention some other observations - they may or may not be relevant for this issue:

A) A3 successfully installed a Python dependency (py-slvs-1.0.3) but then throws an exception. After kill/restart I get "A3 available". Don't know yet if A3 actually works. This one is repeatable here (uninstall/reinstall A3), so it may be helpful as it is also related to "external" github operations: "Noneb'pip 22.3.1 from /tmp/.mount_freecaGmHoBh/usr/lib/python3.10/site-packages/pip (python 3.10)\n'pip 22.3.1 from /tmp/.mount_freecaGmHoBh/usr/lib/python3.10/site-packages/pip (python 3.10) Collecting py-slvs Using cached py_slvs-1.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (294 kB) Installing collected packages: py-slvs Successfully installed py-slvs-1.0.3 Failed to clone https://github.com/realthunder/FreeCAD_assembly3 into /home/gg/.local/share/FreeCAD/Mod/Assembly3 using gitTraceback (most recent call last): File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_git.py", line 368, in _synchronous_call_git proc = subprocess.run( File "/tmp/.mount_freecaGmHoBh/usr/lib/python3.10/subprocess.py", line 526, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/git', 'clone', '--recurse-submodules', 'https://github.com/realthunder/FreeCAD_assembly3', '/home/gg/.local/share/FreeCAD/Mod/Assembly3']' returned non-zero exit status 127.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 107, in run self.run_git(target_dir) File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 141, in run_git self.run_git_update(clonedir) File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 151, in run_git_update self.git_manager.repair(self.repo.url, clonedir) File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_git.py", line 232, in repair raise e File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_git.py", line 224, in repair self.clone(remote, local_path) File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_git.py", line 79, in clone self._synchronous_call_git(final_args) File "/tmp/.mount_freecaGmHoBh/usr/Mod/AddonManager/addonmanager_git.py", line 375, in _synchronous_call_git raise GitFailed(str(e)) from e addonmanager_git.GitFailed: Command '['/usr/bin/git', 'clone', '--recurse-submodules', 'https://github.com/realthunder/FreeCAD_assembly3', '/home/gg/.local/share/FreeCAD/Mod/Assembly3']' returned non-zero exit status 127.

At the end I will have share/FreeCAD/Mod/Assembly3/ and another share/FreeCAD/Mod/Assembly3.backupNNNNN/ folders. The backup folder is empty, the other one has .github/, freecad/ and .git/ subfolders as well as the Readme.md which is not displayed by the addon manager

B) Since FreeCAD is also an AppImage, maybe another problem I have with Ultimaker Cura might be related: Some functions in cura (such as sign-in) fail with a message "/bin/sh: /tmp/.mount_curaSUObPu/libreadline.so.8: no version information available (required by /bin/sh) /bin/sh: symbol lookup error: /bin/sh: undefined symbol: rl_trim_arg_from_keyseq, version READLINE_8.2". I could also not resolve this one yet. I haven't seen this message anywhere around FreeCAD (also not if logged with with -l), but since other errors also don't show up reliably in FC I wanted to mention it.

The above message was the direct cause of the following exception:

Traceback (most recent call last): File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 107, in run self.run_git(target_dir) File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 141, in run_git self.run_git_update(clonedir) File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_workers_installation.py", line 151, in run_git_update self.git_manager.repair(self.repo.url, clonedir) File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_git.py", line 232, in repair raise e File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_git.py", line 224, in repair self.clone(remote, local_path) File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_git.py", line 79, in clone self._synchronous_call_git(final_args) File "/tmp/.mount_freecacXkyoM/usr/Mod/AddonManager/addonmanager_git.py", line 375, in _synchronous_call_git raise GitFailed(str(e)) from e addonmanager_git.GitFailed: Command '['/usr/bin/git', 'clone', '--recurse-submodules', 'https://github.com/execuc/LCInterlocking', '/home/gg/.local/share/FreeCAD/Mod/LCInterlocking']' returned non-zero exit status 127. " After manual execution of "/usr/bin/git clone --recurse-submodules https://github.com/execuc/LCInterlocking /home/gg/.local/share/FreeCAD/Mod/LCInterlocking" the module is installed and works. This happens permanently on systems with an older OS (e.g. openSuse 12.x), but happens rarely on tumbleweed. Everything else works, except the installation from the addon manager (both 0.20 and 0.21 weekly affected).

Somehow all of these problems seem to be related to operations on github from within FC, and all of these would work fine from the shell in which I started FC. That's why I think that it might be somewhat related to the issue in the cura AppImage which refuses some "remote" stuff. Is that some strange permission issue with AppImages, maybe?

Yet another fresh observation in 0.21 weekly:

Code of Conduct

chennes commented 1 year ago

Try forcing the Addon Manager to use ZIP downloads instead of git -- it sounds like there's an issue with the way git is working from within your AppImage. You can set that in the Preferences.

g3grau commented 1 year ago

Hi chennes, thanks for this hint! Changing to zip solves the A3 installation issue. I'll check that on the older OS next week, I guess this will be also solved by that.

The issue of AM not displaying the Readme.md is however not affected by that. Checking the log (started with -l) again, there is no trace of a problem, only the Addon Manager itself displays "Could not load README data from URL". The URL at the bottom of the window is however valid, I can get it with curl.

chennes commented 1 year ago

Can you get it from the FreeCAD Python console (using urllib.request for example)?

g3grau commented 1 year ago

Yes:

import urllib
url='https://github.com/mark1791/3D_Printing_Tools/blob/master/README.md'
readme=urllib.request.urlopen(url)
text=readme.read()
print(text)

works as expected

chennes commented 1 year ago

Thanks: now can you try:

url='https://github.com/mark1791/3D_Printing_Tools/blob/master/README.md'
import NetworkManager
NetworkManager.InitializeNetworkManager()
text = NetworkManager.AM_NETWORK_MANAGER.blocking_get(url)
print(text)
g3grau commented 1 year ago

...although it doesn't look like the readme itself, it starts with b'\n\n\n\n\n\n\n<!DOCTYPE html>\n\n \n \n \n \n \n \n \n \n\n\n\n <link ...

That corresponds to a HTML page which shows the actual readme file embedded. The same happens if I open this URL in a browser. Likely the parser fails to get handle this extra HTML?

g3grau commented 1 year ago

same with NetworkManager. Works as expected, but the result is HTML ?!

chennes commented 1 year ago

It should be HTML -- all of these tests are sort of skirting around the issue, the widget there is actually a QWebEngineWidget, which is actually a wrapper around Chromium. It performs its own network accesses, so we can't really test it directly.

g3grau commented 1 year ago

hm, import PyQt5 or PyQt6 inside FC fails. Doesn't seem to be installed in the AppImage?

I found a sample project 15-minute-apps-master/browser which uses PyQt5.QtWebEngineWidgets and I believe that it did work a year ago .. but right now it claims that I don't have internet. That's funny! I found another code snippet with PySide6 that works. I can't load PySide6 in FC, but PySide (which is a V5). Using an ugly test code I can see a similar effect here: in the shell using PySide6 it works, in FC (removing the 6 and the two lines for the app start/exit) it claims to have no internet. So the issue is in the QtWebEngine of PyQt5?? Not sure why it doesn't work anymore, maybe related to some IPv6 stuff?

Here's the (very dirty) test code. It will open two windows and load two URLs. PySide6 works (from the shell), PySide (5) claims "No internet. There's something wrong with the proxy server, or the address is incorrecr. ...". Different message, likely the same problem.

import sys
# remove the 6 if you load this inside FC
from PySide6.QtCore import QUrl
from PySide6.QtWidgets import QApplication, QMainWindow, QDialog
from PySide6.QtWebEngineWidgets import QWebEngineView

class MainWindow(QMainWindow):

    def popupBrowser(self):
        w = QDialog(self)
        w.resize(600, 500)
        w.setWindowTitle('test1')
        web = QWebEngineView(w)
        web.load(QUrl('https://google.com'))
        web.resize(600, 500)
        web.show()
        w.show() 

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.setWindowTitle('test2')
        self.resize(800, 600)
        self.setFixedSize(self.width(), self.height())
        url = 'https://spiegel.de' 
        self.browser = QWebEngineView()
        self.browser.load(QUrl(url))
        self.setCentralWidget(self.browser) 

# main:
app = QApplication(sys.argv)  # if standalone -- comment out in FC
mainWin = MainWindow()
availableGeometry = mainWin.screen().availableGeometry()
mainWin.resize(availableGeometry.width() * 2 / 3, availableGeometry.height() * 2 / 3)
mainWin.show()
mainWin.popupBrowser()
sys.exit(app.exec())  # if standalone -- comment out in FC

From

import PySide
PySide.__version__

I get '5.15.5' which I guess corresponds to Qt5? So the failure if pyside5 and pyqt5 likely has the same origin. I don't really understand this IPv6 stuff, but I disabled that quite long and now it is running, wlo1 has a local IPv4 and 3 IPv6 addresses. I'll try to find out if that's the problem

chennes commented 1 year ago

The FreeCAD AppImage definitely won't include PyQt*, we use PySide2 (currently, support for PySide6 is in progress). Are you behind a Proxy?

g3grau commented 1 year ago

no proxy, I verified this by checking for potential environment settings. My laptop has never seen any proxy configuration. And the same happens at home and in my office in completely different environments. I've never seen anything in this AM window but "could not load". And essentially the same code with PyQt6/PySide6 works while PyQt5/PySide fails (even outside of FC). I also tried it in a conda environment, same effect... that's really crazy. At least the problem is not specific to FC but rather to PyQt as this test revealed.

I just disabled IPv6 for my connection and verified that there is none assigned. This had no impact :-/ Though, I remember that just the presence of IPv6 capability made a bunch of hard to trace problems in the far past, thus this test may not be valid.

I tried to search for this issue around QtWebEngineWidgets but with little results. Hope the PySide6 version is getting ready soon :)

Thanks for your help!

g3grau commented 1 year ago

Argh! Problem solved in a very strange way! I had to change google-chrome settings!

https://lists.qt-project.org/pipermail/qtwebengine/2017-July/000458.html had the solution. The Qt5 app somehow uses the settings for google-chrome ?! I rarely ever use that. Chrome settings were set to "Use system proxy" followed by a list of empty fields except SOCKS proxy which was set to socks5://localhost:1080

After setting google-chrome to "no proxy", the Qt5 browser code works and also the Addon manager reads the readme.md !!! Switching back chrome to "system settings", AM fails again (even in the same running instance of FC: the setting is checked each time the AM window is opened).

How's that? System setting (yast2) has proxy disabled (and the GUI doesn't even contain SOCKS settings). The environment has zero proxy settings. I made a grep in /etc for this setting with no result. Also in my home I couldn't find such setting yet. Where does the SOCKS5 stuff come from?

Thanks again for your help, I'd never discovered that without!

chennes commented 1 year ago

Nice work figuring it out, this was a challenging one. Thanks for the report, I'll file this away for future reference in case anyone else reports a similar issue.

windsurf11 commented 1 year ago

Hi All,

i get the following error. Tried changing preferences as suggested but no luck. Any thoughts?

error: Could not load README data from URL https://github.com/yorikvanhavre/BIM_Workbench/blob/master/README.md

luzpaz commented 1 year ago

@windsurf11 full About info please

windsurf11 commented 1 year ago

sorry, but i don't get the About option in my help pull down window. I am running FreeCAD on a MAC laptop. There must be something I am not doing correctly considering my use case is typical.

Any comments/suggestions are appreciated.

JohnOCFII commented 1 year ago

sorry, but i don't get the About option in my help pull down window.

On a Mac, the About information is in the menu to the far left (next to the Apple Menu - the one with the name of the Application - usually, in this case, FreeCAD).