search

benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
674 stars 42 forks source link

xidel is loading libcrypto in an unsafe way #69

Closed CornPrincess closed 3 years ago

CornPrincess commented 3 years ago

Hi, everyone I met problem just like title. I have downloaded "xidel-0.9.8.src.tar.gz" from the official website, and I install it on "/usr/local/xidel_repo" with this method(https://www.evernote.com/shard/s69/client/snv?noteGuid=ff1e78f3-a369-4855-b18f-6184ce789c45&noteKey=f3511927d0fb356ce883835f2eb712e0&sn=https%3A%2F%2Fwww.evernote.com%2Fshard%2Fs69%2Fsh%2Fff1e78f3-a369-4855-b18f-6184ce789c45%2Ff3511927d0fb356ce883835f2eb712e0&title=Xidel%253A%2BCompiling%2Bon%2BmacOS%2B-%2Bpublic%2Binstructions%252C%2Bprebuilt%2Bbinaries)

My MacOS is Big Sur 11.2.2, can anyone offer some advice

benibela commented 3 years ago

This patch might help: https://forum.lazarus.freepascal.org/index.php/topic,48423.msg390100.html#msg390100
But it conflicts with my Linux OpenSSL loading patch.

Perhaps it is easier to replace libssl.dylib and libcrypto.dylib in ssl_openssl_lib.pas with the filenames including the correct version number for your system

Or install OpenSSL separately

ghislainfourny commented 3 years ago

Happy to confirm that replacing libssl.dylib and libcrypto.dylib in ssl_openssl_lib.pas with (in my case) libssl.0.9.8.dylib and libcrypto.0.9.8.dylibfixes the issue.

CornPrincess commented 3 years ago

Thanks for all your comments, I solved it by this method: https://dev.to/ruivieira/fixing-libcrypto-ansible-crashes-on-macos-mm2

stessaris commented 8 months ago

I'd suggest to reopen the issue, because there's definitely a problem with the loading of the libssl/libcrypto dynamic libraries, which is worsened by the fact that distributions are moving to openssl 3 and installing the older version could be tricky.

The crash on OSX is due to the fact that the files linked to libssl.dylib and libcrypto.dylib are just stubs that abort on purpose because of the change in ABI between 1.x and 3.x. Using Homebrew the 1.x version of the libraries can be installed alongside the new version (it's just not linked in the standard path) and can be used by specifying the load path:

$ DYLD_LIBRARY_PATH="$(brew --prefix openssl@1.1)/lib:${DYLD_LIBRARY_PATH}" ./xidel -s https://github.com/benibela/xidel -e "//title"
GitHub - benibela/xidel: Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching.  It can also create new or transformed XML/HTML/JSON documents.

The above works on OSX, I guess that on Linux would be the same using LD_LIBRARY_PATH.

Recent Linux distributions are moving to openssl 3.x as well, e.g. Ubuntu moved to 3.x from 22.04 and it's not easy to install the older libraries alongside without resorting to tools like Homebrew.

Aside, I noticed that FreePascal already supports OpenSSL 3.0 (see issue #39659) and maybe their code (see packages/openssl/src/openssl.pas) can be backported to the relevant code in ararat-synapse/ssl_openssl_lib.pas.

As mentioned in the FPC issue, for the time being it might be enough to just load the 3.x version of the libraries.

stessaris commented 8 months ago

BTW I noticed that the Homebrew formula of Xidel uses openssl 3 since a couple of years