Python 3.10 package missing libz

[paulreimer]

I'm excited to start using the eyberg/python:3.10.6 image, but I noticed that I cannot import base64; I believe this is because libz.so.1 is missing from the package image.

(eyberg/python:3.9.7 has it:)

ops pkg contents eyberg/python:3.9.7 | grep libz
File :/sysroot/lib/x86_64-linux-gnu/libz.so.1

Personally I'd also like to see librt.so.1 as well (a dependency I use needs it, but doesn't use it AFAICT); none of the existing python packages have that but I wish they did!

[eyberg]

I went ahead and re-pkg'd w/those 2 missing libs; generally if you are targeting the same libc you'll be able to add them locally but if not you can make your own pkg as described here: https://docs.ops.city/ops/packages#creating-a-custom-or-local-dev-packages - you can also import from a docker image

most of the packages you find on the repo are more of a reference/example package and by no means authoritative, for simple packages they are pretty easy to reproduce but in larger ones (like the python ones) it can be hard to figure out what is missing

one quick trick to look for (espc. dynamically loaded libs) is to turn on trace ala, ops pkg load -l python_3.10.6 -a hi.py --trace &>/tmp/o and then you can grep for libraries that might get loaded at run-time that aren't being found - for other static files --missing-files comes in handy

if you want to grab everything your local interpreter might request then this snippet would grab that:

 find ~/.ops/packages/eyberg/python_3.10.6/sysroot -name "*.so" -exec readelf -d {} \; | grep NEEDED | awk '{print $5}' | sort | uniq

[paulreimer]

Thanks! -- that's helpful.

The next one needed seems to be libssl.so.1.1

[eyberg]

went ahead and added that as well, if you have a snippet of code or project you are trying to load that might iron out what all you need

[paulreimer]

Sure, and thanks! (the next one needed is libcrypto.so.1.1)

The list of imports I'm using is:

import asyncio
import base64
import dataclasses
import datetime
import hashlib
import hmac
import io
import json
import os
import random
import socket
import ssl
import string
import sys
import time
import traceback
import typing

(I'm also using aiohttp with a venv, but I don't think it needs any extra libs)

[eyberg]

ok - I don't get any error msgs w/these imports w/the latest pkg i uploaded

[paulreimer]

OK, I think there may be one more (needed by a private/confidential .so file I am importing): libpthread.so.0.

Sorry for the hassle; I am excited for using 3.10 and that I'll be able to use it for many things in the future!

[eyberg]

I went ahead and included that, all of this that you've requested is fairly standard so I'm not surprised you're finding them

[paulreimer]

Working well now! Thanks so much!

I needed to add a /etc/ssl/cert.pem file for verifying certificates (I used the one from the certifi package, which is Mozilla's default CA root store IIUC); not sure if that would be a useful addition to the default package... (the previous python packages needed it as /usr/lib/ssl/cert.pem)

[eyberg]

nah that makes sense, we auto-include /etc/ssl/certs in almost everything but i added this to the python pkg and this test seems to work now:

import urllib.request
r = urllib.request.urlopen("https://paypal.com")
print(r.read())

[paulreimer]

Confirmed -- working great over here!

[paulreimer]

FWIW, here are a couple other system libraries that I've found useful (when using Python libraries which use native Linux libraries):

• libdl.so.2
• libffi.so.8

[eyberg]

added those for you