katdal does not work with python 3.12

[spassmoor]

When opening a rdb file with katdal in python 3.12 I get the error

    173 def load_from_file(self, file: Union[_PathType, BinaryIO]) -> int:
    174     if rdb_reader is None:
--> 175         raise _rdb_reader_import_error   # noqa: F821
    176     with self._write():
    177         return rdb_reader.load_from_file(MemoryCallback(self._data), file)

NameError: name `_rdb_reader_import_error' is not defined

This is caused by katsdptelstate trying to import distutils in a try..except block giving the error ModuleNotFoundError: No module named 'distutils' that is not reported. distutils was Deprecate in python 3.10 and removed in python 3.12 https://peps.python.org/pep-0632/

I needed to install setuptools to fix this.

[ludwigschwardt]

Thanks, Sean. This was already fixed in katsdptelstate 0.13 🙂 Maybe I'll add a minimum requirement in the case of Python 3.12 somewhere...

[spassmoor]

In [4]: import katsdptelstate.rdb_reader
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[4], line 1
----> 1 import katsdptelstate.rdb_reader

File ~/ska-sa/katsdptelstate/katsdptelstate/rdb_reader.py:21
     18 import os.path
     19 from typing import Optional, Union, BinaryIO, cast
---> 21 from rdbtools import RdbParser, RdbCallback
     23 from .errors import RdbParseError
     24 from .utils import _PathType

File /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rdbtools/__init__.py:3
      1 from rdbtools.parser import RdbCallback, RdbParser, DebugCallback
      2 from rdbtools.callbacks import JSONCallback, DiffCallback, ProtocolCallback, KeyValsOnlyCallback, KeysOnlyCallback
----> 3 from rdbtools.memprofiler import MemoryCallback, PrintAllKeys, StatsAggregator, PrintJustKeys
      5 __version__ = '0.1.15'
      6 VERSION = tuple(map(int, __version__.split('.')))

File /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rdbtools/memprofiler.py:6
      4 import random
      5 import bisect
----> 6 from distutils.version import StrictVersion
      7 try:
      8     import ujson as json

ModuleNotFoundError: No module named 'distutils'

[bennahugo]

May want to replace the latter with 'packaging' package. Distutils is dead

On Wed, Apr 17, 2024, 18:21 Sean Passmoor @.***> wrote:

In [4]: import katsdptelstate.rdb_reader

ModuleNotFoundError Traceback (most recent call last) Cell In[4], line 1 ----> 1 import katsdptelstate.rdb_reader

File ~/ska-sa/katsdptelstate/katsdptelstate/rdb_reader.py:21 18 import os.path 19 from typing import Optional, Union, BinaryIO, cast ---> 21 from rdbtools import RdbParser, RdbCallback 23 from .errors import RdbParseError 24 from .utils import _PathType

File /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rdbtools/init.py:3 1 from rdbtools.parser import RdbCallback, RdbParser, DebugCallback 2 from rdbtools.callbacks import JSONCallback, DiffCallback, ProtocolCallback, KeyValsOnlyCallback, KeysOnlyCallback ----> 3 from rdbtools.memprofiler import MemoryCallback, PrintAllKeys, StatsAggregator, PrintJustKeys 5 version = '0.1.15' 6 VERSION = tuple(map(int, version.split('.')))

File /Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/rdbtools/memprofiler.py:6 4 import random 5 import bisect ----> 6 from distutils.version import StrictVersion 7 try: 8 import ujson as json

ModuleNotFoundError: No module named 'distutils'

— Reply to this email directly, view it on GitHub https://github.com/ska-sa/katdal/issues/379#issuecomment-2061702016, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4RE6WUUDF7ONUPFZRGHTLY52OQPAVCNFSM6AAAAABGLCUSICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRRG4YDEMBRGY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[ludwigschwardt]

The problem actually lies with rdbtools: a dependency of a dependency of katdal 🤦🏾‍♂️ And rdbtools is also practically dead, with the last update in 2020.

For now this can be worked around by installing setuptools, as Sean said (which is why I missed this - I always install that!).

Longer term I'm looking for a replacement, as documented in SPR1-3073.