search

facebook / Ax

Adaptive Experimentation Platform
https://ax.dev
MIT License
2.38k stars 311 forks source link

Ax is incompatible with SQLAlchemy 2.0+; incompatibility errors arise when using only JSON storage, too #1697

Closed ekurtgl closed 4 months ago

ekurtgl commented 1 year ago

Hi,

I am tryinng to save the NAS experiment presented here to a file, so that I can load it later on to do further analysis and visualization based on the results. I use the following code:

import ax.storage ax.storage.json_store.save.save_experiment(experiment, 'ax_nas_results.json')

However, it throws error:

`

JSONEncodeError Traceback (most recent call last) Cell In[71], line 2 1 import ax.storage ----> 2 ax.storage.json_store.save.save_experiment(experiment, 'ax_nas_results.json') 3 # dir(ax.storage)

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/save.py:45, in save_experiment(experiment, filepath, encoder_registry, class_encoder_registry) 42 if not filepath.endswith(".json"): 43 raise ValueError("Filepath must end in .json") ---> 45 json_experiment = object_to_json( 46 experiment, 47 encoder_registry=encoder_registry, 48 class_encoder_registry=class_encoder_registry, 49 ) 50 with open(filepath, "w+") as file: 51 file.write(json.dumps(json_experiment))

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type ---> 83 return { 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type 83 return { ---> 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type ---> 83 return { 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type 83 return { ---> 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type ---> 83 return { 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type 83 return { ---> 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:96, in object_to_json(obj, encoder_registry, class_encoder_registry) 94 return obj 95 elif _type is list: ---> 96 return [ 97 object_to_json( 98 x, 99 encoder_registry=encoder_registry, 100 class_encoder_registry=class_encoder_registry, 101 ) 102 for x in obj 103 ] 104 elif _type is tuple: 105 return tuple( 106 object_to_json( 107 x, (...) 111 for x in obj 112 )

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:97, in (.0) 94 return obj 95 elif _type is list: 96 return [ ---> 97 object_to_json( 98 x, 99 encoder_registry=encoder_registry, 100 class_encoder_registry=class_encoder_registry, 101 ) 102 for x in obj 103 ] 104 elif _type is tuple: 105 return tuple( 106 object_to_json( 107 x, (...) 111 for x in obj 112 )

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type ---> 83 return { 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0) 81 if _type in encoder_registry: 82 obj_dict = encoder_registry_type 83 return { ---> 84 k: object_to_json( 85 v, 86 encoder_registry=encoder_registry, 87 class_encoder_registry=class_encoder_registry, 88 ) 89 for k, v in obj_dict.items() 90 } 92 # Python built-in types + typing module types 93 if _type in (str, int, float, bool, type(None)):

File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:185, in object_to_json(obj, encoder_registry, class_encoder_registry) 178 return {"type": f"torch_{_type.name__}", "value": torch_type_to_str(obj)} 180 err = ( 181 f"Object {obj} passed to object_to_json (of type {_type}, module: " 182 f"{_type.module}) is not registered with a corresponding encoder " 183 "in ENCODER_REGISTRY." 184 ) --> 185 raise JSONEncodeError(err)

JSONEncodeError: Object MyTensorboardMetric('val_acc') passed to object_to_json (of type , module: main) is not registered with a corresponding encoder in ENCODER_REGISTRY `

As far as I understand, I need to encode my metrics, but I am not sure how to do it. Any ideas on this? Is this the best way to export the experiments after the optimization process is done? Thank you.

fernexda commented 1 year ago

I have the same issue. When I use save_experiment(experiment, "test.json") it gives: {JSONEncodeError}JSONEncodeError("Object MetricCp('Cp') passed to object_to_json (of type <class '__main__.MetricCp'>, module: __main__) is not registered with a corresponding encoder in ENCODER_REGISTRY.")

Am I missing something? Thank you very much for your help.

mpolson64 commented 1 year ago

Sorry the two of you are running into this issue -- this process could certainly better documented in our tutorials.

In order to encode custom metrics and runners you must ensure they are included in Ax's storage registries. These are objects that tell Ax how to handle the encoding and decoding of objects. We have a helper abstraction called the RegistryBundle that handles all this for you under the hood; see it in use in the following snippet: https://ax.dev/tutorials/scheduler.html#6.-Leveraging-SQL-storage-and-experiment-resumption

@ekurtgl for you the corrected code will look like this:

bundle = RegistryBundle(
    metric_clss={MyTensorboardMetric: None},
    runner_clss={}
)

save_experiment(
    experiment=experiment,
    filepath='ax_nas_results.json',
    encoder_registry=bundle.encoder_registry,
    class_encoder_registry=bundle.class_encoder_registry,
)

@fernexda yours will look the same as above but with MetricCp rather than MyTensorboardMetric. Let me know if this helped or if you have any other questions.

ekurtgl commented 1 year ago

Thank you for your help @mpolson64 , however the below code throws the following error: image

---------------------------------------------------------------------------
ArgumentError                             Traceback (most recent call last)
Cell In[18], line 2
      1 from ax.storage.json_store.save import save_experiment
----> 2 from ax.storage.registry_bundle import RegistryBundle
      4 bundle = RegistryBundle(
      5     metric_clss={MyTensorboardMetric: None},
      6     runner_clss={}
      7 )
      9 save_experiment(experiment=experiment,
     10                 filepath='ax_nas_mnist.json',
     11                 encoder_registry=bundle.encoder_registry,
     12                 class_encoder_registry=bundle.class_encoder_registry,)

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/registry_bundle.py:21](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/registry_bundle.py:21)
     19 from ax.storage.metric_registry import register_metrics
     20 from ax.storage.runner_registry import register_runners
---> 21 from ax.storage.sqa_store.decoder import Decoder
     22 from ax.storage.sqa_store.encoder import Encoder
     23 from ax.storage.sqa_store.sqa_config import SQAConfig

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/__init__.py:9](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/__init__.py:9)
      1 #![/usr/bin/env](https://file+.vscode-resource.vscode-cdn.net/usr/bin/env) python3
      2 # Copyright (c) Meta Platforms, Inc. and affiliates.
      3 #
   (...)
      7 # necessary to import this file so SQLAlchemy knows about the event listeners
      8 # see https://fburl.com/8mn7yjt2
----> 9 from ax.storage.sqa_store import validation
     10 from ax.storage.sqa_store.load import load_experiment as sqa_load
     11 from ax.storage.sqa_store.save import save_experiment as sqa_save

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/validation.py:11](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/validation.py:11)
      8 from typing import Any, Callable, List, TypeVar
     10 from ax.storage.sqa_store.db import SQABase
---> 11 from ax.storage.sqa_store.reduced_state import GR_LARGE_MODEL_ATTRS
     12 from ax.storage.sqa_store.sqa_classes import (
     13     ONLY_ONE_FIELDS,
     14     ONLY_ONE_METRIC_FIELDS,
   (...)
     18     SQARunner,
     19 )
     20 from ax.utils.common.logger import get_logger

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/reduced_state.py:9](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/reduced_state.py:9)
      1 #![/usr/bin/env](https://file+.vscode-resource.vscode-cdn.net/usr/bin/env) python3
      2 # Copyright (c) Meta Platforms, Inc. and affiliates.
      3 #
      4 # This source code is licensed under the MIT license found in the
      5 # LICENSE file in the root directory of this source tree.
      7 from typing import List
----> 9 from ax.storage.sqa_store.sqa_classes import SQAGeneratorRun
     10 from sqlalchemy.orm import defaultload, lazyload, strategy_options
     11 from sqlalchemy.orm.attributes import InstrumentedAttribute

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/sqa_classes.py:231](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/sqa_store/sqa_classes.py:231)
    227     # pyre-fixme[8]: Attribute has type `int`; used as `Column[int]`.
    228     trial_id: int = Column(Integer, ForeignKey("trial_v2.id"))
--> 231 class SQAGeneratorRun(Base):
    232     __tablename__: str = "generator_run_v2"
    234     # pyre-fixme[8]: Attribute has type `Optional[str]`; used as `Column[str]`.

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_api.py:194](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_api.py:194), in DeclarativeMeta.__init__(cls, classname, bases, dict_, **kw)
    191         cls._sa_registry = reg
    193 if not cls.__dict__.get("__abstract__", False):
--> 194     _as_declarative(reg, cls, dict_)
    195 type.__init__(cls, classname, bases, dict_)

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:247](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:247), in _as_declarative(registry, cls, dict_)
    242 def _as_declarative(
    243     registry: _RegistryType, cls: Type[Any], dict_: _ClassDict
    244 ) -> Optional[_MapperConfig]:
    245     # declarative scans the class for attributes.  no table or mapper
    246     # args passed separately.
--> 247     return _MapperConfig.setup_mapping(registry, cls, dict_, None, {})

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:328](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:328), in _MapperConfig.setup_mapping(cls, registry, cls_, dict_, table, mapper_kw)
    324     return _DeferredMapperConfig(
    325         registry, cls_, dict_, table, mapper_kw
    326     )
    327 else:
--> 328     return _ClassScanMapperConfig(
    329         registry, cls_, dict_, table, mapper_kw
    330     )

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:563](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:563), in _ClassScanMapperConfig.__init__(self, registry, cls_, dict_, table, mapper_kw)
    559 self.allow_dataclass_fields = bool(sdk and cld)
    561 self._setup_declared_events()
--> 563 self._scan_attributes()
    565 self._setup_dataclasses_transforms()
    567 with mapperlib._CONFIGURE_MUTEX:

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:1006](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:1006), in _ClassScanMapperConfig._scan_attributes(self)
   1002     self._collect_annotation(
   1003         name, annotation, base, False, obj
   1004     )
   1005 else:
-> 1006     collected_annotation = self._collect_annotation(
   1007         name, annotation, base, None, obj
   1008     )
   1009     is_mapped = (
   1010         collected_annotation is not None
   1011         and collected_annotation.mapped_container is not None
   1012     )
   1013     generated_obj = (
   1014         collected_annotation.attr_value
   1015         if collected_annotation is not None
   1016         else obj
   1017     )

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:1277](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/decl_base.py:1277), in _ClassScanMapperConfig._collect_annotation(self, name, raw_annotation, originating_class, expect_mapped, attr_value)
   1274     is_dataclass_field = False
   1276 is_dataclass_field = False
-> 1277 extracted = _extract_mapped_subtype(
   1278     raw_annotation,
   1279     self.cls,
   1280     originating_class.__module__,
   1281     name,
   1282     type(attr_value),
   1283     required=False,
   1284     is_dataclass_field=is_dataclass_field,
   1285     expect_mapped=expect_mapped
   1286     and not is_dataclass,  # self.allow_dataclass_fields,
   1287 )
   1289 if extracted is None:
   1290     # ClassVar can come out here
   1291     return None

File [~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/util.py:2363](https://file+.vscode-resource.vscode-cdn.net/home/emre/Desktop/NXP/NAS/tpot/src/~/anaconda3/envs/tpot/lib/python3.10/site-packages/sqlalchemy/orm/util.py:2363), in _extract_mapped_subtype(raw_annotation, cls, originating_module, key, attr_cls, required, is_dataclass_field, expect_mapped, raiseerr)
   2358     elif isinstance(origin, type) and issubclass(
   2359         origin, ORMDescriptor
   2360     ):
   2361         return None
-> 2363     raise sa_exc.ArgumentError(
   2364         f'Type annotation for "{cls.__name__}.{key}" '
   2365         "can't be correctly interpreted for "
   2366         "Annotated Declarative Table form.  ORM annotations "
   2367         "should normally make use of the ``Mapped[]`` generic "
   2368         "type, or other ORM-compatible generic type, as a "
   2369         "container for the actual type, which indicates the "
   2370         "intent that the attribute is mapped. "
   2371         "Class variables that are not intended to be mapped "
   2372         "by the ORM should use ClassVar[].  "
   2373         "To allow Annotated Declarative to disregard legacy "
   2374         "annotations which don't use Mapped[] to pass, set "
   2375         '"__allow_unmapped__ = True" on the class or a '
   2376         "superclass this class.",
   2377         code="zlpr",
   2378     )
   2380 else:
   2381     return annotated, None

ArgumentError: Type annotation for "SQAGeneratorRun.arms" can't be correctly interpreted for Annotated Declarative Table form.  ORM annotations should normally make use of the ``Mapped[]`` generic type, or other ORM-compatible generic type, as a container for the actual type, which indicates the intent that the attribute is mapped. Class variables that are not intended to be mapped by the ORM should use ClassVar[].  To allow Annotated Declarative to disregard legacy annotations which don't use Mapped[] to pass, set "__allow_unmapped__ = True" on the class or a superclass this class. (Background on this error at: https://sqlalche.me/e/20/zlpr)

Do you have any idea on what might be the reason? Thanks.

mpolson64 commented 1 year ago

This looks like a bug on our end -- I will investigate this and report back once I have a repro. In the meantime, could you provide a list of all the packages you're using in this environment as well as their versions? This is typically best done via pip freeze or conda list. Thank you, and hopefully we can get this resolved shortly.

fernexda commented 1 year ago

They you very much, @mpolson64 , your solution works like a charm. I have two objectives, and I tinkered a bit to load the experiment using 'decoder' instead of 'encoder'. This is my code:

    bundle = RegistryBundle(
        metric_clss={MetricCthrust: None, MetricCp: None},
        runner_clss={}
    )
    save_experiment(
        experiment=experiment,
        filepath='test.json',
        encoder_registry=bundle.encoder_registry,
        class_encoder_registry=bundle.class_encoder_registry,
    )

    exp2 = load_experiment(
        'test.json',
        decoder_registry=bundle.decoder_registry,
        class_decoder_registry=bundle.class_decoder_registry,
    )

Thanks again for your quick response!

ekurtgl commented 1 year ago

It still doesn't work for me. My package list is here:

# packages in environment at /home/emre/anaconda3/envs/tpot:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             5.1                       1_gnu  
absl-py                   1.4.0                    pypi_0    pypi
addict                    2.4.0                    pypi_0    pypi
aiohttp                   3.8.4                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
anyio                     3.6.2                    pypi_0    pypi
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.2.3                    pypi_0    pypi
astor                     0.8.1                    pypi_0    pypi
asttokens                 2.2.1                    pypi_0    pypi
async-timeout             4.0.2                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
ax-platform               0.3.3                    pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.12.2                   pypi_0    pypi
blas                      1.0                         mkl  
bleach                    6.0.0                    pypi_0    pypi
botorch                   0.8.5                    pypi_0    pypi
brotlipy                  0.7.0           py310h7f8727e_1002  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2023.01.10           h06a4308_0  
cachetools                5.3.1                    pypi_0    pypi
certifi                   2023.5.7        py310h06a4308_0  
cffi                      1.15.1          py310h5eee18b_3  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
click                     8.1.3                    pypi_0    pypi
cloudpickle               2.2.1                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
comm                      0.1.3                    pypi_0    pypi
configargparse            1.5.3                    pypi_0    pypi
contextlib2               21.6.0                   pypi_0    pypi
contourpy                 1.0.7                    pypi_0    pypi
cryptography              39.0.1          py310h9ce1e76_0  
cuda-cudart               11.8.89                       0    nvidia
cuda-cupti                11.8.87                       0    nvidia
cuda-libraries            11.8.0                        0    nvidia
cuda-nvrtc                11.8.89                       0    nvidia
cuda-nvtx                 11.8.86                       0    nvidia
cuda-runtime              11.8.0                        0    nvidia
cycler                    0.11.0                   pypi_0    pypi
dash                      2.10.2                   pypi_0    pypi
dash-core-components      2.0.0                    pypi_0    pypi
dash-html-components      2.0.0                    pypi_0    pypi
dash-table                5.0.0                    pypi_0    pypi
debugpy                   1.6.7                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
deep-phonemizer           0.0.17                   pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
docker                    6.1.3                    pypi_0    pypi
docstring-parser          0.8.1                    pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
executing                 1.2.0                    pypi_0    pypi
fastjsonschema            2.17.1                   pypi_0    pypi
ffmpeg                    4.3                  hf484d3e_0    pytorch
filelock                  3.9.0           py310h06a4308_0  
flask                     2.2.5                    pypi_0    pypi
fonttools                 4.39.4                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               h4a9f257_0  
frozenlist                1.3.3                    pypi_0    pypi
fsspec                    2023.6.0                 pypi_0    pypi
giflib                    5.2.1                h5eee18b_3  
gmp                       6.2.1                h295c915_3  
gmpy2                     2.1.2           py310heeb90bb_0  
gnutls                    3.6.15               he1e5248_0  
google-auth               2.21.0                   pypi_0    pypi
google-auth-oauthlib      1.0.0                    pypi_0    pypi
gpytorch                  1.10                     pypi_0    pypi
greenlet                  2.0.2                    pypi_0    pypi
grpcio                    1.56.0                   pypi_0    pypi
h5py                      3.8.0                    pypi_0    pypi
idna                      3.4             py310h06a4308_0  
importlib-metadata        6.7.0                    pypi_0    pypi
intel-openmp              2023.1.0         hdb19cb5_46305  
ipykernel                 6.23.1                   pypi_0    pypi
ipython                   8.13.2                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                8.0.6                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
itsdangerous              2.1.2                    pypi_0    pypi
jedi                      0.18.2                   pypi_0    pypi
jinja2                    3.1.2           py310h06a4308_0  
joblib                    1.2.0                    pypi_0    pypi
jpeg                      9e                   h5eee18b_1  
json-tricks               3.17.0                   pypi_0    pypi
jsonpointer               2.3                      pypi_0    pypi
jsonschema                4.17.3                   pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            8.2.0                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.3.0                    pypi_0    pypi
jupyter-events            0.6.3                    pypi_0    pypi
jupyter-server            2.5.0                    pypi_0    pypi
jupyter-server-terminals  0.4.4                    pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-widgets        3.0.7                    pypi_0    pypi
kaleido                   0.2.1                    pypi_0    pypi
kiwisolver                1.4.4                    pypi_0    pypi
kwarray                   0.6.13                   pypi_0    pypi
lame                      3.100                h7b6447c_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.38                 h1181459_1  
lerc                      3.0                  h295c915_0  
libcublas                 11.11.3.6                     0    nvidia
libcufft                  10.9.0.58                     0    nvidia
libcufile                 1.6.1.9                       0    nvidia
libcurand                 10.3.2.106                    0    nvidia
libcusolver               11.4.1.48                     0    nvidia
libcusparse               11.7.5.86                     0    nvidia
libdeflate                1.17                 h5eee18b_0  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 11.2.0               h1234567_1  
libgomp                   11.2.0               h1234567_1  
libiconv                  1.16                 h7f8727e_2  
libidn2                   2.3.4                h5eee18b_0  
libnpp                    11.8.0.86                     0    nvidia
libnvjpeg                 11.9.0.86                     0    nvidia
libpng                    1.6.39               h5eee18b_0  
libstdcxx-ng              11.2.0               h1234567_1  
libtasn1                  4.19.0               h5eee18b_0  
libtiff                   4.5.0                h6a678d5_2  
libunistring              0.9.10               h27cfd23_0  
libuuid                   1.41.5               h5eee18b_0  
libwebp                   1.2.4                h11a3e52_1  
libwebp-base              1.2.4                h5eee18b_1  
lightning-utilities       0.8.0                    pypi_0    pypi
linear-operator           0.4.0                    pypi_0    pypi
lz4-c                     1.9.4                h6a678d5_0  
markdown                  3.4.3                    pypi_0    pypi
markupsafe                2.1.2                    pypi_0    pypi
matplotlib                3.7.1                    pypi_0    pypi
matplotlib-inline         0.1.6                    pypi_0    pypi
mistune                   2.0.5                    pypi_0    pypi
mkl                       2023.1.0         h6d00ec8_46342  
mkl-service               2.4.0           py310h5eee18b_1  
mkl_fft                   1.3.6           py310h1128e8f_1  
mkl_random                1.2.2           py310h1128e8f_1  
mpc                       1.1.0                h10f8cd9_1  
mpfr                      4.0.2                hb69a4c5_1  
mpmath                    1.2.1                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
multipledispatch          1.0.0                    pypi_0    pypi
mypy-extensions           1.0.0                    pypi_0    pypi
nbclassic                 1.0.0                    pypi_0    pypi
nbclient                  0.8.0                    pypi_0    pypi
nbconvert                 7.4.0                    pypi_0    pypi
nbformat                  5.7.0                    pypi_0    pypi
ncurses                   6.4                  h6a678d5_0  
nest-asyncio              1.5.6                    pypi_0    pypi
nettle                    3.7.3                hbbd107a_1  
networkx                  2.8.4           py310h06a4308_1  
nni                       2.10.1                   pypi_0    pypi
notebook                  6.5.4                    pypi_0    pypi
notebook-shim             0.2.3                    pypi_0    pypi
numpy                     1.24.3          py310h5f9d8c6_1  
numpy-base                1.24.3          py310hb5e798b_1  
oauthlib                  3.2.2                    pypi_0    pypi
open3d                    0.17.0                   pypi_0    pypi
opencv-python             4.7.0.72                 pypi_0    pypi
openh264                  2.1.1                h4ff587b_0  
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   1.1.1t               h7f8727e_0  
opt-einsum                3.3.0                    pypi_0    pypi
packaging                 23.1                     pypi_0    pypi
pandas                    2.0.1                    pypi_0    pypi
pandocfilters             1.5.0                    pypi_0    pypi
parso                     0.8.3                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    9.4.0           py310h6a678d5_0  
pip                       23.0.1          py310h06a4308_0  
platformdirs              3.5.1                    pypi_0    pypi
plotly                    5.14.1                   pypi_0    pypi
prettytable               3.8.0                    pypi_0    pypi
prometheus-client         0.16.0                   pypi_0    pypi
prompt-toolkit            3.0.38                   pypi_0    pypi
protobuf                  4.23.3                   pypi_0    pypi
psutil                    5.9.5                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
pyasn1                    0.5.0                    pypi_0    pypi
pyasn1-modules            0.3.0                    pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0  
pygments                  2.15.1                   pypi_0    pypi
pyopenssl                 23.0.0          py310h06a4308_0  
pyparsing                 3.0.9                    pypi_0    pypi
pyquaternion              0.9.9                    pypi_0    pypi
pyre-extensions           0.0.30                   pypi_0    pypi
pyro-api                  0.1.2                    pypi_0    pypi
pyro-ppl                  1.8.5                    pypi_0    pypi
pyrsistent                0.19.3                   pypi_0    pypi
pysocks                   1.7.1           py310h06a4308_0  
python                    3.10.11              h7a1cb2a_2  
python-dateutil           2.8.2                    pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
pythonwebhdfs             0.2.3                    pypi_0    pypi
pytorch                   2.0.1           py3.10_cuda11.8_cudnn8.7.0_0    pytorch
pytorch-cuda              11.8                 h7e8668a_5    pytorch
pytorch-lightning         2.0.3                    pypi_0    pypi
pytorch-mutex             1.0                        cuda    pytorch
pytz                      2023.3                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
pyzmq                     25.0.2                   pypi_0    pypi
qtconsole                 5.4.3                    pypi_0    pypi
qtpy                      2.3.1                    pypi_0    pypi
readline                  8.2                  h5eee18b_0  
requests                  2.29.0          py310h06a4308_0  
requests-oauthlib         1.3.1                    pypi_0    pypi
responses                 0.23.1                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rsa                       4.9                      pypi_0    pypi
schema                    0.7.5                    pypi_0    pypi
scikit-learn              1.2.2                    pypi_0    pypi
scipy                     1.10.1                   pypi_0    pypi
seaborn                   0.12.2                   pypi_0    pypi
send2trash                1.8.2                    pypi_0    pypi
setuptools                66.0.0          py310h06a4308_0  
simplejson                3.19.1                   pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
sniffio                   1.3.0                    pypi_0    pypi
soupsieve                 2.4.1                    pypi_0    pypi
sqlalchemy                2.0.18                   pypi_0    pypi
sqlite                    3.41.2               h5eee18b_0  
stack-data                0.6.2                    pypi_0    pypi
sympy                     1.11.1          py310h06a4308_0  
tabulate                  0.9.0                    pypi_0    pypi
tbb                       2021.8.0             hdb19cb5_0  
tenacity                  8.2.2                    pypi_0    pypi
tensorboard               2.13.0                   pypi_0    pypi
tensorboard-data-server   0.7.1                    pypi_0    pypi
terminado                 0.17.1                   pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tinycss2                  1.2.1                    pypi_0    pypi
tk                        8.6.12               h1ccaba5_0  
torchaudio                2.0.2               py310_cu118    pytorch
torchmetrics              0.11.4                   pypi_0    pypi
torchtriton               2.0.0                     py310    pytorch
torchvision               0.15.2              py310_cu118    pytorch
torchx                    0.5.0                    pypi_0    pypi
tornado                   6.3.2                    pypi_0    pypi
tqdm                      4.65.0                   pypi_0    pypi
traitlets                 5.9.0                    pypi_0    pypi
typeguard                 2.13.3                   pypi_0    pypi
types-pyyaml              6.0.12.10                pypi_0    pypi
typing-inspect            0.9.0                    pypi_0    pypi
typing_extensions         4.5.0           py310h06a4308_0  
tzdata                    2023.3                   pypi_0    pypi
ubelt                     1.3.2                    pypi_0    pypi
uri-template              1.2.0                    pypi_0    pypi
urllib3                   1.26.15         py310h06a4308_0  
wcwidth                   0.2.6                    pypi_0    pypi
webcolors                 1.13                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.5.2                    pypi_0    pypi
websockets                11.0.3                   pypi_0    pypi
werkzeug                  2.2.3                    pypi_0    pypi
wheel                     0.38.4          py310h06a4308_0  
widgetsnbextension        4.0.7                    pypi_0    pypi
xz                        5.4.2                h5eee18b_0  
yarl                      1.9.2                    pypi_0    pypi
zipp                      3.15.0                   pypi_0    pypi
zlib                      1.2.13               h5eee18b_0  
zstd                      1.5.5                hc292b87_0
16acdg commented 1 year ago

Hi @ekurtgl, this is just a short-term work-around, but I was able to resolve this problem by downgrading to SQLAlchemy v1.

E.g. pip3 install SQLAlchemy==1.4.49

But in my opinion this is definitely a bug because if someone only wants to save experiments to JSON, they shouldn't have to install sql alchemy.

mpolson64 commented 1 year ago

Thank you @16acdg -- yes, Ax is currently incompatible with SQLAlchemy 2+. We are working on getting a fix out for this soon. We will also intend to look into solving this behavior where JSON storage is broken due to an incompatible SQLAlchemy version; currently this code is too tightly coupled and users should be able to save custom JSON objects without importing SQLAlchemy code at all.

mpolson64 commented 4 months ago

Upgrading SQLAlchemy to 2.0 will likely take some substantial investment on our part, moving this task to our master tracking Issue so it can make it on to our team's roadmap #566.