Error occurred when exporting database to other formats

[xieduo7]

Bug description [A clear and concise description of what the bug is.] An error occurred when I exported the database to CSV

abc-export --db test.db --out exported.csv --format csv
/home/xieduo/.local/lib/python3.8/site-packages/pandas/io/sql.py:1402: SAWarning: TypeDecorator BytesStorage() will not produce a cache key because the ``cache_ok`` attribute is not set to True.  This can have significant performance implications including some performance degradations in comparison to prior SQLAlchemy versions.  Set this attribute to True if this type object's state is safe to use in a cache key, or False to disable this warning. (Background on this error at: https://sqlalche.me/e/14/cprf)
  return self.connectable.execution_options().execute(*args, **kwargs)

The output file:

$ cat exported.csv
,index,epsilon,samples,m,model_name,p_model,w,particle_id,distance,par_name,par_val,sumstat_name,sumstat_val

Expected behavior [A clear and concise description of what you expected to happen.] Output the CSV file successfully To reproduce [A minimal example to reproduce this error, ideally a python script.] First run:

import os
import tempfile

import matplotlib.pyplot as plt
import numpy as np

import pyabc

pyabc.settings.set_figure_params('pyabc')  # for beautified plots

def model(parameter):
    return {"data": parameter["mu"] + 0.5 * np.random.randn()}

prior = pyabc.Distribution(mu=pyabc.RV("uniform", 0, 5))

def distance(x, x0):
    return abs(x["data"] - x0["data"])

abc = pyabc.ABCSMC(model, prior, distance, population_size=1000)

observation = 2.5
db_path = pyabc.storage.create_sqlite_db_id(file_="/data/xieduo/ABC_model/ABC_SMC/pyabc/test.db")
abc.new(db_path, {"data": observation})

history = abc.run(minimum_epsilon=0.1, max_nr_populations=10)

fig, ax = plt.subplots()
for t in range(history.max_t + 1):
    df, w = history.get_distribution(m=0, t=t)
    pyabc.visualization.plot_kde_1d(
        df,
        w,
        xmin=0,
        xmax=5,
        x="mu",
        xname=r"$\mu$",
        ax=ax,
        label=f"PDF t={t}",
    )
ax.axvline(observation, color="k", linestyle="dashed")
ax.legend();

And then:

abc-export --db test.db --out exported.csv --format csv

Environment [Report at least the pyABC version (pip show pyabc),

pip show pyabc
Name: pyabc
Version: 0.12.3
Summary: Distributed, likelihood-free ABC-SMC inference
Home-page: https://github.com/icb-dcm/pyabc
Author: The pyABC developers
Author-email: yannik.schaelte@gmail.com
License: BSD-3-Clause
Location: /home/xieduo/software/miniconda3/envs/jupyter-env/lib/python3.8/site-packages
Requires: scikit-learn, jabbar, redis, pandas, matplotlib, cloudpickle, click, sqlalchemy, scipy, distributed, numpy, gitpython
Required-by:

python version (which python; python --version),

Python 3.8.13

and system (Ubuntu/Mac/...).]

Linux pbs-master 3.10.0-1160.41.1.el7.x86_64 #1 SMP Tue Aug 31 14:52:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

And sqlalchemy version:

$ pip show sqlalchemy
Name: SQLAlchemy
Version: 1.4.32
Summary: Database Abstraction Library
Home-page: https://www.sqlalchemy.org
Author: Mike Bayer
Author-email: mike_mp@zzzcomputing.com
License: MIT
Location: /home/xieduo/.local/lib/python3.8/site-packages
Requires: greenlet
Required-by: pyabc

Thank you.

Best, Duo

[yannikschaelte]

Hi, is the csv file generated correctly still?

[yannikschaelte]

Closing due to inactivity. Feel free to re-open if the issue persists!

[xieduo7]

Hi @yannikschaelte , I used the latest version of pyabc and this issue also appeared

[yannikschaelte]

Hi @xieduo7 thanks for reporting. Your output file should however be generated correctly, as this is only a warning about efficiency of (non) caching.

[yannikschaelte]

Added cache in https://github.com/ICB-DCM/pyABC/pull/590

[xieduo7]

Hi @yannikschaelte , I tried it again and it works. Thank you!