search

ICB-DCM / pyABC

distributed, likelihood-free inference
https://pyabc.rtfd.io
BSD 3-Clause "New" or "Revised" License
206 stars 44 forks source link

abc-server-dash FileNotFoundError: [Errno 2] No such file or directory: '.../pyabc/visserver/assets/square_v2.png' #615

Closed omsai closed 1 year ago

omsai commented 1 year ago

Bug description Running abc-server-dash fails with this error:

Traceback (most recent call last):
  File "/Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/bin/abc-server-dash", line 7, in <module>
    from pyabc.visserver.server_dash import run_app
  File "/Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/lib/python3.11/site-packages/pyabc/visserver/server_dash.py", line 31, in <module>
    square_base64 = base64.b64encode(open(square_png, 'rb').read()).decode('ascii')
                                     ^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/lib/python3.11/site-packages/pyabc/visserver/assets/square_v2.png'
$ tree "/Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/lib/python3.11/site-packages/pyabc/visserver/"
/Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/lib/python3.11/site-packages/pyabc/visserver/
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-311.pyc
│   ├── server_dash.cpython-311.pyc
│   └── server_flask.cpython-311.pyc
├── server_dash.py
├── server_flask.py
├── static
│   ├── abc_run_list_panel_image.svg
│   ├── abcstyle.css
│   └── server_info_panel_image.svg
└── templates
    ├── 404.html
    ├── abc_detail.html
    ├── abc_overview.html
    ├── base.html
    ├── index.html
    ├── model.html
    └── server_info.html

4 directories, 16 files

Expected behavior abc-server-dash should launch without errors.

To reproduce

$ cat pyproject.toml 
[build-system]
build-backend = "hatchling.build"
requires = [
    "hatchling",
]

[project]
name = "calibrate"
version = "0.1"
description = "Calibrate model to available datasets"
dependencies = [
    "dash",          # for abc-server-dash
    "dash-bootstrap-components", # for abc-server-dash
    "lxml",
    "pyabc[pyarrow]",
]

$ hatch shell

$ abc-server-dash

Environment

$ hatch dep show table
                Project                 
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Name                      ┃ Features ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ dash                      │          │
│ dash-bootstrap-components │          │
│ lxml                      │          │
│ pyabc                     │ pyarrow  │
└───────────────────────────┴──────────┘

$ # Enter virtual environment for subsequent commands
$ hatch shell

$ python -V
Python 3.11.5

$ pip show pyabc
Name: pyabc
Version: 0.12.12
Summary: Distributed, likelihood-free ABC-SMC inference
Home-page: https://github.com/icb-dcm/pyabc
Author: The pyABC developers
Author-email: yannik.schaelte@gmail.com
License: BSD-3-Clause
Location: /Users/pnanda/Library/Application Support/hatch/env/virtual/gransimcal/YDO745f3/gransimcal/lib/python3.11/site-packages
Requires: click, cloudpickle, distributed, gitpython, jabbar, matplotlib, numpy, pandas, redis, scikit-learn, scipy, sqlalchemy
Required-by: gransimcal

$ pip list
Package                   Version      Editable project location
------------------------- ------------ ----------------------------------------------------------------------
ansi2html                 1.8.0
certifi                   2023.7.22
charset-normalizer        3.3.0
click                     8.1.7
cloudpickle               2.2.1
contourpy                 1.1.1
cycler                    0.11.0
dash                      2.14.0
dash-bootstrap-components 1.5.0
dash-core-components      2.0.0
dash-html-components      2.0.0
dash-table                5.0.0
dask                      2023.9.2
distributed               2023.9.2
Flask                     2.2.5
fonttools                 4.42.1
fsspec                    2023.9.2
gitdb                     4.0.10
GitPython                 3.1.37
gransimcal                0.0.1        /Users/pnanda/immunology/GR-ABM-ODE/simulation/scripts/calibration/abc
idna                      3.4
importlib-metadata        6.8.0
itsdangerous              2.1.2
jabbar                    0.0.16
Jinja2                    3.1.2
joblib                    1.3.2
kiwisolver                1.4.5
locket                    1.0.0
lxml                      4.9.3
MarkupSafe                2.1.3
matplotlib                3.8.0
msgpack                   1.0.6
nest-asyncio              1.5.8
numpy                     1.26.0
packaging                 23.1
pandas                    2.1.1
partd                     1.4.1
Pillow                    10.0.1
pip                       23.2.1
plotly                    5.17.0
psutil                    5.9.5
pyabc                     0.12.12
pyarrow                   13.0.0
pyparsing                 3.1.1
python-dateutil           2.8.2
pytz                      2023.3.post1
PyYAML                    6.0.1
redis                     5.0.1
requests                  2.31.0
retrying                  1.3.4
scikit-learn              1.3.1
scipy                     1.11.2
setuptools                68.2.0
setuptools-scm            8.0.3
setuptools-svn            1.3.0
six                       1.16.0
smmap                     5.0.1
sortedcontainers          2.4.0
SQLAlchemy                2.0.21
tblib                     2.0.0
tenacity                  8.2.3
threadpoolctl             3.2.0
toolz                     0.12.0
tornado                   6.3.3
typing_extensions         4.8.0
tzdata                    2023.3
urllib3                   2.0.5
Werkzeug                  2.2.3
wheel                     0.41.2
zict                      3.0.0
zipp                      3.17.0

$ sw_vers 
ProductName:        macOS
ProductVersion:     13.6
BuildVersion:       22G120

$ system_profiler SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: MacBook Air
      Model Identifier: Mac14,2
      Model Number: Z15W001BLLL/A
      Chip: Apple M2
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 16 GB
      System Firmware Version: 8422.141.2
      OS Loader Version: 8422.141.2
      Serial Number (system): V0MVP21HX4
      Hardware UUID: 77AB0079-DE3C-54F0-8708-0262EF5ABEB5
      Provisioning UDID: 00008112-001151A93EF0C01E
      Activation Lock Status: Disabled

$ uname -a
Darwin ltfra117.wifi.med.umich.edu 22.6.0 Darwin Kernel Version 22.6.0: Fri Sep 15 13:39:54 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_ARM64_T8112 arm64
omsai commented 1 year ago

The documentation suggests using the command here: https://pyabc.readthedocs.io/en/latest/visualization.html#visualization-server

omsai commented 1 year ago

Fault persists even after running pip install "pyabc[webserver_dash]"

omsai commented 1 year ago

I tried installing from GitHub but get the same error. It looks like the entire assets directory is not being packaged.

stephanmg commented 1 year ago

Thanks for reporting @omsai - will have a look.

stephanmg commented 1 year ago

By the way, did you try the develop branch?

omsai commented 1 year ago

By the way, did you try the develop branch?

I had only tried the main branch, not develop. Here are the changes to the pyproject.toml:

diff -u /tmp/abctest/pyproject.toml.orig /tmp/abctest/pyproject.toml
--- /tmp/abctest/pyproject.toml.orig    2023-11-06 08:08:02.396314808 -0500
+++ /tmp/abctest/pyproject.toml 2023-11-06 08:14:05.269804589 -0500
@@ -12,5 +12,8 @@
     "dash",             # for abc-server-dash
     "dash-bootstrap-components", # for abc-server-dash
     "lxml",
-    "pyabc[pyarrow]",
+    "pyabc[pyarrow] @ git+https://github.com/ICB-DCM/pyABC.git@develop",
 ]
+
+[tool.hatch.metadata]
+allow-direct-references = true

No change using develop:

$ hatch env prune

$ hatch shell
Traceback (most recent call last):
  File "/home/omsai/.local/share/hatch/env/virtual/calibrate/TIT3we0a/calibrate/bin/abc-server-dash", line 5, in <module>
    from pyabc.visserver.server_dash import run_app
  File "/home/omsai/.local/share/hatch/env/virtual/calibrate/TIT3we0a/calibrate/lib/python3.9/site-packages/pyabc/visserver/server_dash.py", line 31, in <module>
    square_base64 = base64.b64encode(open(square_png, 'rb').read()).decode('ascii')
FileNotFoundError: [Errno 2] No such file or directory: '/home/omsai/.local/share/hatch/env/virtual/calibrate/TIT3we0a/calibrate/lib/python3.9/site-packages/pyabc/visserver/assets/square_v2.png'

No difference if I use the zip link in the dependency either: pyabc[pyarrow] @ https://github.com/ICB-DCM/pyABC/archive/refs/heads/develop.zip

stephanmg commented 1 year ago

Can't reproduce the error in pyabc 0.12.10 and 0.12.9. Just curious, do these versions work for you?

stephanmg commented 1 year ago

Should be fixed indevelop now. Please try. Will create a new release with some more fixes soon.

omsai commented 1 year ago

I still get the same error.

stephanmg commented 1 year ago

Okay, we will work on it and come back to you. Meanwhile 0.12.10 or 0.12.9 should work, if not please let me know.

omsai commented 1 year ago

Neither of those versions 0.12.9 and 0.12.10 even provide the abc-server-dash executable:

$ hatch shell
$ abc-
abc-export         abc-redis-manager  abc-server         
abc-migrate        abc-redis-worker

With the current version:

$ hatch shell
$ abc-
abc-export         abc-redis-manager  abc-server         abc-server-flask
abc-migrate        abc-redis-worker   abc-server-dash
stephanmg commented 1 year ago

@omsai ah yes, you are right, there will be only abc-server. We will see what causes the issue and prepare a new pyabc release with bugfixes.

stephanmg commented 1 year ago

@omsai problem should now (as of today) be fixed. please try to install from develop branch. thanks for your patience.

omsai commented 1 year ago

@stephanmg It works! I confirmed also with loading one of my databases. Thank you for all your work on this!

stephanmg commented 1 year ago

Okay cool, no problem, we will cut out new patch release later today.