bson.errors.InvalidDocument: cannot encode object: True, of type: <class 'numpy.bool_'>

ShiQiaoL commented 5 months ago

Python version

Python 3.11.5

Pymatgen version

2024.3.1

Operating system version

CentOS Linux release 7.6.1810 (Core)

Current behavior

/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymatgen/io/vasp/sets.py:676: BadInputSetWarning: LASPH = True should be set for +U, meta-GGAs, hybrids, and vdW-DFT
  warnings.warn("LASPH = True should be set for +U, meta-GGAs, hybrids, and vdW-DFT", BadInputSetWarning)
Traceback (most recent call last):
  File "/public/home/xqliu/HTC-test/test-workshop-2022.7.29/submit.py", line 33, in <module>
    lp.add_wf(hse_wf)
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/fireworks/core/launchpad.py", line 409, in add_wf
    self.workflows.insert_one(wf.to_db_dict())
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/collection.py", line 669, in insert_one
    self._insert_one(
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/collection.py", line 609, in _insert_one
    self.__database.client._retryable_write(acknowledged, _insert_command, session)
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/mongo_client.py", line 1523, in _retryable_write
    return self._retry_with_session(retryable, func, s, bulk)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/mongo_client.py", line 1421, in _retry_with_session
    return self._retry_internal(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/_csot.py", line 107, in csot_wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/mongo_client.py", line 1462, in _retry_internal
    ).run()
      ^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/mongo_client.py", line 2315, in run
    return self._read() if self._is_read else self._write()
                                              ^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/mongo_client.py", line 2423, in _write
    return self._func(self._session, conn, self._retryable)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/collection.py", line 597, in _insert_command
    result = conn.command(
             ^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/helpers.py", line 322, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/pool.py", line 996, in command
    self._raise_connection_failure(error)
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/pool.py", line 968, in command
    return command(
           ^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/network.py", line 151, in command
    request_id, msg, size, max_doc_size = message._op_msg(
                                          ^^^^^^^^^^^^^^^^
  File "/public/home/xqliu/software/miniconda3/lib/python3.11/site-packages/pymongo/message.py", line 762, in _op_msg
    return _op_msg_uncompressed(flags, command, identifier, docs, opts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
bson.errors.InvalidDocument: cannot encode object: True, of type: <class 'numpy.bool_'>

Expected Behavior

I hope it can run successfully. I have located the error caused by lp.add_wf(opt_wf). Additionally, I have successfully verified that this is not an issue with the mongoDB connection. Successfully tested the connection to mongoDB

Minimal example

import os
# the path of the structure file
#import os
from pymatgen.core import Structure
from fireworks import LaunchPad
from atomate.vasp.drones import VaspDrone
from atomate.vasp.workflows.presets.core import wf_structure_optimization, wf_bandstructure, wf_bandstructure_hse, wf_static
from atomate.vasp.powerups import add_additional_fields_to_taskdocs, add_namefile, add_priority, add_modify_incar, use_custodian, add_stability_check
from atomate.common.powerups import add_tags

db_file = "/public/home/xqliu/HTC-test/test-workshop-2022.7.29/dbxqliu0407.json"
lp = LaunchPad.from_file("/public/home/xqliu/HTC-test/test-workshop-2022.7.29/xqliu_launchpad.yaml")

for file in os.listdir(path):
    struct = Structure.from_file(os.path.join(path,file))
    opt_wf = wf_structure_optimization(structure=struct,c={"DB_FILE":db_file})
    lp.add_wf(opt_wf)

Relevant files to reproduce this bug

janosh commented 5 months ago

migrated to fireworks since this is not a pymatgen issue.

i suspect this is related to the recent numpy v2 release. what's your version?

it would be good for fireworks to replace all numpy.bool_ with regular Python bool before trying to insert workflows into the DB

ShiQiaoL commented 5 months ago

it would be good for fireworks to replace all numpy.bool_ with regular Python bool before trying to insert workflows into the DB

Hi, my numpy version is 1.26.4. sorry that I new studier, I don't know to do it replace all numpy.bool_ with regular Python bool by code.

janosh commented 5 months ago

i would first try to iterate through the wf.metadata and print the types of all values:

for key, val in wf.metadata.items():
    print(f"{key}: {type(val)}")

to see if the offending value is in there. once you know the key, you just cast to bool

wf.metadata[key] = bool(wf.metadata[key])

ShiQiaoL commented 5 months ago

i would first try to iterate through the wf.metadata and print the types of all values:
for key, val in wf.metadata.items():
    print(f"{key}: {type(val)}")
to see if the offending value is in there. once you know the key, you just cast to bool
wf.metadata[key] = bool(wf.metadata[key])

Thank you very much, it has worked successfully

materialsproject / fireworks