_dill TypeError when loading graph_CS.pk

[tamaramueller]

Hi,

I downloaded the data you referenced and tried to run your code on it, but unfortunately ran into some issues.

(1) I am working on a machine with cuda 10.0, which means that I can't install the specified versions of the python packages. So I switched to the newest versions. (Just mentioning this here, in case this might be the reason for my problems.)

(2) Building the docker conatiner failed due to memory issues. (on this note: there's a typo in the docker command in the README.md file in the Docker folder. It should be Dockerfile instead of DockerFile:

$ docker build -t "pyhgt:latest" -f ./Docker/Dockerfile .

(3) When I try to run train_paper_field.py I get an error message when the graph_CS.pk file is loaded as an RenameUnpickler(). I couldn't figure out where the error comes from. This is the specific error message I get:

Traceback (most recent call last):
  File "train_paper_field.py", line 72, in <module>
    graph = renamed_load(open(os.path.join(args.data_dir, 'graph%s.pk' % args.domain), 'rb'))
  File "/home/tamara/Documents/coding/pyHGT/OAG/pyHGT/data.py", line 269, in renamed_load
    return RenameUnpickler(file_obj).load()
  File "/home/tamara/.virtualenvs/playground/lib/python3.8/site-packages/dill/_dill.py", line 473, in load
    obj = StockUnpickler.load(self)
TypeError: an integer is required (got type bytes)

Do you have an idea what the issue might be in point (3)?

Many thanks!

Tamara

[empty-id]

Same error here! Need help! @acbull

[acbull]

Hi:

I checked the dill loading code, and it works well from my side. I search this error message and found people arguing that Python 3.8 (https://github.com/uqfoundation/dill/issues/318) seems to lead to this problem. Can I know which python version are you using?

If it still didn't work probably I can use another serialization package, say Pytorch save/load for this.

[empty-id]

Yeah, I am using Python 3.8.5 and dill 0.3.2 (latest). Maybe I can try another python version, but if you could provide other serialization data, it's better~ Thank you!

[ShadyAlkhouri]

Using Python 3.7.9 fixed the problem for me, with dill 0.3.0 as in the requirements file

[wukaiyeah]

Yeah, I have fixed this problem and the solution may be suit for everyone! I found the versions of Python, dill, and other packages did effect the successfully loading (Graph_CS.pk), and I could not install the same version as 'requirements.txt'. It's a huge contradiction. So I performed the preprocess step ('preprocess_OAG.py') and rebuilt the Graph_CS.pk with the latest versions of dill. Finally, it worked.

[Saladin58]

Hi:

I checked the dill loading code, and it works well from my side. I search this error message and found people arguing that Python 3.8 (uqfoundation/dill#318) seems to lead to this problem. Can I know which python version are you using?

If it still didn't work probably I can use another serialization package, say Pytorch save/load for this.

I am using python 3.11.0 and had same issue. If you could provide other serialization data then that would be very helpful. Thanks!