commonsense / conceptnet5

Code for building ConceptNet from raw data.
Other
2.78k stars 355 forks source link

possible to continue the installation after error ? #310

Closed zababurinsv closed 2 years ago

zababurinsv commented 3 years ago

After 6 hours of installation, I got an error.

[Mon Sep 13 22:53:42 2021]
Finished job 2.
463 of 472 steps (98%) done

[Mon Sep 13 22:53:42 2021]
rule shuffle_gin:
    input: data/psql/edges_gin.csv
    output: data/psql/edges_gin.shuf.csv
    jobid: 3

/usr/bin/bash: line 1: 72598 Killed                  cn5-vectors intersect data/vectors/crawl-300d-2M-retrofit.h5 data/vectors/w2v-google-news-retrofit.h5 data/vectors/glove12-840B-retrofit.h5 data/vectors/fasttext-opensubtitles-retrofit.h5 data/vectors/numberbatch-retrofitted.h5 data/vectors/intersection-projection.h5
[Mon Sep 13 22:54:18 2021]
Error in rule merge_intersect:
    jobid: 177
    output: data/vectors/numberbatch-retrofitted.h5, data/vectors/intersection-projection.h5
    shell:
        cn5-vectors intersect data/vectors/crawl-300d-2M-retrofit.h5 data/vectors/w2v-google-news-retrofit.h5 data/vectors/glove12-840B-retrofit.h5 data/vectors/fasttext-opensubtitles-retrofit.h5 data/vectors/numberbatch-retrofitted.h5 data/vectors/intersection-projection.h5
        (exited with non-zero exit code)

Removing temporary output file data/psql/edges_gin.csv.
[Mon Sep 13 22:56:03 2021]
Finished job 3.
464 of 472 steps (98%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/zb/Desktop/conceptnet5/.snakemake/log/2021-09-13T175652.539242.snakemake.log
(base) zb@ConceptNet:~/Desktop/conceptnet5$ pip install pytest PyLD
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pytest in /home/zb/anaconda3/lib/python3.8/site-packages (6.2.3)
Requirement already satisfied: PyLD in /home/zb/.local/lib/python3.8/site-packages (2.0.3)
Requirement already satisfied: frozendict in /home/zb/.local/lib/python3.8/site-packages (from PyLD) (2.0.6)
Requirement already satisfied: lxml in /home/zb/anaconda3/lib/python3.8/site-packages (from PyLD) (4.6.3)
Requirement already satisfied: cachetools in /home/zb/.local/lib/python3.8/site-packages (from PyLD) (4.2.2)
Requirement already satisfied: attrs>=19.2.0 in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (20.3.0)
Requirement already satisfied: iniconfig in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (1.1.1)
Requirement already satisfied: packaging in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (20.9)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (0.13.1)
Requirement already satisfied: py>=1.8.2 in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (1.10.0)
Requirement already satisfied: toml in /home/zb/anaconda3/lib/python3.8/site-packages (from pytest) (0.10.2)
Requirement already satisfied: pyparsing>=2.0.2 in /home/zb/anaconda3/lib/python3.8/site-packages (from packaging->pytest) (2.4.7)
(base) zb@ConceptNet:~/Desktop/conceptnet5$ pytest
============================= test session starts ==============================
platform linux -- Python 3.8.8, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/zb/Desktop/conceptnet5, configfile: setup.cfg
plugins: anyio-2.2.0
collected 104 items                                                            

conceptnet5/edges.py .                                                   [  0%]
conceptnet5/nodes.py ....                                                [  4%]
conceptnet5/uri.py ................                                      [ 20%]
conceptnet5/builders/combine_assertions.py .                             [ 21%]
conceptnet5/formats/semantic_web.py ....                                 [ 25%]
conceptnet5/language/token_utils.py .                                    [ 25%]
conceptnet5/readers/dbpedia.py .                                         [ 26%]
conceptnet5/tests/test_convert.py ..                                     [ 28%]
conceptnet5/tests/test_languages.py F                                    [ 29%]
conceptnet5/tests/test_propagate.py ....                                 [ 33%]
conceptnet5/tests/test_sounds_like.py .                                  [ 34%]
conceptnet5/tests/test_vectors.py ................                       [ 50%]
conceptnet5/tests/full_build/test_all_relations_recorded.py s            [ 50%]
conceptnet5/tests/full_build/test_enough_data_exists.py ssssssssssssss   [ 64%]
conceptnet5/tests/small_build/test_api.py EE                             [ 66%]
conceptnet5/tests/small_build/test_built.py FFFFFFE                      [ 73%]
conceptnet5/tests/small_build/test_json_ld.py EEF                        [ 75%]
conceptnet5/tests/small_build/test_lemmatizer.py EEEEE                   [ 80%]
conceptnet5/tests/small_build/test_queries.py EEEEEEEEEE                 [ 90%]
conceptnet5/util/sounds_like.py ..........                               [100%]

==================================== ERRORS ====================================
_____________________ ERROR at setup of test_related_query _____________________

Please tell me if it is possible to continue the installation after a failure?