error during compilation

[wuqui]

Hi, thanks for sharing this tool!

I just wanted to try it, but I get the following error when I try to compile it on macOS:

nim c --threads:on --app:lib -d:danger --opt:speed -d:nimStrictDelete --gc:orc --out:mining.so mining

command line(1, 2) Warning: `gc:option` is deprecated; use `mm:option` instead [Deprecated]
Hint: used config file '/opt/homebrew/Cellar/nim/2.0.2/nim/config/nim.cfg' [Conf]
Hint: used config file '/opt/homebrew/Cellar/nim/2.0.2/nim/config/config.nims' [Conf]
..............................................................................................................................
/Users/user/tools/constraction/src/measures.nim(84, 40) Error: type mismatch: got 'array[0..3, int]' for 'getBigramContingency(word_count1, word_count2, ngram_count,
                     summed_all_ngram_counts)' but expected 'tuple'

Here’s my Nim version:

Nim Compiler Version 2.0.2 [MacOSX: arm64]
Compiled at 2023-12-15
Copyright (c) 2006-2023 by Andreas Rumpf

active boot switches: -d:release -d:nimUseLinenoise

Is there a fix?

[victoryhb]

Hi, thanks for the message. Constraction was developed with Nim 1.6, whose syntax may be incompatible with Nim 2.0 in subtle ways. We suggest that you use Nim 1.6 for the compilation.

[wuqui]

Thanks for your help. Unfortunately, I still get an error using Nim 1.6.0:

nim c --threads:on --app:lib -d:danger --opt:speed -d:nimStrictDelete --gc:orc --out:mining.so mining
Hint: used config file '/Users/USER/.choosenim/toolchains/nim-1.6.0/config/nim.cfg' [Conf]
Hint: used config file '/Users/USER/.choosenim/toolchains/nim-1.6.0/config/config.nims' [Conf]
..........................................................................................................
/Users/USER/.choosenim/toolchains/nim-1.6.0/lib/impure/db_sqlite.nim(184, 1) Warning: Circular dependency detected. `codeReordering` pragma may not be able to reorder some nodes properly [User]
...
/Users/USER/tools/constraction/src/data_manager.nim(5, 8) Error: cannot open file: jsony

Nim Compiler Version 1.6.0 [MacOSX: amd64]
Compiled at 2024-01-09
Copyright (c) 2006-2021 by Andreas Rumpf

active boot switches: -d:release

[victoryhb]

Hi, jsony is a Nim library for processing json. It can be installed with: nimble install jsony

[wuqui]

Thanks. I installed the necessary packages now and got quite a bit further, but I have two more errors to report, which seem to be unrelated.

First, the annotation process doesn’t complete because of this error:

python3.11 annotator.py ~/tools/constraction/data/coca-sample/in ~/tools/constraction/data/coca-sample/patterns.json

using device cpu
{'pipeline': 'entity,supersense', 'model': 'small', 'spacy_model': 'en_core_web_sm'}
--- startup: 1.262 seconds ---
skipped text_fic
skipped text_spok
skipped text_mag
skipped text_tvm
skipped text_blog
skipped text_acad
skipped text_web
skipped text_news
processing text_web.tokens
already processed text_web , skipping
processing text_mag.tokens
already processed text_mag , skipping
processing text_tvm.tokens
already processed text_tvm , skipping
processing text_news.tokens
/Users/quirin/tools/constraction/src/annotator.py:40: ParserWarning: Falling back to the 'python' engine because ord(quotechar) > 127, meaning the quotechar is larger than one byte, and the 'c' engine does not support such quotechars; you can avoid this warning by specifying engine='python'.
  df = pd.read_csv(token_path, sep="\t", quotechar='非')
Traceback (most recent call last):
  File "/Users/quirin/tools/constraction/src/annotator.py", line 85, in <module>
    process_folder(sys.argv[1], sys.argv[2])
  File "/Users/quirin/tools/constraction/src/annotator.py", line 79, in process_folder
    annotations_to_csv(annotation_folder, csv_folder)
  File "/Users/quirin/tools/constraction/src/annotator.py", line 45, in annotations_to_csv
    supersenses = pd.read_csv(os.path.join(input_folder, file_id + ".supersense"),
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 948, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 617, in _read
    return parser.read(nrows)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1748, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 234, in read
    chunks = self._reader.read_low_memory(nrows)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "parsers.pyx", line 843, in pandas._libs.parsers.TextReader.read_low_memory
  File "parsers.pyx", line 904, in pandas._libs.parsers.TextReader._read_rows
  File "parsers.pyx", line 879, in pandas._libs.parsers.TextReader._tokenize_rows
  File "parsers.pyx", line 890, in pandas._libs.parsers.TextReader._check_tokenize_status
  File "parsers.pyx", line 2058, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: EOF inside string starting at row 269656

Second, if I ignore this error and try to run Streamlit, I get this error:

streamlit run server.py

2024-01-22 17:27:35.629 Uncaught app exception
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 535, in _run_script
    exec(code, module.__dict__)
  File "/Users/quirin/tools/constraction/src/server.py", line 9, in <module>
    import mining
ImportError: dlopen(/Users/quirin/tools/constraction/src/mining.so, 0x0002): tried: '/Users/quirin/tools/constraction/src/mining.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/quirin/tools/constraction/src/mining.so' (no such file), '/Users/quirin/tools/constraction/src/mining.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
2024-01-22 17:28:04.340 Uncaught app exception
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 535, in _run_script
    exec(code, module.__dict__)
  File "/Users/quirin/tools/constraction/src/server.py", line 9, in <module>
    import mining
ImportError: dlopen(/Users/quirin/tools/constraction/src/mining.so, 0x0002): tried: '/Users/quirin/tools/constraction/src/mining.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/quirin/tools/constraction/src/mining.so' (no such file), '/Users/quirin/tools/constraction/src/mining.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))