Extractors not working - Githubissues

HTsybenko commented 2 weeks ago

1) Python 3.12.4, Ubuntu 24.04, pasta-eln 2.6.0b in venv 2) Extractor files copied into the corresponding folder of pasta-eln 3) In pasta-eln GUI, go to System and Update extractor list 4) The list of extractors and methods is displayed correctly:

png: measurement/image/crop: Crop 3/4 of the image measurement/image: Default | uncropped txt: measurement/indentation/procedure: plot indentation procedure measurement/indentation/force_depth: Default | plot force-depth curve measurement/ebsd/rd: plot map in RD measurement/ebsd/td: plot map in TD measurement/ebsd/nd: Default | plot map in ND direction ser: measurement/image/tem/threshold: crop contrast to 200-500 measurement/image/tem/color: colorful output measurement/image/tem/: Default | unchanged ipynb: procedure/jupyter: Default cpr: measurement/ebsd/phase: Phase map with MAD measurement/ebsd/rd: plot map in RD measurement/ebsd/td: plot map in TD measurement/ebsd/nd: Default | plot map in ND direction md: procedure/markdown: Default ang: measurement/ebsd/rd: plot map in RD measurement/ebsd/td: plot map in TD measurement/ebsd/nd: Default | plot map in ND direction gz: measurement/MRT/3D: plot 3D representation measurement/MRT/3D-2: plot 3D representation 2 measurement/MRT/inverted: plot in color measurement/MRT: Default | plot gray scale py: procedure/python: Default gwy: measurement/confocal/image: image measurement/confocal/image/no_colorbar: image without colorbar measurement/confocal/height: height measurement/confocal/height/no_colorbar: Default | height without colorbar crc: -/crcFile: Default tif: measurement/image/no_scale/adaptive: plot with no scale bar, adaptive grayscales and crop bottom white databar measurement/image/scale/adaptive: plot with scale bar, adaptive grayscales and crop bottom white databar measurement/image/no_scale/rescale: plot with no scale bar, rescale grayscales and crop bottom white databar measurement/image/scale/rescale: Default | plot with scale bar, rescale grayscales and crop bottom white databar json: procedure/json: Default csv: measurement/dkfi/fit: With fitting line measurement/dkfi: Default | raw data measurement/table/red: Draw with red curve measurement/table/blue: Default | blueish curve hdf5: measurement/indentation/procedure: plot indentation procedure measurement/indentation/force_depth: Default | plot force-depth curve xlsx: measurement/table: Default xls: measurement/indentation/procedure: plot indentation procedure measurement/indentation/force_depth: Default | plot force-depth curve jpg: measurement/image: Default org: procedure/org-mode: Default jpeg: measurement/image: Default

5) Go to System and Test extraction from a file

This leads to the error:

check file: /home/htsybenko/test datasets/Collection_HT/TEM/twin-16.02.34 CCD Acquire_1.tif use extractor: /home/htsybenko/pasta-env/lib/python3.12/site-packages/pasta_eln/Extractors/extractor_tif.py Python error in extractor Please visit website Traceback (most recent call last): File "/home/htsybenko/pasta-env/lib/python3.12/site-packages/pasta_eln/backend.py", line 580, in testExtractor module = importlib.import_module(pyFile[:-3]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/importlib/init.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1387, in _gcd_import ModuleNotFoundError: No module named 'micromechanics'

I tried with tif, gwy, hdf5, other extractors might have the same response.

SteffenBrinckmann commented 2 weeks ago

The default extractors should work. You seem to use custom extractors, which have additional requirements. Has nothing to do with Pasta. In your case, there might be a requirements.txt file and use "pip install -r requirements.txt"

HTsybenko commented 2 weeks ago

@SteffenBrinckmann It would be nice to have this mentioned on the IEK-2 Jugit page too.

A related error occurs during installation of orix package and has to be reported here:

Building wheels for collected packages: pycifrw Building wheel for pycifrw (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for pycifrw (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [47 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-312 creating build/lib.linux-x86_64-cpython-312/CifFile copying src/init.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/CifFile_module.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/yapps3_compiled_rt.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/YappsStarParser_1_1.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/YappsStarParser_1_0.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/YappsStarParser_STAR2.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/YappsStarParser_2_0.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/StarFile.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/TypeContentsParser.py -> build/lib.linux-x86_64-cpython-312/CifFile copying src/parsetab.py -> build/lib.linux-x86_64-cpython-312/CifFile creating build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/parsetab.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/drel_runtime.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/drel_ast_yacc.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/drel_lex.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/py_from_ast.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel copying src/drel/init.py -> build/lib.linux-x86_64-cpython-312/CifFile/drel running build_ext building 'CifFile.StarScan' extension creating build/temp.linux-x86_64-cpython-312 creating build/temp.linux-x86_64-cpython-312/src creating build/temp.linux-x86_64-cpython-312/src/lib x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/htsybenko/pasta-env/include -I/usr/include/python3.12 -c src/lib/lex.yy.c -o build/temp.linux-x86_64-cpython-312/src/lib/lex.yy.o star.l: In function ‘star_scanner’: star.l:111:26: warning: suggest parentheses around assignment used as truth value [-Wparentheses] lex.yy.c:195:36: warning: comparison of integer expressions of different signedness: ‘int’ and ‘yy_size_t’ {aka ‘long unsigned int’} [-Wsign-compare] lex.yy.c:213:9: note: in expansion of macro ‘YY_LESS_LINENO’ star.l:117:24: note: in expansion of macro ‘yyless’ star.l:136:26: warning: suggest parentheses around assignment used as truth value [-Wparentheses] lex.yy.c:195:36: warning: comparison of integer expressions of different signedness: ‘int’ and ‘yy_size_t’ {aka ‘long unsigned int’} [-Wsign-compare] lex.yy.c:213:9: note: in expansion of macro ‘YY_LESS_LINENO’ star.l:142:24: note: in expansion of macro ‘yyless’ At top level: lex.yy.c:1394:16: warning: ‘input’ defined but not used [-Wunused-function] lex.yy.c:1349:17: warning: ‘yyunput’ defined but not used [-Wunused-function] x86_64-linux-gnu-gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/htsybenko/pasta-env/include -I/usr/include/python3.12 -c src/lib/py_star_scan.c -o build/temp.linux-x86_64-cpython-312/src/lib/py_star_scan.o src/lib/py_star_scan.c:2:10: fatal error: Python.h: No such file or directory 2 | #include "Python.h" | ^~~~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for pycifrw

The issue is then fixed by "sudo apt-get install python3.12-dev".

SteffenBrinckmann commented 2 weeks ago

Let's keep a record so we do not forget

HTsybenko commented 2 weeks ago

@SteffenBrinckmann Since some extractors were added and some updated on Jugit recently, they also should be retested.

I have checked some with the files that I had using 2.6.0b1, Python 3.12 environment. Please see the "Extractors 09.07" folder on sciebo for individual reports. Some formats need additional modules included in the requirements.txt, but still have some errors after the modules are installed.

If you happen to have files of other newly added or uncommon formats, you can provide them to me for testing as well.

PASTA-ELN / pasta-eln

Extractors not working #311