Closed ilayn closed 1 year ago
EDIT: I made some modifications to the library to get your example to work. Make sure you update to fmodpy=1.5.5
!
Thanks for using it! I'll do my best to help, acknowledging I don't actively test on Windows (so there might be some hiccups). On my Windows test machine I only verify that things work on the subsystem for Linux.
If you're using gfortran
to compile and have OpenBLAS
installed in a place that gfortran
understands, then the -lopenblas
option should work. If you want to permanently configure your local install of fmodpy
to use that flag when linking BLAS
you could do:
python3 -c "import fmodpy; fmodpy.configure(link_blas='-lopenblas')"
Then in your Python source code you could just do
import fmodpy
MB01OE = fmodpy.fimport(
"MB01OE.f",
blas=True,
)
Behind the scenes, the link_blas
configuration string will be inserted into the build commands, and the fact that the file ends in .f
will trigger end_is_named=False
(because fixed format Fortran files have END
statements for subroutines that are not explicitly named).
If you just want to customize the compilation arguments for the current project without changing your global configuration, you can manually set any arguments to the compiler / linker like this:
import fmodpy
MB01OE = fmodpy.fimport(
"MB01OE.f",
f_compiler_args="-fPIC -shared -O3 -lopenblas",
)
And if you want information about everything the wrapper is doing in the process, include the verbose=True
option when calling fimport
.
Let me know how it goes and I'm happy to keep helping you debug!
If you want a slightly more complicated example https://github.com/SLICOT/SLICOT-Reference/blob/main/src/MB01OD.f depends on both BLAS and
MB01OE.f
mentioned above which is my second mini-goal if I can get this working.
Ohh thanks for the challenge. I got this to work by cloning that repository, navigating to the src
directory, and running:
import fmodpy
mb10d = fmodpy.fimport('MB01OD.f', dependencies=['MB01OE.f'], blas=True, lapack=True)
Notably, I needed lapack=True
because the code had a missing dependency of dlascl
. I also needed to adjust my parsing logic to handle some fixed format details differently, so make sure you update to fmodpy==1.5.5
!
As a bonus, I wanted to see if I could create a wrapper for the entire library. So I ran the following, and it appears to have worked!
cd SLICOT-Reference/src
cat *.f > SLICOT.f
python3 -m fmodpy SLICOT.f blas=True lapack=True f_compiler_args='-fPIC -shared -O3 -std=legacy'
python3 -c "import SLICOT; help(SLICOT)"
The potentially cryptic-seeming f_compiler_args
is just the standard arguments for compiling a shared library, maximum optimization, and importantly the -std=legacy
flag to tell the compiler that it is working with F77
code. Overall, the command line call to fmodpy
that I ran above is equivalent to the following in Python3 code:
import fmodpy
SLICOT = fmodpy.fimport(
"SLICOT.f",
blas=True,
lapack=True,
f_compiler_args="-fPIC -shared -O3 -std=legacy"
)
On your machine, all of this will only work if you make sure to configure the link_blas
and link_lapack
flags based on your local installation of those libraries. Some setup will be required, like fmody.configure(link_blas='-lopenblas', link_lapack='-lopenlapack')
or whatever the correct link flag is for linking to the LAPACK code.
Oh my, thank you so much for this detailed explanation. I'm going to try right away and report back. Much appreciated!
After trying a few things I got the following situation
C:\Users\ilhan\Documents\GitHub\SLICOT-Reference\src [meson]> python -m fmodpy .\MB01OE.f f_compiler_args="-fPIC -shared -O3 -std=legacy -L/opt/64/lib -lopenblas"
C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\fmodpy.py:414: UserWarning:
Automatic compilation is taking longer than expected, consider
providing explicitly ordered dependencies (precompiled or not) with
fmodpy.fimport('<target>', dependencies=['<path>', ...], ...)
This is due to the fact that gfortran is running through MSYS2 UCRT64 system and I built OpenBLAS in some folder that the gfortran doesn't know about namely C:\opt\64{\bin, \include, \lib}
. Hence my strange -L
flag. But this leads to a hangup apparently. I wouldn't need blas or lapack flags since all symbols are inside libopenblas.dll. On Linux it's much more straightforward on anything MS.. .sigh. But at least I know that on Linux it will definitely work :)
Is there a way to actually debug why this hang-up might have happened?
Yeah, to debug try setting verbose=True
as an option. You should see very early on that it tries to compile the source with something like:
gfortran -fPIC -shared -O3 -std=legacy -L/opt/64/lib -lopenblas MB01OE.f
And it is probably failing to compile.
Hanging on autocompilation in that project will happen because there are hundreds of files. If it fails to compile the source directly the first time, then it will try to recompile the source in combination with all possible files in the directory 1 at a time to identify the unmet dependency. If you don't want it to try to automatically compile things, then you can set autocompile=False
. However to do that, you'll need to make sure you precompile an object file and provide an argument that links with the precompiled object.
It actually looks like the following command succeeds for me (mind the slight flag modifications):
python3 -m fmodpy MB01OE.f f_compiler_args='-fPIC -shared -O3 -std=legacy -lblas -llapack' verbose=True autocompile=False
Indeed that's what I have tried here is a single file attempt
C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile> dir
Directory: C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 18/02/2023 21:03 9080 MB01OE.f
C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile> python -m fmodpy .\MB01OE.f f_compiler_args="-fPIC -shared -O3 -std=legacy -L/opt/64/lib -lopenblas"
Traceback (most recent call last):
File "C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\__main__.py", line 33, in <module>
fimport(file_path, **command_line_args)
File "C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\fmodpy.py", line 189, in fimport
built, failed = autocompile_files(build_dir, dependencies)
File "C:\Users\ilhan\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\fmodpy.py", line 485, in autocompile_files
raise(CompileError(f"Failed to compile {failed_dependencies}.\n Current compilation arguments are:\n {f_compiler_args}\n Is a necessary compilation argument or dependency missing?"))
fmodpy.exceptions.CompileError: Failed to compile {'MB01OE.f'}.
Current compilation arguments are:
['-fPIC', '-shared', '-O0', '-std=legacy', '-L/opt/64/lib', '-lopenblas']
Is a necessary compilation argument or dependency missing?
Then I tried inside IPython terminal
In [5]: fmodpy.fimport(
...: "MB01OE.f",
...: blas=False,
...: lapack=False,
...: f_compiler_args="-fPIC -O2 -shared -L:/opt/64/bin/libopenblas.dll",
...: verbose=True
...: )
______________________________________________________________________
fimport
fmodpy configuration:
autocompile = True
blas = False
config_file = '.fmodpy.py'
debug_line_numbers = False
end_is_named = True
f_compiler = 'gfortran'
f_compiler_args = ['-fPIC', '-O2', '-shared', '-L:/opt/64/bin/libopenblas.dll']
home_directory = 'C:\\Users\\ilhan'
implicit_typing = False
lapack = False
libraries = ['C:\\Users\\ilhan\\AppData\\Local\\Programs\\Python\\Python310\\lib\\site-packages\\numpy', '/usr/lib', '/lib', '/usr/lib64', '/lib64', '/usr/lib32', '/lib32', '/opt/homebrew/Cellar/openblas', '/opt/homebrew/Cellar/libomp']
library_extensions = ['so', 'dylib']
library_recursion = 2
link_blas = ['-lblas']
link_lapack = ['-lblas', '-llapack']
link_omp = ['-fopenmp']
log_file = <colorama.ansitowin32.StreamWrapper object at 0x000002815B4C8B20>
omp = False
overwrite = False
rebuild = False
show_warnings = True
symbol_command = 'nm -gU "{path}" 2> /dev/null || nm -gD "{path}" 2> /dev/null'
verbose = True
verbose_module = True
wait_warning_sec = 5
wrap = True
=============================================================================
Input file directory: C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile
Input file name: MB01OE.f
Base module name: MB01OE
Using build dir: temporary directory
fortran wrapper: MB01OE_c_wrapper.f90
python wrapper: MB01OE_python_wrapper.py
Output module path: C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile
=============================================================================
Using temporary build directory at 'C:\Users\ilhan\AppData\Local\Temp\tmp2n7ha4p_'.
Build directory is different from source directory..
sym-linking 'C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile\MB01OE.f' to 'C:\Users\ilhan\AppData\Local\Temp\tmp2n7ha4p_\MB01OE.f'
Attempting to autocompile..
reading 'MB01OE.f' to check if it can be autocompiled.. yes.
Compiling 'MB01OE.f'..
gfortran -fPIC -O0 -shared -L:/opt/64/bin/libopenblas.dll -o .\fmodpy_get_size MB01OE.f
failed.
NAME: MB01OE.f
----------------------------------------------------------------------
STANDARD ERROR:
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xed): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x115): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x13a): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x178): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x1b0): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x28a): undefined reference to `xerbla_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x39f): undefined reference to `dlaset_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x441): undefined reference to `dlascl_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x556): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x616): undefined reference to `dcopy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x67f): undefined reference to `dscal_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x735): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x801): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x8a8): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x974): undefined reference to `dcopy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x9f9): undefined reference to `dscal_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xae1): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xba0): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xc80): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xd50): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xe46): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xeb1): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0xfda): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x112b): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x1283): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccOdCC5g.o:MB01OE.f:(.text+0x12ee): more undefined references to `ddot_' follow
collect2.exe: error: ld returned 1 exit status
----------------------------------------------------------------------
Compiling 'MB01OE.f'..
gfortran -fPIC -O0 -shared -L:/opt/64/bin/libopenblas.dll -o .\fmodpy_get_size MB01OE.f
failed.
NAME: MB01OE.f
----------------------------------------------------------------------
STANDARD ERROR:
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xed): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x115): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x13a): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x178): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x1b0): undefined reference to `lsame_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x28a): undefined reference to `xerbla_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x39f): undefined reference to `dlaset_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x441): undefined reference to `dlascl_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x556): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x616): undefined reference to `dcopy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x67f): undefined reference to `dscal_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x735): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x801): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x8a8): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x974): undefined reference to `dcopy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x9f9): undefined reference to `dscal_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xae1): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xba0): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xc80): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xd50): undefined reference to `daxpy_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xe46): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xeb1): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0xfda): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x112b): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x1283): undefined reference to `ddot_'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:\Users\ilhan\AppData\Local\Temp\ccgo6Giz.o:MB01OE.f:(.text+0x12ee): more undefined references to `ddot_' follow
collect2.exe: error: ld returned 1 exit status
----------------------------------------------------------------------
Failed to compile 'MB01OE.f'.
Failed to compile 'MB01OE.f'.
---------------------------------------------------------------------------
CompileError Traceback (most recent call last)
<ipython-input-5-a33861fefd15> in <cell line: 1>()
----> 1 fmodpy.fimport(
2 "MB01OE.f",
3 blas=False,
4 lapack=False,
5 f_compiler_args="-fPIC -O2 -shared -L:/opt/64/bin/libopenblas.dll",
~\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\fmodpy.py in fimport(input_fortran_file, name, build_dir, output_dir, dependencies, symbols, **kwargs)
187 if autocompile:
188 print("Attempting to autocompile..")
--> 189 built, failed = autocompile_files(build_dir, dependencies)
190 dependencies = dependencies + [f for f in built if (f not in dependencies)]
191 print()
~\AppData\Local\Programs\Python\Python310\lib\site-packages\fmodpy\fmodpy.py in autocompile_files(build_dir, dependencies)
483 if (len(failed_dependencies) > 0):
484 from fmodpy.exceptions import CompileError
--> 485 raise(CompileError(f"Failed to compile {failed_dependencies}.\n Current compilation arguments are:\n {f_compiler_args}\n Is a necessary compilation argument or dependency missing?"))
486 # Return the list of files that were successfully compiled in
487 # the order they were compiled and the files that failed to compile.
CompileError: Failed to compile {'MB01OE.f'}.
Current compilation arguments are:
['-fPIC', '-O0', '-shared', '-L:/opt/64/bin/libopenblas.dll']
Is a necessary compilation argument or dependency missing?
So apparently linking is not working and optimization flag is somehow going back to 0 (nevermind the dll silliness I was testing things out)
However, if I run directly the gfortran command on the prompt
C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile> gfortran -v -shared .\MB01OE.f -o someoutput.o -L"C:\opt\64\lib" -lopenblas
Driving: C:\msys64\ucrt64\bin\gfortran.exe -v -shared .\MB01OE.f -o someoutput.o -LC:\opt\64\lib -lopenblas -l gfortran -shared-libgcc
Using built-in specs.
COLLECT_GCC=C:\msys64\ucrt64\bin\gfortran.exe
COLLECT_LTO_WRAPPER=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-12.2.0/configure --prefix=/ucrt64 --with-local-prefix=/ucrt64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/ucrt64/include --libexecdir=/ucrt64/lib --enable-bootstrap --enable-checking=release --with-arch=nocona --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++,jit --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts --enable-libstdcxx-time --disable-libstdcxx-pch --enable-lto --enable-libgomp --disable-multilib --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/ucrt64 --with-mpfr=/ucrt64 --with-mpc=/ucrt64 --with-isl=/ucrt64 --with-pkgversion='Rev10, Built by MSYS2 project' --with-bugurl=https://github.com/msys2/MINGW-packages/issues --with-gnu-as --with-gnu-ld --disable-libstdcxx-debug --with-boot-ldflags=-static-libstdc++ --with-stage1-ldflags=-static-libstdc++
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (Rev10, Built by MSYS2 project)
COLLECT_GCC_OPTIONS='-v' '-shared' '-o' 'someoutput.o' '-LC:\opt\64\lib' '-shared-libgcc' '-mtune=generic' '-march=nocona' '-dumpdir' 'someoutput.o-'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/f951.exe .\MB01OE.f -ffixed-form -quiet -dumpdir someoutput.o- -dumpbase MB01OE.f -dumpbase-ext .f -mtune=generic -march=nocona -version -fintrinsic-modules-path C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/finclude -o C:\Users\ilhan\AppData\Local\Temp\ccEIFeO7.s
GNU Fortran (Rev10, Built by MSYS2 project) version 12.2.0 (x86_64-w64-mingw32)
compiled by GNU C version 12.2.0, GMP version 6.2.1, MPFR version 4.2.0, MPC version 1.3.1, isl version isl-0.25-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU Fortran2008 (Rev10, Built by MSYS2 project) version 12.2.0 (x86_64-w64-mingw32)
compiled by GNU C version 12.2.0, GMP version 6.2.1, MPFR version 4.2.0, MPC version 1.3.1, isl version isl-0.25-GMP
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-v' '-shared' '-o' 'someoutput.o' '-LC:\opt\64\lib' '-shared-libgcc' '-mtune=generic' '-march=nocona' '-dumpdir' 'someoutput.o-'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/as.exe -v -o C:\Users\ilhan\AppData\Local\Temp\ccqAWu5h.o C:\Users\ilhan\AppData\Local\Temp\ccEIFeO7.s
GNU assembler version 2.40 (x86_64-w64-mingw32) using BFD version (GNU Binutils) 2.40
Reading specs from C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/libgfortran.spec
rename spec lib to liborig
COLLECT_GCC_OPTIONS='-v' '-shared' '-o' 'someoutput.o' '-LC:\opt\64\lib' '-shared-libgcc' '-mtune=generic' '-march=nocona' '-dumpdir' 'someoutput.o-'
COMPILER_PATH=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/;C:/msys64/ucrt64/bin/../lib/gcc/;C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/bin/
LIBRARY_PATH=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/;C:/msys64/ucrt64/bin/../lib/gcc/;C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib/../lib/;C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../lib/;C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib/;C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../
COLLECT_GCC_OPTIONS='-v' '-shared' '-o' 'someoutput.o' '-LC:\opt\64\lib' '-shared-libgcc' '-mtune=generic' '-march=nocona' '-dumpdir' 'someoutput.o.'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/collect2.exe -plugin C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/liblto_plugin.dll -plugin-opt=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/lto-wrapper.exe -plugin-opt=-fresolution=C:\Users\ilhan\AppData\Local\Temp\ccLLYuor.res -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lquadmath -plugin-opt=-pass-through=-lm -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-ladvapi32 -plugin-opt=-pass-through=-lshell32 -plugin-opt=-pass-through=-luser32 -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -m i386pep --shared -Bdynamic -e DllMainCRTStartup --enable-auto-image-base -o someoutput.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../lib/dllcrt2.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/crtbegin.o -LC:\opt\64\lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0 -LC:/msys64/ucrt64/bin/../lib/gcc -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib/../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../../../x86_64-w64-mingw32/lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/../../.. C:\Users\ilhan\AppData\Local\Temp\ccqAWu5h.o -lopenblas -lgfortran -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 -lquadmath -lm -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/12.2.0/crtend.o
COLLECT_GCC_OPTIONS='-v' '-shared' '-o' 'someoutput.o' '-LC:\opt\64\lib' '-shared-libgcc' '-mtune=generic' '-march=nocona' '-dumpdir' 'someoutput.o.'
C:\Users\ilhan\Documents\pydump\mesonslicot\singlefile>
it manages to link and compile it. Very intriguing
Well the -O0
that you're seeing only happens while it's trying to automatically compile the codes to check for the type sizes of the routine inputs & outputs. So the "final" compiled object will use your specified optimization argument.
Your compilation that is working might still be failing to resolve some symbols, but you won't see that until you try to compile and run an executable (or include the library in the python runtime).
I think the culprit here is something about the .dll
file that you're linking not having all of the expected symbols defined. But I'd need to try to recreate the environment to play with it.
Do you have the sequence of MS terminal commands that you used to arrive here? I can try and run it on my Windows machine to see if I get similar results.
Sure there are none :)
If you download any Openblas lib from Anaconda it already has the same structure I have. I'd say either this https://anaconda.org/multibuild-wheels-staging/openblas-libs/v0.3.21/download/openblas-v0.3.21-win_amd64-gcc_10_3_0.zip
or 64-bit index version
and that's it.
mingw-w64-x86_64-toolchain
C:\msys64\ucrt64\bin
folder on the pathif yes then you have my machine :)
Okay, so here's what I've done:
64
directory, moved to C:\opt\
C:\msys64
pacman -Syu
three times:
The first time it installed some things that required a restart of the application.
The second time is installed a few more things but did not require a restart.
The third time it said everything was up to date.pacman -S mingw-w64-x86_64-toolchain
.PATH
to include C:\msys64\ucrt64\bin
(even though that folder is empty..)gfortran
is not found as a command, neither is it found in the MSYS environment.Ideas?
Probably you didn't run the pacman -S mingw-w64-x86_64-toolchain
command from a UCRT64 command prompt but from another prompt maybe msys2? They are like separate environments, comparable to venv's of python. That might be the reason
That might have been what I did. I deleted everything and retried, being careful to always use the UCRT64 environment. Same outcome. Still can't find where it's putting gfortran
.
I have some chores I need to do, so I'll have to wait to pick this up tomorrow.
In the meantime, if you want to keep trying, then I'd suggest getting anything that requires BLAS and LAPACK functions to compile, link, and run in your environment (without python). Once you can do that, linking it correctly to python should be easy.
No problem at all. I'll double check that it actually installs gfortran with that toolchain command. Maybe there was an additional package that I forgot. Regardless, thank you for your wonderful support.
Here are some screenshots from my system
These are the user environment variables
Oh damn. Completely my mistake the package name needs to be
pacman -S mingw-w64-ucrt-x86_64-toolchain
Picked up the wrong package. This one is the one suitable for the ucrt, the one I mentioned above is for regular environment. So sorry about that. I'm still getting used to this weird way of doing things on windows.
Got around my original problem with an alternative path. But thank you for your support anyways, much appreciated!
Hi! thank you for this great tool. After wrestling with f2py for way too long this is indeed great to have.
For simple tasks I've managed to make it working so far and then I wanted to step up my game with linking to OpenBLAS on windows. It has its own complications obviously but what I am trying to do can be described as;
1- Let's say I downloaded OpenBLAS from Anaconda https://anaconda.org/conda-forge/libopenblas/files 2- My target is to wrap the FORTRAN library https://github.com/SLICOT/SLICOT-Reference but in it let's pick a very simple file
/src/MB01OE.f
that has only BLAS dependencies but the rest is standalone; a good candidate for a linking exercise. 3- Now I know somehow I have to usefmodpy.config.{something}
to arrive at the commandgfortran -shared .\MB01OE.f -o .\MB01OE.o -L"C:\opt\64\lib" -lopenblas
(or I completely misunderstood the configuration mechanism).I have not been able to do this step go forward yet. Could you point me to any direction ? Or is this even possible with openblas both having blas and lapack symbols.
If you want a slightly more complicated example https://github.com/SLICOT/SLICOT-Reference/blob/main/src/MB01OD.f depends on both BLAS and
MB01OE.f
mentioned above which is my second mini-goal if I can get this working.