search

qgis / QGIS

QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS)
https://qgis.org
GNU General Public License v2.0
10.65k stars 3.01k forks source link

Windows Build - not building missing Qsci/qscimod6.sip #59134

Open danodom-tcb opened 1 month ago

danodom-tcb commented 1 month ago

What is the bug or the crash?

I have followed the recently updated instructions and have installed a # of python packages, but getting the following error on building the python_module_qgis__gui project. At this point, I am not sure how to resolve this dependency in the solution. I have all of the other 500+ projects building successfully in Visual Studio 2022.

4>sip-build: C:\QGIS\QGIS\build\python\gui\gui.sip: line 15: column 9: 'Qsci/qscimod6.sip' could not be found 4>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\QGIS\QGIS\build\python\gui\gui.sip' exited with code 1. 4>Done building project "python_module_qgis__gui.vcxproj" -- FAILED.

Steps to reproduce the issue

Build the "qgis"solution built after running the following commands with the corresponding values.

We assume you have a copy of the QGIS source code available

and have changed the working directory into it

Configure

cmake -S . \ -B build \ -DSDK_PATH="path/to/vcpkg-export-[date]" \ -DBUILD_WITH_QT6=ON \ -DWITH_QTWEBKIT=OFF \ -DVCPKG_TARGET_TRIPLET=x64-windows-release

Versions

I am unable to get to the help menu as the error is occurring during the build (@jef-n thank you for updating the install.md recently as I am getting close to building QGIS on Windows 11 and just need some help getting to the finish line.

Supported QGIS version

New profile

Additional context

No response

m-kuhn commented 1 month ago

Could you try deleting the folder vcpkg/ports/py-qscintilla from the source tree and rerun cmake?

... p.s. sorry, that would require a build with dependencies, I'll follow up with a PR soon. Not sure yet if this dolves the problem though

danodom-tcb commented 1 month ago

Could you try deleting the folder vcpkg/ports/py-qscintilla from the source tree and rerun cmake?

... p.s. sorry, that would require a build with dependencies, I'll follow up with a PR soon. Not sure yet if this dolves the problem though

@m-kuhn , would you be changing something in the QGIS source, or something in the QGIS Master SDK?

I am working on trying to get a build of QGIS on Windows 11 and it feels like I am really close.

m-kuhn commented 1 month ago

In the SDK (which is produced from the vcpkg subfolder in QGIS)

Do you find a file qscimod6.sip somewhere in the SDSK?

danodom-tcb commented 1 month ago

@m-kuhn yes, here.

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\site-packages\PyQt6\bindings\Qsci

danodom-tcb commented 1 month ago

@m-kuhn do you need me to delete or replace this file? Just let me know and happy to try whatever is needed here.

m-kuhn commented 1 month ago

can you run 

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.__file__), end='')
m-kuhn commented 1 month ago

and ideally, if you could run cmake --trace-expand build and post the output to gist.github.com and link to this here, that would also be great.

danodom-tcb commented 1 month ago

can you run

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.__file__), end='')

I did this and same error on the build.

image

m-kuhn commented 1 month ago

can you run

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.__file__), end='')

I did this and same error on the build.

I am not yet sure where exactly that problem is originating from, so this is just debugging so far. What was the output of this?

danodom-tcb commented 1 month ago

cmake --trace-expand buil

I am running the Build command on the VS solution for the project that is failing, but let me go out and run this command and I'll get you output.

danodom-tcb commented 1 month ago

can you run

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.__file__), end='')

I did this and same error on the build.

I am not yet sure where exactly that problem is originating from, so this is just debugging so far. What was the output of this?

C:>C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.file), end='') Added vcpkg bin path to search for dlls. Path added: C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\bin

danodom-tcb commented 1 month ago

can you run

C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.__file__), end='')

I did this and same error on the build.

I am not yet sure where exactly that problem is originating from, so this is just debugging so far. What was the output of this?

C:>C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\python.exe -c "import os; import PyQt6; print(os.path.dirname(PyQt6.file), end='') Added vcpkg bin path to search for dlls. Path added: C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\bin

Hmm.. thinking out loud here.. does this need to be added to the Environment Variables in Windows as part of the path?

m-kuhn commented 1 month ago

No, that's not needed. This is a python internal mechanism to look for dlls in this path. What we are interested right now is to find out, where sip looks for additional files. I.e. the content of PYQT_SIP_DIR when it reaches here https://github.com/qgis/QGIS/blob/master/cmake/SIPMacros.cmake#L104 This is initialized by https://github.com/qgis/QGIS/blob/master/cmake/FindPyQt6.cmake

danodom-tcb commented 1 month ago

SIPMacros doesn't appear to be in the solution created when the following command is ran. I was looking for a line like that in the solution through all projects, and didn't find it.

image

The output of this creates a Visual Studio solution that I am building. The other python projects in here are building, just not this last one. image

danodom-tcb commented 1 month ago

No, that's not needed. This is a python internal mechanism to look for dlls in this path. What we are interested right now is to find out, where sip looks for additional files. I.e. the content of PYQT_SIP_DIR when it reaches here https://github.com/qgis/QGIS/blob/master/cmake/SIPMacros.cmake#L104 This is initialized by https://github.com/qgis/QGIS/blob/master/cmake/FindPyQt6.cmake

The only thing that I found in the solution and this was in the QGIS\CMakeLists.txt

image

I am not seeing any reference to ${PYQT6_SIP_DIR} anywhere, so I am not sure where this is getting set. Is this read in from somewhere?

m-kuhn commented 1 month ago

If you could add a line

message(FATAL_ERROR "pyqt6 dir ${PYQT6_SIP_DIR}")

Just there, we would also get our answer

danodom-tcb commented 1 month ago

If you could add a line

message(FATAL_ERROR "pyqt6 dir ${PYQT6_SIP_DIR}")

Just there, we would also get our answer

@m-kuhn

Everything in the build is contained within the "build" folder that is created using the screenshot I posted above. I am wondering if that parameter needs to be set as part of that cmake above that sets the SDK path. I am not building the base CMake, but the solution that's been built out.

image

Also, also, I did a grep on the QGIS folder and this is all I found.

image

m-kuhn commented 1 month ago
danodom-tcb commented 1 month ago
  • You did only grep some files, e.g. not .cmake files
  • Could you run cmake --trace-expand build and share the output?

from where? root of the build solution? or root of the source directory?

danodom-tcb commented 1 month ago
  • You did only grep some files, e.g. not .cmake files

I basically had it search everything looking for that.. cmake files included as well as basically anything not binary.

danodom-tcb commented 1 month ago
  • You did only grep some files, e.g. not .cmake files
  • Could you run cmake --trace-expand build and share the output (from outside vs)?

This thing outputted like a couple of thousand lines, but am guessing you are looking for this portion..

C:/QGIS/vcpkg-export-20241015-154141/scripts/buildsystems/vcpkg.cmake(872):  if(z_vcpkg_find_package_backup_id LESS 0 )
C:/QGIS/QGIS/CMakeLists.txt(1103):  set(PYQT_FOUND TRUE )
C:/QGIS/QGIS/CMakeLists.txt(1104):  set(PYQT_SIP_FLAGS  )
C:/QGIS/QGIS/CMakeLists.txt(1105):  set(PYQT_SIP_DIR C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings )
C:/QGIS/QGIS/CMakeLists.txt(1106):  message(FATAL_ERROR pyqt6 dir C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings )
CMake Error at CMakeLists.txt:1106 (message):
  pyqt6 dir
  C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings

-- Configuring incomplete, errors occurred!
danodom-tcb commented 1 month ago

So, it looks like I played around with it and got it to resolve that dependency. I ended up copying the Qsci folder from the vcpkg folder structure under "C:\Program Files\Python313\Lib\site-packages\PyQt6\bindings" and then it ended up resolving that dependency.

However, now I am on to the next issue after this resolved. I can open another issue or we can keep going in this thread. Seems like it would be good to know why copying the folder worked when it was pointing to the other folder.

However, these looks somewhat related. 1. 3>Generating ui_qgsdirectionallightwidget.h 2>Added vcpkg bin path to search for dlls. Path added: C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\bin 3>Generating ui_qgsdiscoverrelationsdialogbase.h 3>Generating ui_qgsdisplayanglebase.h 2>-- Found PyQt6 version: 6.7.1 2>CMake Error at CMakeLists.txt:1106 (message): 2> pyqt6 dir 2> C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings 2> 2> 2>-- Configuring incomplete, errors occurred! 3>Generating ui_qgsdualviewbase.h 2>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\QGIS\QGIS\build\CMakeFiles\4f936f24e2671abb107f35661b43692c\version.rule;C:\QGIS\QGIS\CMakeLists.txt' exited with code 1. 2>Done building project "version.vcxproj" -- FAILED.

2. 1>Added vcpkg bin path to search for dlls. Path added: C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\bin 1>-- Found PyQt6 version: 6.7.1 1>CMake Error at CMakeLists.txt:1106 (message): 1> pyqt6 dir 1> C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings 1> 1> 1>-- Configuring incomplete, errors occurred! 1>CMake Configure step failed. Build files cannot be regenerated correctly. Attempting to stop IDE build. 1>The system cannot find the batch label specified - VCEnd 1>C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(237,5): error MSB8066: Custom build for 'C:\QGIS\QGIS\build\CMakeFiles\4f936f24e2671abb107f35661b43692c\generate.stamp.rule' exited with code 1. 1>Done building project "ZERO_CHECK.vcxproj" -- FAILED.

m-kuhn commented 1 month ago

So, it looks like I played around with it and got it to resolve that dependency. I ended up copying the Qsci folder from the vcpkg folder structure under "C:\Program Files\Python313\Lib\site-packages\PyQt6\bindings" and then it ended up resolving that dependency.

oh, it looks like there are some leftovers from system python (not the one in the SDK), possibly an env variable PYTHONPATH or similar?

2>CMake Error at CMakeLists.txt:1106 (message): 2> pyqt6 dir 2> C:\QGIS\vcpkg-export-20241015-154141\installed\x64-windows-release\tools\python3\Lib\PyQt6/bindings

do you still have the message(FATAL_ERROR ...) from above in there? That needs to be removed again

danodom-tcb commented 1 month ago

do you still have the message(FATAL_ERROR ...) from above in there? That needs to be removed again

oops, yes I do and forgot about that. With building the solution, does it still reference that makefile? Taking out just in case..

Thanks for pointing that out and I'll remove and see what happens.

Also, greatly appreciate your help working through this... ty very much.

m-kuhn commented 1 month ago

So, it looks like I played around with it and got it to resolve that dependency. I ended up copying the Qsci folder from the vcpkg folder structure under "C:\Program Files\Python313\Lib\site-packages\PyQt6\bindings" and then it ended up resolving that dependency.

oh, it looks like there are some leftovers from system python (not the one in the SDK), possibly an env variable PYTHONPATH or similar?

In summary: to me this seems to be a local issue, without more information here that could lead to an improvement inside QGIS, I would close this.

danodom-tcb commented 1 month ago

@m-kuhn, thanks again for your assistance on this. So, question regarding the python setup. I had to install python along with sip, pyqt6 and a couple of other things as part of the process to get the project this far. Should I have been able to point everything to the vcpkg sdk and not have to do this?

Also, and I am going to open another issue, but I am putting custom build options for a few projects in the solution and it appears that something is overwriting them and creating compiler issues. A few projects need the /bigobj compiler option to build, but something appears to be randomly removing it. is there a place I can force this to be set to where when it is overwritten, it contains the additional compiler option?

m-kuhn commented 1 month ago

@m-kuhn, thanks again for your assistance on this. So, question regarding the python setup. I had to install python along with sip, pyqt6 and a couple of other things as part of the process to get the project this far. Should I have been able to point everything to the vcpkg sdk and not have to do this?

yes

danodom-tcb commented 1 month ago

@m-kuhn there were no instructions for that in the Install.md file. If this was something that was supposed to happen by passing the SDK path into it, it feels like something didn't get set right. My initial try was without python installed and it was throwing all sorts of missing python(proper) and dependencies.

was this another parameter that needs to be added on the cmake for vcpkg?

m-kuhn commented 1 month ago

No instructions to not do something? 😜

It should have just worked, if it didn't I'd have to see an error message to say more

danodom-tcb commented 1 month ago

Does this mean anything to you?

[cid:fde993fb-b04e-4f96-ae2e-fab567b9a3d7]

It looks like it's trying to load some plugins, but not exactly what this is expecting or how to fix it.

From: Matthias Kuhn @.> Sent: Tuesday, October 22, 2024 5:09 PM To: qgis/QGIS @.> Cc: danodom-tcb @.>; Author @.> Subject: Re: [qgis/QGIS] Windows Build - not building missing Qsci/qscimod6.sip (Issue #59134)

No instructions to not do something? 😜

It should have just worked, if it didn't I'd have to see an error message to say more

— Reply to this email directly, view it on GitHubhttps://github.com/qgis/QGIS/issues/59134#issuecomment-2429820539, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BMHU5D5BQ6CTM3DSMD4XCHTZ42BDRAVCNFSM6AAAAABQGTAORCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRZHAZDANJTHE. You are receiving this because you authored the thread.Message ID: @.***>

github-actions[bot] commented 2 weeks ago

The QGIS project highly values your report and would love to see it addressed. However, this issue has been left in feedback mode for the last 14 days and is being automatically marked as "stale". If you would like to continue with this issue, please provide any missing information or answer any open questions. If you could resolve the issue yourself meanwhile, please leave a note for future readers with the same problem and close the issue. In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this issue. If there is no further activity on this issue, it will be closed in a week.