Potential bugs may cause the recent Windows failures with GMT dev builds

[seisman]

Description of the problem

See the results of the "GMT Latest Tests" action (https://github.com/GenericMappingTools/pygmt/actions?query=workflow%3A%22GMT+Latest+Tests%22).

There are 10 failures on Windows. The error messages are:

_____________ ERROR at setup of test_grdview_grid_dataarray[png] ______________

region = (-116, -109, -47, -44)

    @pytest.fixture(scope="module", name="xrgrid")
    def fixture_xrgrid(region):
        """
        Load the xarray.DataArray grid from the sample earth_relief file.
        """
>       return grdcut(grid="@earth_relief_01d_g", region=region)

C:\Miniconda3\envs\test\lib\site-packages\pygmt\tests\test_grdview.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
C:\Miniconda3\envs\test\lib\site-packages\pygmt\helpers\decorators.py:267: in new_module
    return module_func(*args, **kwargs)
C:\Miniconda3\envs\test\lib\site-packages\pygmt\helpers\decorators.py:411: in new_module
    return module_func(*args, **kwargs)
C:\Miniconda3\envs\test\lib\site-packages\pygmt\gridops.py:116: in grdcut
    return result
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pygmt.helpers.tempfile.GMTTempFile object at 0x0000019EBCD6D190>
args = (None, None, None)

    def __exit__(self, *args):
        if os.path.exists(self.name):
>           os.remove(self.name)
E           PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\pygmt-xqghaxyy.nc'

C:\Miniconda3\envs\test\lib\site-packages\pygmt\helpers\tempfile.py:70: PermissionError

It's unclear if it's a bug in GMT dev version or a PyGMT bug.

[weiji14]

I'll get on Windows and work on this today (and also #717). The CI failures are getting too annoying.

[weiji14]

The "windows-latest - GMT master" workflow is failing at the pygmt install step now, see https://github.com/GenericMappingTools/pygmt/runs/1803109048?check_suite_focus=true#step:10:796. Edit: Was an intermittent issue, should be fixed as cftime==1.4.0 wheels for Windows are now available at https://pypi.org/project/cftime/1.4.0/#files

``` Building wheels for collected packages: pygmt, cftime Building wheel for pygmt (PEP 517): started Building wheel for pygmt (PEP 517): finished with status 'done' Created wheel for pygmt: filename=pygmt-0.2.2.dev90+g8e2b369-py3-none-any.whl size=6820979 sha256=c2d1af0eeaa17c1b6bf2a35d357d35af6377fc8a5cd79d62f5e54021c53257be Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\5f\1d\d8\0ec77787b2f8db5f5eb3c9173c0eeaded3f6a35efdaae2a6d5 Building wheel for cftime (PEP 517): started Building wheel for cftime (PEP 517): finished with status 'error' ERROR: Command errored out with exit status 1: command: 'C:\Miniconda3\envs\test\python.exe' 'C:\Miniconda3\envs\test\lib\site-packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\RUNNER~1\AppData\Local\Temp\tmpy8_5atiw' cwd: C:\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435 Complete output (27 lines): running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.9 creating build\lib.win-amd64-3.9\cftime copying src\cftime\__init__.py -> build\lib.win-amd64-3.9\cftime running build_ext building 'cftime._cftime' extension creating build\temp.win-amd64-3.9 creating build\temp.win-amd64-3.9\Release creating build\temp.win-amd64-3.9\Release\Users creating build\temp.win-amd64-3.9\Release\Users\runneradmin creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp Successfully built pygmt Failed to build cftime creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435 creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src creating build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -IC:\Users\runneradmin\AppData\Local\Temp\pip-build-env-1pu1oz50\overlay\Lib\site-packages\numpy\core\include -IC:\Miniconda3\envs\test\include -IC:\Miniconda3\envs\test\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\ATLMFC\include -IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\include -IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\um -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\winrt -IC:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\cppwinrt /TcC:\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.c /Fobuild\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.obj _cftime.c C:\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.c(16852): warning C4996: '_PyUnicode_get_wstr_length': deprecated in 3.3 C:\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.c(17046): warning C4996: '_PyUnicode_get_wstr_length': deprecated in 3.3 C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\bin\HostX86\x64\link.exe /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:C:\Miniconda3\envs\test\libs /LIBPATH:C:\Miniconda3\envs\test\PCbuild\amd64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\ATLMFC\lib\x64 /LIBPATH:C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29333\lib\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\ucrt\x64 /LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.19041.0\um\x64 /EXPORT:PyInit__cftime build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.obj /OUT:build\lib.win-amd64-3.9\cftime\_cftime.cp39-win_amd64.pyd /IMPLIB:build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.cp39-win_amd64.lib LINK : fatal error LNK1104: cannot open file 'build\temp.win-amd64-3.9\Release\Users\runneradmin\AppData\Local\Temp\pip-install-cx5f1nhp\cftime_6eb00282ca204302be377e9d56ab8435\src\cftime\_cftime.cp39-win_amd64.exp' error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.28.29333\\bin\\HostX86\\x64\\link.exe' failed with exit code 1104 ---------------------------------------- ERROR: Failed building wheel for cftime ERROR: Could not build wheels for cftime which use PEP 517 and cannot be installed directly Error: Process completed with exit code 1. ```

[seisman]

If we can't fix it before v0.3.0. I think the workaround would be codes like

try:
    os.remove(self.name)
except PermissionError:
    print("PyGMT can't delete the temporary file 'xxx' due to unknown causes.")

[seisman]

@weiji14 I think the big issue in GMT dev was just fixed in https://github.com/GenericMappingTools/gmt/pull/4777.

Could you please bump conda-forge's GMT dev version (https://github.com/conda-forge/gmt-feedstock/tree/devel) again, so that we can have full tests running to make sure all tests pass?

[weiji14]

Ok. Hard to do it on mobile at the conference but I'll do it as soon as I possibly can, maybe in a day or two.

[seisman]

As it's hard for you, I'll manually submit a PR later.

[seisman]

Done in https://github.com/conda-forge/gmt-feedstock/pull/134, and recent tests with GMT dev pass (https://github.com/GenericMappingTools/pygmt/actions/runs/553525167).

[weiji14]

Ok, I'll close this issue since it seems to have been fixed in https://github.com/GenericMappingTools/gmt/pull/4777