search

GenericMappingTools / pygmt

A Python interface for the Generic Mapping Tools.
https://www.pygmt.org
BSD 3-Clause "New" or "Revised" License
758 stars 220 forks source link

pylint 2.7.0 breaks the style check #946

Closed seisman closed 3 years ago

seisman commented 3 years ago

Description of the problem

See the "Style Checks" runs in https://github.com/GenericMappingTools/pygmt/pull/944 for details.

weiji14 commented 3 years ago

Lots of duplicate-code recommendations in the error message log:

``` pylint pygmt setup.py doc/conf.py ************* Module conf doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.plot3d:166 ==pygmt.src.plot:203 extra_arrays = [] if "S" in kwargs and kwargs["S"][0] in "vV" and direction is not None: extra_arrays.extend(direction) if "G" in kwargs and not isinstance(kwargs["G"], str): if kind != "vectors": raise GMTInvalidInput( "Can't use arrays for color if data is matrix or file." ) extra_arrays.append(kwargs["G"]) del kwargs["G"] if sizes is not None: if kind != "vectors": raise GMTInvalidInput( "Can't use arrays for sizes if data is matrix or file." ) extra_arrays.append(sizes) if "t" in kwargs and is_nonstr_iter(kwargs["t"]): extra_arrays.append(kwargs["t"]) kwargs["t"] = "" with Session() as lib: # Choose how data will be passed in to the module if kind == "file": file_context = dummy_context(data) elif kind == "matrix": file_context = lib.virtualfile_from_matrix(data) elif kind == "vectors": file_context = lib.virtualfile_from_vectors( (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.tests.test_contour:12 ==pygmt.tests.test_plot:16 TEST_DATA_DIR = os.path.join(os.path.dirname(__file__), "data") POINTS_DATA = os.path.join(TEST_DATA_DIR, "points.txt") @pytest.fixture(scope="module") def data(): """ Load the point data from the test file. """ return np.loadtxt(POINTS_DATA) @pytest.fixture(scope="module") def region(): """ The data region. """ return [10, 70, -5, 10] (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.plot3d:3 ==pygmt.src.plot:3 import numpy as np from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, is_nonstr_iter, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( A="straight_line", B="frame", C="cmap", D="offset", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.contour:4 ==pygmt.src.grdcontour:3 from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( A="annotation", B="frame", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 3 files ==pygmt.src.plot3d:3 ==pygmt.src.plot:3 ==pygmt.src.text:3 import numpy as np from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, is_nonstr_iter, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.grdcut:90 ==pygmt.src.grdfilter:143 kind = data_kind(grid) with GMTTempFile(suffix=".nc") as tmpfile: with Session() as lib: if kind == "file": file_context = dummy_context(grid) elif kind == "grid": file_context = lib.virtualfile_from_grid(grid) else: raise GMTInvalidInput("Unrecognized data type: {}".format(type(grid))) with file_context as infile: if "G" not in kwargs.keys(): # if outgrid is unset, output to tempfile kwargs.update({"G": tmpfile.name}) outgrid = kwargs["G"] arg_str = " ".join([infile, build_arg_string(kwargs)]) (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.grdcut:4 ==pygmt.src.grdfilter:4 import xarray as xr from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( GMTTempFile, build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.basemap:4 ==pygmt.src.coast:4 from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( args_in_kwargs, build_arg_string, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( R="region", J="projection", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 3 files ==pygmt.src.grdcut:5 ==pygmt.src.grdfilter:5 ==pygmt.src.grdinfo:3 from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( GMTTempFile, build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 3 files ==pygmt.src.grdcut:4 ==pygmt.src.grdfilter:4 ==pygmt.src.surface:4 import xarray as xr from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( GMTTempFile, build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.grdview:7 ==pygmt.src.meca:8 from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, use_alias, ) @fmt_docstring @use_alias( R="region", J="projection", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 5 files ==pygmt.src.contour:4 ==pygmt.src.grd2cpt:4 ==pygmt.src.grdcontour:3 ==pygmt.src.grdimage:3 ==pygmt.src.grdview:5 from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, file_context = dummy_context(data) elif kind == "matrix": file_context = lib.virtualfile_from_matrix(data) elif kind == "vectors": (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.tests.test_clib:7 ==pygmt.tests.test_info:5 import numpy as np import numpy.testing as npt import pandas as pd import pytest import xarray as xr (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 7 files ==pygmt.src.contour:7 ==pygmt.src.grd2cpt:7 ==pygmt.src.grdcontour:6 ==pygmt.src.grdimage:6 ==pygmt.src.grdview:8 ==pygmt.src.meca:9 ==pygmt.x2sys:12 build_arg_string, data_kind, dummy_context, fmt_docstring, kwargs_to_strings, (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 4 files ==pygmt.src.meca:8 ==pygmt.src.plot3d:6 ==pygmt.src.plot:6 ==pygmt.src.text:6 from pygmt.helpers import ( build_arg_string, data_kind, dummy_context, fmt_docstring, (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 9 files ==pygmt.src.contour:4 ==pygmt.src.grd2cpt:4 ==pygmt.src.grdcontour:3 ==pygmt.src.grdimage:3 ==pygmt.src.grdview:5 ==pygmt.src.legend:4 ==pygmt.src.plot3d:4 ==pygmt.src.plot:4 ==pygmt.src.text:4 from pygmt.clib import Session from pygmt.exceptions import GMTInvalidInput from pygmt.helpers import ( build_arg_string, data_kind, (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 3 files ==pygmt.src.colorbar:8 ==pygmt.src.meca:18 ==pygmt.src.text:17 @fmt_docstring @use_alias( R="region", J="projection", B="frame", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 2 files ==pygmt.src.contour:27 ==pygmt.src.text:26 V="verbose", W="pen", X="xshift", Y="yshift", c="panel", (duplicate-code) doc/conf.py:1:0: R0801: Similar lines in 3 files ==pygmt.src.basemap:25 ==pygmt.src.coast:30 ==pygmt.src.logo:15 U="timestamp", V="verbose", X="xshift", Y="yshift", c="panel", (duplicate-code) ------------------------------------ Your code has been rated at 10.00/10 make: *** [Makefile:54: lint] Error 8 Error: Process completed with exit code 2. ```

The docs at http://pylint.pycqa.org/en/2.7/technical_reference/features.html#similarities-checker-messages says:

Similar lines in %s files Indicates that a set of similar lines has been detected among multiple file. This usually means that the code should be refactored to avoid this duplication.

So we should tweak our .pylintrc file a bit following https://stackoverflow.com/questions/29206482/pylint-duplicate-code-false-positive (using ignore-imports=yes should fix most warnings).

Also take this as a cue to refactor some of the duplicate code. I was going to create a virtualfile_from_data() function to replace this common block of code in our functions:

    with Session() as lib:
        # Choose how data will be passed in to the module
        if kind == "file":
            file_context = dummy_context(data)
        elif kind == "matrix":
            file_context = lib.virtualfile_from_matrix(data)
        elif kind == "vectors":
            file_context = lib.virtualfile_from_vectors()
seisman commented 3 years ago

Still a lot of warnings after setting ignore-imports=yes. Perhaps better to disable duplicate-code and try to refactor later.

weiji14 commented 3 years ago

Yes, I'll open a separate issue for refactoring. Do you want to go ahead and update .pylintrc?