search

holoviz / holoviews

With Holoviews, your data visualizes itself.
https://holoviews.org
BSD 3-Clause "New" or "Revised" License
2.68k stars 402 forks source link

Optimize import startup time #1892

Closed philippjfr closed 5 years ago

philippjfr commented 6 years ago

We have on occasion discussed how long it takes to import holoviews. After coming across the import_profiler library I just ran it so now we have something concrete to discuss, here's the output:

from import_profiler import profile_import

with profile_import() as context:
    import holoviews
context.print_info()
cumtime (ms)    intime (ms)  name
        3858.4            2.4  holoviews
         665.3            2.4  +numpy
         626.8            4.1  ++
         620.7            1.6  +++numpy.lib
         596.1            0.8  ++++type_check
         594.6            1.3  +++++numpy.core.numeric
         543            543    ++++++
           5.6            5.6  ++++++
           8              1.8  ++++++
           2.2            0.5  +++++++numpy.compat
           1              0.6  ++++++++
           4              4    +++++++numerictypes
           4.9            1.6  ++++++
           3.2            1.1  +++++++arrayprint
           2              1.3  ++++++++fromnumeric
           1.1            1.1  ++++++
           1.5            1.4  ++++++
           1.7            1.6  ++++++memmap
           5.8            5.7  ++++++
          18              0.9  ++++++numpy.testing.nosetester
          13              1    +++++++unittest
           2.1            0.8  ++++++++result
           1.1            1    +++++++++
           3.8            1.5  ++++++++case
           2              2    +++++++++difflib
           2.3            2.2  ++++++++suite
           1.8            1.6  ++++++++loader
           2              0.7  ++++++++main
           1.2            0.7  +++++++++
           3.3            0.7  +++++++
           2.6            1.2  ++++++++utils
           1.2            1.2  +++++++++numpy.lib.utils
           5.5            0.9  ++++index_tricks
           2.2            1.3  +++++
           1.7            0.8  +++++numpy.matrixlib
           6.4            1.1  ++++polynomial
           5.3            0.9  +++++numpy.linalg
           3.9            1    ++++++linalg
           2.8            2.8  +++++++numpy.linalg
           3.8            1    ++++npyio
           4.5            0.8  ++
           3.1            0.9  +++fftpack
           1.3            1.3  ++++
          10.6            0.8  ++
           3.8            0.8  +++polynomial
           1.9            1.9  ++++
           1.1            1    ++++_polybase
           1              1    +++chebyshev
           1              1    +++legendre
           1.9            1.7  +++hermite
           1              1    +++hermite_e
          11.2            0.9  ++
           9.9            9.6  +++mtrand
           1.1            1    ++
           5.6            0.7  ++
           3.6            3.5  +++
           1.3            1.2  +++
           9.1            4.5  +param
           3.4            2.3  ++parameterized
        3001.4            1.9  +core
        2989.5            6.6  ++data
        2090              3.5  +++dimension
        2079              4    ++++core.util
           7.5            1    +++++cyordereddict
           4.3            2.5  ++++++_cyordereddict
           1.4            1.3  +++++++backports_abc
           1.9            1.4  ++++++benchmark.benchmark
        1347.9            2.3  +++++pandas
           6.9            2.8  ++++++pytz
           1.2            1.1  +++++++pytz.lazy
           1.2            1.1  +++++++pytz.tzinfo
          16.3            2.4  ++++++pandas.compat.numpy
           1.7            1.6  +++++++distutils.version
           2              1.1  +++++++httplib
           1.2            0.5  +++++++pandas.compat.chainmap
           8.3            2.7  +++++++dateutil
           1.6            0.9  ++++++++
           3.6            0.6  ++++++++
           3.1            1.1  +++++++++tz
          27.7            3.5  ++++++pandas._libs
          14.9           14.1  +++++++tslib
           9.2            4.8  +++++++pandas._libs.lib
           4.1            3.9  ++++++++interval
          12              3.2  ++++++pandas.core.config_init
           6.3            2    +++++++pandas.core.config
           4.2            2.3  ++++++++pandas.io.formats.printing
           1.8            1.6  +++++++++pandas.core.dtypes.inference
           1.2            0.4  +++++++pandas.io.formats.console
        1125.2            0.6  ++++++pandas.core.api
          19.3            1.2  +++++++pandas.core.algorithms
          12.8            1.1  ++++++++pandas.core.dtypes.cast
          10.6            1.2  +++++++++common
           6.6            6.5  ++++++++++pandas._libs
           1.5            1.5  ++++++++++dtypes
           1.2            1.2  ++++++++++generic
           5.2            1.1  ++++++++pandas.core
           2.8            1.3  +++++++++pandas.api
           1.2            1.1  +++++++++pandas.errors
          28.6            1.6  +++++++pandas.core.categorical
          25.8            1.6  ++++++++pandas.core.base
           5.4            1.4  +++++++++pandas.util._validators
           3.1            1.4  ++++++++++pandas.core.util.hashing
           1.6            1.5  +++++++++++pandas._libs
          17.4            2.1  +++++++++pandas.core.nanops
          15.1            1.3  ++++++++++bottleneck
           1.8            1.8  +++++++++++reduce
           1.3            1.2  +++++++++++nonreduce
           1.4            1.4  +++++++++++nonreduce_axis
           1.3            1.3  +++++++++++move
           3.7            1    +++++++++++
           1.7            1.1  +++++++++++bottleneck.benchmark.bench
           1.4            1.3  +++++++++++bottleneck.tests.util
           1.3            1.2  +++++++++pandas.compat.numpy
        1057.6           10.5  +++++++pandas.core.groupby
         695              0.6  ++++++++pandas.core.index
         694.5            1.3  +++++++++pandas.core.indexes.api
         662.3            3.6  ++++++++++pandas.core.indexes.base
           7.4            7    +++++++++++pandas._libs
         647.6            1.2  +++++++++++pandas.core.ops
         646.2            1.6  ++++++++++++pandas.core.computation.expressions
         644.4            2.1  +++++++++++++numexpr
         598.7          598.4  ++++++++++++++cpuinfo
           4.3            1.6  ++++++++++++++numexpr.expressions
           2.4            2.4  +++++++++++++++numexpr
           2.3            1.4  ++++++++++++++numexpr.necompiler
          35.8            1.1  ++++++++++++++numexpr.tests
          34.7           34.6  +++++++++++++++numexpr.tests.test_numexpr
           2.5            2.4  +++++++++++pandas.core.strings
           1.9            1.7  ++++++++++pandas.core.indexes.category
           2.8            2.5  ++++++++++pandas.core.indexes.multi
           2              1.9  ++++++++++pandas.core.indexes.interval
           2              1.9  ++++++++++pandas.core.indexes.numeric
           1.5            1.4  ++++++++++pandas.core.indexes.range
          17.2            1.6  ++++++++++pandas.core.indexes.timedeltas
           8.4            3.1  +++++++++++pandas.tseries.frequencies
           4.9            2.8  ++++++++++++pandas.tseries.offsets
           1.6            1.4  +++++++++++++pandas.core.tools.datetimes
           5.1            1.3  +++++++++++pandas.core.indexes.datetimelike
           3.7            3.5  ++++++++++++pandas._libs.period
           1.1            1.1  +++++++++++encodings.unicode_escape
           3.5            1.6  ++++++++++pandas.core.indexes.period
           1.8            1.6  +++++++++++pandas.core.indexes.datetimes
         343.5           10    ++++++++pandas.core.frame
         225.8            4.8  +++++++++pandas.core.generic
           2.4            2.3  ++++++++++pandas.core.indexing
           7.6            2.3  ++++++++++pandas.core.internals
           5.2            2.1  +++++++++++pandas.core.sparse.array
           2.8            2.7  ++++++++++++pandas._libs.sparse
         210.7            2.1  ++++++++++pandas.io.formats.format
         207.7            1.1  +++++++++++pandas.io.common
           1.1            0.4  ++++++++++++csv
           1.6            1.6  ++++++++++++mmap
         196.5            1.4  ++++++++++++s3fs
         194.3            1.5  +++++++++++++core
         175.7            1.3  ++++++++++++++boto3
         174.3            1.4  +++++++++++++++boto3.session
         154.1            3.8  ++++++++++++++++botocore.session
          64.5            0.7  +++++++++++++++++botocore.configloader
           1.6            0.1  ++++++++++++++++++six.moves
           1.5            1.5  +++++++++++++++++++ConfigParser
          62.2            1.8  ++++++++++++++++++botocore.exceptions
          60.2            1.6  +++++++++++++++++++botocore.vendored.requests.exceptions
          24.5            2    ++++++++++++++++++++packages.urllib3.contrib
          21.9            1.1  +++++++++++++++++++++
          19.8            1.1  ++++++++++++++++++++++connectionpool
           1.1            0.7  +++++++++++++++++++++++queue
           1.2            1.2  +++++++++++++++++++++++exceptions
           1.2            0.4  +++++++++++++++++++++++packages.ssl_match_hostname
           1.5            1.3  +++++++++++++++++++++++packages
           8.2            1    +++++++++++++++++++++++connection
           6.6            0.8  ++++++++++++++++++++++++util.ssl_
           1.3            0.9  +++++++++++++++++++++++++ssl_
           1.3            1.2  +++++++++++++++++++++++++url
           3.1            0.7  +++++++++++++++++++++++request
           2.3            0.5  ++++++++++++++++++++++++filepost
           2.1            0.9  +++++++++++++++++++++++response
           1.1            1    ++++++++++++++++++++++++_collections
          26.9            1.2  ++++++++++++++++++++
           1              0.5  +++++++++++++++++++++
          21              0.6  +++++++++++++++++++++compat
           1              1    ++++++++++++++++++++++packages
           9.2            1.9  ++++++++++++++++++++++simplejson
           2.8            1.2  +++++++++++++++++++++++scanner
           1.5            1.5  ++++++++++++++++++++++++simplejson._speedups
           2.7            1.3  +++++++++++++++++++++++decoder
           1.8            1.8  +++++++++++++++++++++++encoder
           1.4            1.3  ++++++++++++++++++++++urllib2
           5.6            5.1  ++++++++++++++++++++++cookielib
           1.9            1.9  ++++++++++++++++++++++Cookie
           1.1            1    ++++++++++++++++++++++packages.urllib3.packages.ordered_dict
           1.1            0.9  +++++++++++++++++++++cookies
           1              1    +++++++++++++++++++++exceptions
           4.1            1.2  ++++++++++++++++++++models
           2.3            1.1  +++++++++++++++++++++auth
           1.1            1.1  ++++++++++++++++++++++status_codes
           3.1            0.8  ++++++++++++++++++++api
           2.3            1    +++++++++++++++++++++
           1.2            1    ++++++++++++++++++++++adapters
          10.8            1.8  +++++++++++++++++botocore.credentials
           7              1    ++++++++++++++++++botocore.compat
           1.9            1.9  +++++++++++++++++++botocore.vendored
           3.6            0.3  +++++++++++++++++++xml.etree.cElementTree
           3.3            1.6  ++++++++++++++++++++_elementtree
           1.6            0.5  +++++++++++++++++++++xml.etree
           1.7            1.4  ++++++++++++++++++botocore.utils
          69.5            1.3  +++++++++++++++++botocore.client
          25.5            1    ++++++++++++++++++botocore
           7.4            1.5  +++++++++++++++++++jmespath
           5.9            0.9  ++++++++++++++++++++jmespath
           2.6            0.9  +++++++++++++++++++++jmespath
           1.5            0.9  ++++++++++++++++++++++jmespath.exceptions
           1.7            0.7  +++++++++++++++++++++jmespath
          17              1.5  +++++++++++++++++++botocore.docs.docstring
          15.5            0.8  ++++++++++++++++++++botocore.docs.service
           1.3            1.3  +++++++++++++++++++++botocore.docs.utils
           4.6            0.7  +++++++++++++++++++++botocore.docs.client
           2.8            0.8  ++++++++++++++++++++++botocore.docs.method
           1.2            0.7  +++++++++++++++++++++++botocore.docs.params
           7.4            1.4  +++++++++++++++++++++botocore.docs.bcdoc.restdoc
           4.8            1    ++++++++++++++++++++++botocore.docs.bcdoc.docstringparser
           3.8            2.9  +++++++++++++++++++++++HTMLParser
           1.1            1.1  ++++++++++++++++++++++botocore.docs.bcdoc.style
           1.7            1.1  ++++++++++++++++++botocore.auth
           1.3            1.1  ++++++++++++++++++botocore.awsrequest
           1.5            1.5  ++++++++++++++++++botocore.hooks
           2.3            2.3  ++++++++++++++++++botocore.model
           1.2            1.1  ++++++++++++++++++botocore.paginate
          34.5            0.7  ++++++++++++++++++botocore.args
          28.3           27    +++++++++++++++++++botocore.serialize
           1.2            1.1  ++++++++++++++++++++botocore
           1.1            1    +++++++++++++++++++botocore.signers
           4.4            0.8  +++++++++++++++++++botocore.config
           3.6            1.2  ++++++++++++++++++++botocore.endpoint
           1.8            0.5  +++++++++++++++++++++botocore.response
           1.3            1.2  ++++++++++++++++++++++botocore
           3.2            1.5  +++++++++++++++++botocore
           1              0.9  ++++++++++++++++++botocore
           1.1            1    +++++++++++++++++botocore.loaders
           1.5            1.2  ++++++++++++++++boto3.utils
          16.1            2.3  ++++++++++++++++resources.factory
          11.9            0.8  +++++++++++++++++action
           1              1    ++++++++++++++++++model
           8.7            1.7  ++++++++++++++++++boto3.docs.docstring
           6.8            0.8  +++++++++++++++++++boto3.docs.service
           5.5            0.7  ++++++++++++++++++++boto3.docs.resource
           1.7            0.5  +++++++++++++++++++++boto3.docs.action
           1.2            0.9  +++++++++++++++++collection
          15              2    ++++++++++++++boto3.s3.transfer
          12.5            3.2  +++++++++++++++concurrent
           1.9            1.7  ++++++++++++++++concurrent.futures._base
           6.4            0.8  ++++++++++++++++concurrent.futures.process
           5.6            1.7  +++++++++++++++++multiprocessing
           1              0.9  ++++++++++++++++++multiprocessing.process
           1.8            1.7  ++++++++++++++++++_multiprocessing
           1.2            0.9  ++++++++++++++utils
           3.1            1.9  ++++++++++++py.path
           1.3            1.1  +++++++++++++py
           3.6            2.1  ++++++++++++py._path.local
           1.2            1.1  +++++++++++++py._path
          97.8            5.3  +++++++++pandas.core.series
           1.4            1.3  ++++++++++pandas.core.indexes.accessors
          10.5            3.9  ++++++++++pandas.core
           6.4            6.3  +++++++++++pandas._libs.window
          80.5            1.2  ++++++++++pandas.plotting._core
          72.7            1.3  +++++++++++pandas.plotting
          46.7           14.1  ++++++++++++matplotlib.units
           1.3            0.9  +++++++++++++distutils.sysconfig
           3.3            2.7  +++++++++++++matplotlib.cbook
           4.9            1.6  +++++++++++++matplotlib.compat
           3.3            1.7  ++++++++++++++subprocess32
           1.5            1.5  +++++++++++++++_posixsubprocess
          21.7            2.2  +++++++++++++matplotlib.rcsetup
          13.2            2.8  ++++++++++++++matplotlib.fontconfig_pattern
          10.4            9.9  +++++++++++++++pyparsing
           5.1            2.2  ++++++++++++++matplotlib.colors
           2.8            2.7  +++++++++++++++_color_data
           1.2            1.1  ++++++++++++++cycler
          23.6            2    ++++++++++++matplotlib.dates
           5              1.5  +++++++++++++dateutil.rrule
           3.3            3.3  ++++++++++++++fractions
          16.4            2.2  +++++++++++++matplotlib.ticker
          14.1           10.2  ++++++++++++++matplotlib
           2              2    +++++++++++++++matplotlib._path
           1.7            1.6  +++++++++++++++path
           3.1            0.9  +++++++++++pandas.plotting._misc
           1.2            1    ++++++++++++pandas.plotting._style
           3.5            3.3  +++++++++++pandas.plotting._core
           9.5            0.7  +++++++++pandas.core.computation.eval
           6.8            2.1  ++++++++++pandas.core.computation.expr
           1.2            1.1  +++++++++++pandas.core.reshape.util
           3.3            1.3  +++++++++++pandas.core.computation.ops
           1.2            1    ++++++++++++pandas.core.computation.scope
           2              1.1  ++++++++++pandas.core.computation.engines
           3.8            3.7  ++++++++pandas.core.panel
           1              1    ++++++++pandas.core.sorting
           3.5            3.4  ++++++++pandas._libs
           3              2.1  +++++++pandas.core.panel4d
          11.8            1.4  +++++++pandas.core.reshape.reshape
           8.1            0.5  ++++++++pandas.core.sparse.api
           1.2            1.1  +++++++++pandas.core.sparse.list
           3.3            2.5  +++++++++pandas.core.sparse.series
           3.1            3    +++++++++pandas.core.sparse.frame
           2.1            2    ++++++++pandas._libs
           2.6            2.5  +++++++pandas.core.resample
           2.8            1.6  ++++++pandas.stats.api
           1.2            1.1  +++++++pandas.stats.moments
           4.7            0.5  ++++++pandas.core.reshape.api
           1.6            1.4  +++++++pandas.core.reshape.merge
           1.2            1.1  ++++++pandas.tools.plotting
          73              0.5  ++++++pandas.io.api
           9.5            3.8  +++++++pandas.io.parsers
           4.7            4.5  ++++++++pandas._libs.parsers
           4.5            1.8  +++++++pandas.io.excel
           1.8            1.7  ++++++++pandas._libs.json
           5.2            3.5  +++++++pandas.io.pytables
           1.3            1.2  ++++++++pandas.core.computation.pytables
           3.3            0.6  +++++++pandas.io.json
           2.7            1.1  ++++++++json
           1.2            1.1  +++++++pandas.io.html
           1.2            1.1  +++++++pandas.io.sql
           1              0.5  +++++++pandas.io.sas
           2.7            2.5  +++++++pandas.io.stata
           1.6            0.8  +++++++pandas.io.pickle
          40.4            1.1  +++++++pandas.io.packers
           7.9            1.3  ++++++++pandas.io.msgpack
           3              3    +++++++++pandas.io.msgpack._packer
           1.9            1.9  +++++++++pandas.io.msgpack._unpacker
          30.3            1.3  ++++++++blosc
           6.7            6.7  +++++++++blosc.blosc_extension
          20.7            0.8  +++++++++blosc.test
          19.7            1.8  ++++++++++psutil
           8.7            8.5  +++++++++++
           1.3            1.2  +++++++++++_compat
           7.4            3.9  +++++++++++
           1.6            1.6  ++++++++++++
           1.1            1.1  ++++++++++++
          67.2            0.5  ++++++pandas.util._tester
          66.6            6.7  +++++++pytest
          18.3            4.2  ++++++++_pytest.config
           5.7            0.8  +++++++++_pytest._code
           3.5            1.6  ++++++++++code
           1.3            1.1  ++++++++++source
           3.2            0.9  +++++++++_pytest.hookspec
           2.4            0.5  ++++++++++_pytest._pluggy
           1.9            1.8  +++++++++++_pytest.vendored_packages.pluggy
           3.8            1    +++++++++_pytest.assertion
           1.8            1.5  ++++++++++_pytest.assertion
           1.1            0.9  +++++++++_pytest.compat
           2.6            1.3  ++++++++_pytest.main
           1.2            1.1  +++++++++_pytest.runner
           6.1            1.4  ++++++++_pytest.python
           4.5            1.9  +++++++++_pytest
           2.6            1.9  ++++++++++py._code.code
           1.9            0.9  ++++++++_pytest.unittest
           1              1    +++++++++_pytest.skipping
           3.1            1.2  ++++++++_pytest.capture
           1.8            1.6  +++++++++py._io.capture
           2              0.8  ++++++++_pytest.tmpdir
           1.2            1.1  +++++++++_pytest.monkeypatch
          16.1           14    ++++++++_pytest.junitxml
           1.9            1.8  +++++++++py._xmlgen
           4.5            0.4  ++++++pandas.testing
           4              1.9  +++++++pandas.util.testing
           1.6            1.5  ++++++++pandas._libs
           1.1            0.5  ++++++_version
         718.8            2.3  +++++dask.dataframe
           2.1            1    ++++++core
           1.1            1    +++++++utils_test
           6.1            1.1  ++++++local
           2.1            1    +++++++compatibility
           1.2            1.1  +++++++optimize
          32.1            5.1  ++++++delayed
          12.9            4.2  +++++++toolz
           2.4            1    ++++++++itertoolz
           2.7            1.2  ++++++++functoolz
           1.3            1.3  +++++++++
           2              0.8  ++++++++
           9.7            1.9  +++++++
           2              0.5  ++++++++hashing
           1.4            1.3  ++++++++utils
           3.6            1.4  ++++++++multiprocessing.pool
           2              1    +++++++++dummy
           3.1            1.3  +++++++cloudpickle
           1.7            1.2  ++++++++cloudpickle.cloudpickle
          83.4           10.6  ++++++core
           4.1            1.2  +++++++chest
           2.4            0.9  ++++++++core
           1.1            1.1  +++++++++heapdict
          64.5            1.4  +++++++
          10.5            2.2  ++++++++core
           3.9            1.4  +++++++++toolz.curried
           1.8            1.8  ++++++++++
           2.2            1    +++++++++
           1.2            1.1  ++++++++++
           5.1            5    ++++++++ufunc
           2.2            1.1  ++++++++reductions
           1.1            0.8  +++++++++wrap
          42.5           12    ++++++++
          12.6            2.4  +++++++++scipy
           4.1            2.3  ++++++++++scipy._lib._version
           1.7            1.5  +++++++++++scipy._lib.six
           4.4            1.3  ++++++++++scipy._lib._ccallback
           3.1            3    +++++++++++
          16.9            1.7  +++++++++scipy.fftpack
           9.7            1.3  ++++++++++basic
           8.4            8.4  +++++++++++
           2.4            1.1  ++++++++++pseudo_diffs
           1.3            1.2  +++++++++++
           1              1    ++++++++rechunk
           1.9            0.9  +++++++
         586.8            1.2  ++++++io
           3.1            1.2  +++++++io
           1.7            1.2  ++++++++shuffle
          12              1.7  +++++++csv
          10.1            1    ++++++++bytes
           7.9            0.9  +++++++++core
           6.1            0.6  ++++++++++compression
           4.1            1.3  +++++++++++snappy
           2.8            2.8  ++++++++++++_snappy
           1.4            1.4  +++++++++++lz4
           5.9            0.9  +++++++hdf
           4.9            0.8  ++++++++
           4              2.4  +++++++++tblib.pickling_support
         562.7            1    +++++++parquet
         560.8            1.8  ++++++++fastparquet
          13              1.5  +++++++++thriftpy
          11.4            0.8  ++++++++++hook
          10.6            0.8  +++++++++++parser
           9.7            1.2  ++++++++++++parser
           5              4.6  +++++++++++++ply
           1.3            0.6  +++++++++++++lexer
           1.6            1.5  +++++++++++++thrift
          48.1           47.5  +++++++++thrift_structures
         493.4            0.9  +++++++++core
          20.1            0.8  ++++++++++thriftpy.protocol.compact
           1.5            0.9  +++++++++++binary
           1              1    +++++++++++compact
          15.1            2    +++++++++++cybin
          13              0.9  ++++++++++++thriftpy.transport.cybase
           1.9            0.9  +++++++++++++sslsocket
           4.6            0.8  +++++++++++++buffered
           3.7            1.9  ++++++++++++++cybuffered
           1.8            1.8  +++++++++++++++thriftpy.transport.cybase
           2.7            0.9  +++++++++++++framed
           1.7            1.7  ++++++++++++++cyframed
           2.1            0.8  +++++++++++++memory
           1.3            1.3  ++++++++++++++cymemory
         467             79.8  ++++++++++
         382.9            4.9  +++++++++++numba
         197.5           98.4  ++++++++++++
          72.5            2.8  +++++++++++++llvmlite.binding
          49.3            0.7  ++++++++++++++dylib
          48.6           45.5  +++++++++++++++
           1.9            1.8  ++++++++++++++++six
           2.8            1.2  ++++++++++++++executionengine
           1.6            1.6  +++++++++++++++
           3.1            0.9  ++++++++++++++module
           2.2            2.1  +++++++++++++++value
          10.3            0.6  ++++++++++++++analysis
           9.7            1    +++++++++++++++llvmlite
           2.6            2    ++++++++++++++++types
           1.2            1.2  ++++++++++++++++values
           1.3            0.8  ++++++++++++++++module
           2.9            1.2  ++++++++++++++++builder
           1.7            1.6  +++++++++++++++++
           4.1            1.7  +++++++++++++errors
           2.4            2.3  ++++++++++++++numba
           8              1.7  +++++++++++++abstract
           6.2            1.3  ++++++++++++++utils
           2              1.4  +++++++++++++++funcsigs
           2.7            1.3  +++++++++++++++singledispatch
           1.3            1.3  ++++++++++++++++singledispatch_helpers
           7.6            2.1  +++++++++++++containers
           1.3            1.2  ++++++++++++++common
           4.2            1.6  ++++++++++++++misc
           2.6            1.3  +++++++++++++++typeconv
           1.4            1.3  ++++++++++++++++castgraph
           1.2            1.2  +++++++++++++functions
           1.1            1    +++++++++++++iterators
           1.5            1.5  +++++++++++++npytypes
           3.1            1.9  +++++++++++++scalars
           1.1            1.1  ++++++++++++++
          29.1            1.1  ++++++++++++special
          28              1.9  +++++++++++++typing.typeof
          26.1            1.1  ++++++++++++++context
          17.6           15.3  +++++++++++++++numba.typeconv
           2.2            0.9  ++++++++++++++++typeconv
           1.3            1.3  +++++++++++++++++
           1.7            1.6  +++++++++++++++
           5.7            1.5  +++++++++++++++typeof
           4.2            3.3  ++++++++++++++++numba
         136.9            1    ++++++++++++decorators
         135              2.7  +++++++++++++targets
         114.9            0.8  ++++++++++++++
           1.6            1.5  +++++++++++++++llvmlite.llvmpy.core
           1.1            1.1  +++++++++++++++numba
           2.5            0.7  +++++++++++++++numba.callwrapper
           1.8            1.7  ++++++++++++++++numba
          92.4            1.4  +++++++++++++++base
           3.3            2    ++++++++++++++++numba
           1.2            1.2  +++++++++++++++++
           2.3            2.3  ++++++++++++++++numba
          67.6            3.3  ++++++++++++++++numba.pythonapi
          63.5            1.5  +++++++++++++++++numba
           3.4            3.3  ++++++++++++++++++
          57.6            1.1  ++++++++++++++++++
           9.3            1.9  +++++++++++++++++++numba
           3.3            1.5  ++++++++++++++++++++numba
           1.7            1.6  +++++++++++++++++++++numba.controlflow
           3.3            3    ++++++++++++++++++++
          47.2            1.9  +++++++++++++++++++numba
           6              5.9  ++++++++++++++++++++numba.typing
          39.2           35.9  ++++++++++++++++++++
           3.3            1.8  +++++++++++++++++++++numba.interpreter
           1.4            1.3  ++++++++++++++++++++++
          16.5            4.1  ++++++++++++++++
           9.6            0.8  +++++++++++++++++numba
           1.5            1.4  ++++++++++++++++++targets.imputils
           7.1            1.3  ++++++++++++++++++datamodel
           1.1            1.1  +++++++++++++++++++manager
           1              1    +++++++++++++++++++packer
           3.1            3    +++++++++++++++++++models
           2.7            2.6  +++++++++++++++++
          14.9            7    +++++++++++++++numba.targets
           6.1            2.5  ++++++++++++++++numba.runtime.nrtopt
           3.4            1.4  +++++++++++++++++nrt
           1.3            1.3  ++++++++++++++++++
          16.8            1.7  ++++++++++++++
          11.5            4.1  +++++++++++++++numba
           4.9            4.7  ++++++++++++++++numba
           2.4            1.8  ++++++++++++++++numba.annotations
           2.7            1.8  +++++++++++++++caching
           9.4            1.1  ++++++++++++npyufunc
           6.5            0.7  +++++++++++++decorators
           4.9            2.1  ++++++++++++++
           2.8            0.9  +++++++++++++++
           1.2            1.2  ++++++++++++++++wrappers
           1.7            1.6  +++++++++++++
           4.5            0.8  ++++++++++++jitclass
           2.8            0.5  +++++++++++++decorators
           2.3            1.3  ++++++++++++++base
           1.6            1.5  +++++++++++speedups
           1.1            0.8  +++++++++++util
           1.5            1.5  +++++++++++numba.targets
           2.1            0.8  ++++++++++compression
           2              1.3  ++++++++++converted_types
           4.5            2.5  +++++++++writer
           1.7            1    ++++++++++
           1              1    ++++++multi
           1.4            1.3  ++++++rolling
           7.2            4.9  ++++options
           2.1            0.8  +++++tree
           1.3            1.2  ++++++pprint
          41.9            3.8  +++interface
          37.9            7.2  ++++element
           8.5            3.8  +++++layout
           3.8            3.7  ++++++ndmapping
           3.4            3.3  +++++overlay
          18.7            6    +++++spaces
          12.6           12.1  ++++++streams
           4              3.9  +++array
           4.5            4.3  +++dictionary
           1.4            1.3  +++grid
           2.9            2.8  +++multipath
           2.1            1.1  +++image
           3.6            3.5  +++pandas
         729.1            1.9  +++iris
           2.2            2.2  ++++iris.config
         601.6            3.3  ++++iris.cube
           5.8            2.6  +++++xml.dom.minidom
           1.3            0.6  ++++++domreg
           1.8            1.1  ++++++xml.dom.xmlbuilder
           7.1            1.1  +++++biggus
           6              5.5  ++++++_init
           2.7            1.1  +++++iris._deprecation
           1.5            1.5  ++++++iris.exceptions
         569.7            2.2  +++++iris.analysis
         271.5            2    ++++++scipy.interpolate
         266              2    +++++++interpolate
          47.3            3.4  ++++++++scipy.linalg
          15.3            0.9  +++++++++misc
           8.1            1    ++++++++++blas
           7              7    +++++++++++scipy.linalg
           6.3            1.2  ++++++++++lapack
           3.8            3.7  +++++++++++scipy.linalg
           1.3            1.2  +++++++++++scipy._lib._util
           8.5            1.2  +++++++++basic
           3.1            1    ++++++++++flinalg
           2.1            2.1  +++++++++++
           1.4            1.3  ++++++++++decomp
           2.2            2.1  ++++++++++_solve_toeplitz
           3.2            0.8  +++++++++matfuncs
          10.3            2.2  +++++++++_decomp_update
           2.8            2.8  ++++++++++scipy.linalg.cython_blas
           5.2            5.2  ++++++++++scipy.linalg.cython_lapack
         137.7            1.7  ++++++++scipy.special
           5.4            3.7  +++++++++_ufuncs
           1.6            1.5  ++++++++++scipy.special._ufuncs_cxx
           5.6            1.2  +++++++++basic
           2.2            2.1  ++++++++++
           1.1            1.1  ++++++++++
         122.3            0.6  +++++++++_ellip_harm
         121.7            1.2  ++++++++++_ellip_harm_2
         120.3            1.3  +++++++++++scipy.integrate
           2              0.7  ++++++++++++odepack
           1.3            1.3  +++++++++++++
           2.6            1.2  ++++++++++++quadpack
           1.3            1.3  +++++++++++++
           5.8            1.2  ++++++++++++_ode
           1.5            1.5  +++++++++++++
           1.6            1.6  +++++++++++++
           1.4            1.4  +++++++++++++
         107.8            0.9  ++++++++++++_bvp
          30.6            1.2  +++++++++++++scipy.sparse
           2.1            1.3  ++++++++++++++base
           5.4            0.8  ++++++++++++++csr
           1.9            1.9  +++++++++++++++_sparsetools
           2.6            1    +++++++++++++++compressed
           4.7            0.7  ++++++++++++++lil
           3.9            3.8  +++++++++++++++
          12              1.3  ++++++++++++++csgraph
           3.8            1.6  +++++++++++++++_shortest_path
           2              0.5  ++++++++++++++++scipy.sparse.csgraph._validation
           1.6            1.5  +++++++++++++++++_tools
           1.6            1.5  +++++++++++++++_traversal
           1.9            1.8  +++++++++++++++_min_spanning_tree
           2.1            2.1  +++++++++++++++_reordering
          31              1.4  +++++++++++++scipy.sparse.linalg
          15.2            1.2  ++++++++++++++isolve
          10.8            2.4  +++++++++++++++iterative
           2.4            2.4  ++++++++++++++++
           1.1            1    ++++++++++++++++scipy.sparse.linalg.interface
           2.2            2    ++++++++++++++++scipy._lib.decorator
           1.7            1.7  ++++++++++++++++utils
           1              1    ++++++++++++++++scipy._lib._threadsafety
           1.1            1.1  +++++++++++++++minres
           4.7            0.9  ++++++++++++++dsolve
           3              0.9  +++++++++++++++linsolve
           1.4            1.4  ++++++++++++++++
           6.6            1    ++++++++++++++eigen
           4              0.5  +++++++++++++++arpack
           3.5            1.2  ++++++++++++++++arpack
           2.3            2.3  +++++++++++++++++
           1.6            0.7  +++++++++++++++lobpcg
           1              1    ++++++++++++++matfuncs
          45.3            1.6  +++++++++++++scipy.optimize
           3.9            1.4  ++++++++++++++optimize
           2.5            0.8  +++++++++++++++linesearch
           1.6            1.6  ++++++++++++++++scipy.optimize
          12.4            0.7  ++++++++++++++_minimize
           1.5            0.7  +++++++++++++++_trustregion_dogleg
           2.5            0.8  +++++++++++++++lbfgsb
           1.7            1.7  ++++++++++++++++
           2.7            0.8  +++++++++++++++tnc
           1.8            1.8  ++++++++++++++++scipy.optimize
           2.2            0.6  +++++++++++++++cobyla
           1.6            1.5  ++++++++++++++++scipy.optimize
           2.2            0.8  +++++++++++++++slsqp
           1.3            1.3  ++++++++++++++++scipy.optimize._slsqp
          19.3            0.9  ++++++++++++++_root
          14.7            0.7  +++++++++++++++minpack
           1.4            1.4  ++++++++++++++++
          12.6            1.3  ++++++++++++++++_lsq
           6.4            0.8  +++++++++++++++++least_squares
           2.9            0.8  ++++++++++++++++++scipy.optimize._numdiff
           2              2    +++++++++++++++++++_group_columns
           1.8            0.7  ++++++++++++++++++trf
           1              0.9  +++++++++++++++++++common
           4.8            0.9  +++++++++++++++++lsq_linear
           3.1            0.8  ++++++++++++++++++trf_linear
           2.2            2.1  +++++++++++++++++++givens_elimination
           2.8            2.7  +++++++++++++++
           2.2            1    ++++++++++++++zeros
           1.2            1.2  +++++++++++++++
           2.1            0.7  ++++++++++++++nnls
           1.4            1.4  +++++++++++++++
           1              1    ++++++++++++++_differentialevolution
          10.3            0.8  ++++++++
           5.7            1.2  +++++++++_fitpack_impl
           1.7            1.7  ++++++++++
           2.8            2.8  ++++++++++
           3.9            0.9  +++++++++_bsplines
           2.9            2.8  ++++++++++
           1              1    ++++++++polyint
           3              3    ++++++++
           1.3            1.3  ++++++++fitpack2
          63.1            2.9  ++++++++interpnd
          60.1            1.2  +++++++++scipy.spatial.qhull
           1              1    ++++++++++kdtree
          42.9           42.6  ++++++++++ckdtree
           4.9            4.7  ++++++++++qhull
           3.9            0.8  ++++++++++_spherical_voronoi
           1.8            1.7  +++++++++++
           1.1            1.1  ++++++++++_plotutils
           4.5            1.2  ++++++++++
           1.6            1.6  +++++++++++
           1.7            1.7  +++++++++++
           1.5            1.4  +++++++_cubic
         162.3            1.4  ++++++scipy.stats.mstats
         140.8           12.2  +++++++stats
         118.9            0.7  ++++++++
          24.9            2.5  +++++++++_distn_infrastructure
          21.1            1.2  ++++++++++scipy.misc
          18.3            1    +++++++++++pilutil
          17.3            9    ++++++++++++PIL
           5.4            5.4  +++++++++++++
          78.3           74    +++++++++
           1.2            1.1  ++++++++++scipy._lib._numpy_compat
           2.6            2.4  ++++++++++
          14.9           14.9  +++++++++
           9.5            8.2  ++++++++
           1.3            1.2  +++++++++_stats_mstats_common
          10.2            7.1  +++++++morestats
           2              2    ++++++++
           2              2    +++++++_binned_statistic
           2.5            0.8  +++++++kde
           1.7            1.7  ++++++++
           1.5            0.5  +++++++
           3.8            3.6  +++++++_multivariate
         131              0.9  ++++++iris.analysis._area_weighted
         125.2            1.6  +++++++iris.analysis._interpolation
           1.4            1.4  ++++++++iris.analysis._scipy_interpolate
         122.1            2    ++++++++iris.analysis.cartography
          33.3           21.3  +++++++++cf_units
           9.9            1.4  ++++++++++netcdftime
           8.4            8.2  +++++++++++_netcdftime
          72              2.4  +++++++++cartopy.img_transform
          45.4            3.9  ++++++++++shapely.speedups
          38.2            1.2  +++++++++++shapely.geometry
          30.6            1.8  ++++++++++++base
          25.6            0.7  +++++++++++++shapely.coords
          23.8           22.2  ++++++++++++++shapely.geos
           2.9            1    +++++++++++++shapely.impl
           5.8            0.4  ++++++++++++geo
           1.3            0.8  +++++++++++++point
           2.5            2.4  +++++++++++shapely.speedups
          17.4            8.4  ++++++++++cartopy.crs
           5.7            5.4  +++++++++++cartopy._crs
           2.3            2.2  +++++++++++cartopy.trace
           6.2            1.8  ++++++++++cartopy.feature
           4.3            2.4  +++++++++++cartopy.io.shapereader
           1.8            1.7  ++++++++++++shapefile
          13.1            3.5  +++++++++iris.coords
           8.7            1.4  ++++++++++iris.aux_factory
           5.6            1.1  +++++++++++iris._cube_coord_common
           4.4            4.4  ++++++++++++iris.std_names
           1.6            1.5  +++++++++++iris.util
           1.5            1.4  +++++++++iris.coord_systems
           4.9            3.6  +++++++iris.experimental.regrid
           1.2            1.1  ++++++++iris.analysis._regrid
           2.7            1.1  ++++++
           1.4            1.3  +++++++
           1.3            1.2  +++++iris.analysis.maths
           3.5            3.5  +++++iris._concatenate
           1              1    +++++iris._constraints
           7              6.9  +++++iris._merge
         121.8            2.4  ++++iris.fileformats
           2.7            2.5  +++++iris.io.format_picker
           1.2            1.1  +++++
          63.8            1    +++++
          27.5            1.2  ++++++_ff_replacement
          26.2            1.3  +++++++iris.fileformats._ff
           7.7            7.6  ++++++++iris.fileformats._ff_cross_references
          17.2            3.7  ++++++++
           7.4            3.7  +++++++++iris.fileformats.rules
           2.7            2.7  ++++++++++iris.fileformats.um_cf_map
           1.1            1    +++++++++iris.fileformats.pp_rules
           4.7            1.2  +++++++++mo_pack
           3.5            3.4  ++++++++++_packing
           1.3            1.2  ++++++_fast_load
          34              1.4  ++++++_fast_load_structured_fields
          30.3            1.4  +++++++netCDF4
          28.9           27.2  ++++++++_netCDF4
           1.2            1    +++++++++utils
           2.2            0.7  +++++++iris.fileformats.um._optimal_array_structuring
           1.5            1.4  ++++++++iris.fileformats._structured_array_identification
          35.2            1.8  +++++iris_grib
          17.5            2    ++++++gribapi
          15.5            2.3  +++++++gribapi
          13             12.3  ++++++++gribapi_swig
           9.2            6.4  ++++++
           2.8            2.8  +++++++
           6.6            0.9  ++++++
           5.7            3.2  +++++++_load_convert
           1.6            1.6  ++++++++message
          13.4            3.4  +++++
           7              3.7  ++++++pyke
           2.9            1.1  +++++++pyke
           1.8            1.8  ++++++++pyke
           2.2            2.1  ++++++iris.fileformats.cf
           2.1            1    +++++
           1.1            1    ++++++iris.fileformats.nimrod_load_rules
           1.4            1.3  ++++iris_sample_data
           4.6            4.5  +++iris
          94.1            1.8  +++xarray
          62.8            2.9  ++++core.alignment
           5.2            2.6  +++++
           2.7            1.1  +++++common
           1.6            0.9  ++++++
           2.1            2    +++++indexing
          49.7           48.6  +++++variable
           1.5            0.8  ++++core.combine
          14.6            0.8  ++++core.extensions
          13.8            1.7  +++++dataarray
           3.6            0.9  ++++++plot.plot
           2.7            0.9  +++++++plot
           1.6            1.5  ++++++
           1              1    ++++++
           1.1            1    ++++++coordinates
           3.9            2.2  ++++++dataset
           1.5            1.5  +++++++
           9.4            1.7  ++++backends.api
           1.3            1.2  +++++common
           1.7            0.9  +++++netCDF4_
           1.2            1.1  +++++pydap_
           1.1            0.9  ++++backends.rasterio_
           1.2            1.2  ++++
           1              1    +++xarray
           3.5            3.4  +++dask
           2.6            2.5  ++operation
           6.5            4.8  ++io
           1.5            1.4  +++tarfile
          80.5            1.2  +interface
           3.3            3.2  ++pandas
          73.9            6.1  ++seaborn
          67.7            2.7  +++element
           8.3            8    ++++annotation
          17             14.6  ++++chart
           2.2            1.8  +++++util
          19.9            4.9  ++++chart3d
          15             11.4  +++++raster
           2.5            2.4  ++++++tabular
          19.8            7.4  ++++graphs
          12.3           12.2  +++++path
           2.1            1.9  ++collector
          11.7            1.4  +operation
          10.3           10    ++element
           4              2.5  +
           1.2            1.1  ++settings
          83.9            1.6  +ipython
           1.6            0.5  ++IPython.display
           1.1            1    +++IPython.lib.display
           6.8            6.6  ++element.comparison
          62.3            1.5  ++plotting.renderer
          24.1            4.1  +++plot
          19.8            7.9  ++++util
          11.7            9.3  +++++matplotlib.cm
           1.2            1.2  ++++++matplotlib._cm
           1.1            1.1  ++++++matplotlib._cm_listed
          36.6            2.2  +++renderer
          33.1           32.8  ++++widgets
           7              1.8  ++magics
           5              4.4  +++holoviews.util.parser
           1.4            1.2  ++display_hooks
           3              2.8  ++widgets
philippjfr commented 6 years ago

After an initial look I don't think there's much we can do about numpy, pandas and probably scipy. I would however consider moving the iris extension into GeoViews, which will shave off about 700ms. We could also investigate why numba gets imported by default which could shave another 500ms off.

philippjfr commented 6 years ago

Just tested it without the iris data interface, which brings it down to 1549.9 ms vs. 3858.4 ms with it. I'd say we should definitely move it to GeoViews.

jlstevens commented 6 years ago

Sounds good.

I would also like to defer scipy imports to where they are needed, presumably in operations but not much else of holoviews needs scipy afaik.

jbednar commented 6 years ago

Moving iris support does make sense; there surely will be very few users who want Iris without GeoViews. (Conversely, not sure if it's possible to copy WMTS from GeoViews to HV? Seems like that part wouldn't depend on the rest of GeoViews, as long as projections are left up to the user, and I think that's the part of GV people are mostly likely to want (and thus could have without having the other heavy dependencies of GV. But I digress. :-)

I always get confused by profiler output, but it seems to me to be saying that 0.6s is spent importing numpy. That's odd, because launching python and importing numpy by itself together take only 0.2s:

$ time python -c "import numpy ; print(numpy.__version__)"
1.13.1
0.186u 0.027s 0:00.21 95.2% 0+0k 0+0io 0pf+0w

Maybe it's importing specific things from numpy that is expensive, not numpy as a module?

philippjfr commented 6 years ago

So I've just had a go at this but the main issue that there is no way for me to move the tests into geoviews since they rely on baseclasses which are in the holoviews root directory. I think we should start moving tests into the modules they apply to rather than keeping them all separate. This is also the structure other projects like bokeh follow.

jbednar commented 5 years ago

Just to clarify, this issue is "closed - done", not "closed - wontfix", despite the last comment above...