lithomas1
commented
1 year ago Can you paste output of python -X importtime -c "import pandas as pd"?
Open pyrito opened 1 year ago
lithomas1
commented
1 year ago Can you paste output of python -X importtime -c "import pandas as pd"?
pyrito
commented
1 year ago @lithomas1 sure thing!
pyrito
commented
1 year ago @lithomas1 for further context, it seems like this is a one time cost that we pay in a new environment. I see that a fair amount of time is spent importing numpy, so I'm not sure if there's anything we can do here. Future imports of pandas is much faster, which I presume is some caching done by conda/python.
lithomas1
commented
1 year ago Yeah, the numpy import is taking 5 seconds for some reason.
Ideally pandas should load faster, but the main reason for the slowness is that we load all of our extension modules at import time (which is slow, since some are quite large).
rhshadrach
commented
1 year ago Something indeed seems off; your walltime is significantly higher than your CPU time. This indicates to me that something is blocked (e.g. IO). Is your pandas installation on a networked drive (e.g. an NFS)?
pyrito
commented
1 year ago @rhshadrach no this is all on my local M1 macbook pro.
carbon-starlight
commented
5 months ago @lithomas1 for further context, it seems like this is a one time cost that we pay in a new environment. I see that a fair amount of time is spent importing numpy, so I'm not sure if there's anything we can do here. Future imports of
pandasis much faster, which I presume is some caching done by conda/python.
I face the same problem (more severe—importing pandas may take up to 114 seconds on my old Debian 12 IA-32 computer, no GUI running), but is takes long every time I run a script with importing it in the same virtual environment. I don't use conda though.
(.venv) user@debian:~/Projects/WikiSchedule$ python3 -X importtime -c "import pandas as pd" import time: self [us] | cumulative | imported package import time: 1577 | 1577 | _io import time: 660 | 660 | marshal import time: 3310 | 3310 | posix import time: 3527 | 9073 | _frozen_importlib_external import time: 1082 | 1082 | time import time: 1439 | 2520 | zipimport import time: 532 | 532 | _codecs import time: 2349 | 2880 | codecs import time: 3530 | 3530 | encodings.aliases import time: 5398 | 11807 | encodings import time: 1827 | 1827 | encodings.utf_8 import time: 779 | 779 | _signal import time: 275 | 275 | _abc import time: 1010 | 1284 | abc import time: 1568 | 2851 | io import time: 297 | 297 | _stat import time: 506 | 802 | stat import time: 5039 | 5039 | _collections_abc import time: 357 | 357 | genericpath import time: 719 | 1075 | posixpath import time: 2526 | 9440 | os import time: 600 | 600 | _sitebuiltins import time: 3685 | 3685 | _distutils_hack import time: 559 | 559 | apport_python_hook import time: 1159 | 1718 | sitecustomize import time: 5526 | 20968 | site import time: 1278 | 1278 | __future__ import time: 2351 | 2351 | warnings import time: 2379 | 2379 | types import time: 569 | 569 | _operator import time: 2450 | 3019 | operator import time: 662 | 662 | itertools import time: 1415 | 1415 | keyword import time: 1723 | 1723 | reprlib import time: 593 | 593 | _collections import time: 6484 | 10875 | collections import time: 534 | 534 | _functools import time: 3875 | 15284 | functools import time: 9638 | 30319 | enum import time: 1295 | 1295 | numpy._utils._convertions import time: 1449 | 2744 | numpy._utils import time: 3009 | 36070 | numpy._globals import time: 1886 | 1886 | numpy.exceptions import time: 1043 | 1043 | numpy.version import time: 329 | 329 | numpy._distributor_init_local import time: 1189 | 1518 | numpy._distributor_init import time: 1164 | 1164 | numpy._utils._inspect import time: 917 | 917 | math import time: 964 | 964 | _datetime import time: 5872 | 7752 | datetime import time: 2026 | 2026 | numpy.core._exceptions import time: 932 | 932 | numpy.dtypes import time: 18367 | 29075 | numpy.core._multiarray_umath import time: 2252 | 32490 | numpy.core.overrides import time: 4234 | 36724 | numpy.core.multiarray import time: 1607 | 1607 | numpy.core.umath import time: 2309 | 2309 | numbers import time: 1335 | 1335 | numpy.core._string_helpers import time: 463 | 463 | _sre import time: 2365 | 2365 | re._constants import time: 2753 | 5117 | re._parser import time: 1122 | 1122 | re._casefix import time: 2315 | 9015 | re._compiler import time: 1314 | 1314 | copyreg import time: 4142 | 14470 | re import time: 1324 | 15794 | fnmatch import time: 637 | 637 | errno import time: 1256 | 1256 | urllib import time: 7189 | 8444 | urllib.parse import time: 707 | 707 | _winapi import time: 804 | 804 | nt import time: 769 | 769 | nt import time: 905 | 905 | nt import time: 795 | 795 | nt import time: 762 | 762 | nt import time: 1328 | 6066 | ntpath import time: 6268 | 37207 | pathlib import time: 993 | 993 | pickle5 import time: 497 | 497 | _struct import time: 1300 | 1796 | struct import time: 2048 | 2048 | _compat_pickle import time: 903 | 903 | _pickle import time: 770 | 770 | org import time: 196 | 966 | org.python import time: 195 | 1161 | org.python.core import time: 10052 | 15957 | pickle import time: 2316 | 56471 | numpy.compat.py3k import time: 1511 | 57982 | numpy.compat import time: 1446 | 1446 | numpy.core._dtype import time: 3725 | 63152 | numpy.core._type_aliases import time: 3081 | 69875 | numpy.core.numerictypes import time: 4207 | 4207 | contextlib import time: 1226 | 1226 | collections.abc import time: 1152 | 1152 | _contextvars import time: 1432 | 2584 | contextvars import time: 2036 | 5845 | numpy.core._ufunc_config import time: 2120 | 12171 | numpy.core._methods import time: 8317 | 20488 | numpy.core.fromnumeric import time: 2870 | 23358 | numpy.core.shape_base import time: 3892 | 3892 | numpy.core.arrayprint import time: 1310 | 1310 | numpy.core._asarray import time: 7273 | 35832 | numpy.core.numeric import time: 7647 | 7647 | numpy.core.defchararray import time: 2325 | 2325 | numpy.core.records import time: 1499 | 1499 | numpy.core.memmap import time: 2339 | 2339 | numpy.core.function_base import time: 1394 | 1394 | numpy.core._machar import time: 2029 | 2029 | numpy.core.getlimits import time: 2154 | 2154 | numpy.core.einsumfunc import time: 1526 | 1526 | numpy.core._multiarray_tests import time: 13246 | 14771 | numpy.core._add_newdocs import time: 4127 | 4127 | numpy.core._add_newdocs_scalars import time: 1247 | 1247 | numpy.core._dtype_ctypes import time: 8175 | 8175 | _ast import time: 6927 | 15102 | ast import time: 2517 | 2517 | _ctypes import time: 2265 | 2265 | ctypes._endian import time: 6526 | 11306 | ctypes import time: 6280 | 32687 | numpy.core._internal import time: 1416 | 1416 | numpy._pytesttester import time: 10000 | 227664 | numpy.core import time: 254 | 227917 | numpy.core._multiarray_umath import time: 3043 | 230960 | numpy.__config__ import time: 2039 | 2039 | numpy.lib.mixins import time: 1278 | 1278 | numpy.lib.ufunclike import time: 2839 | 4116 | numpy.lib.type_check import time: 2556 | 6672 | numpy.lib.scimath import time: 1136 | 1136 | _typing import time: 15208 | 16344 | typing import time: 1674 | 1674 | numpy.lib.stride_tricks import time: 3128 | 4801 | numpy.lib.twodim_base import time: 6303 | 6303 | numpy.linalg._umath_linalg import time: 1930 | 1930 | numpy._typing._nested_sequence import time: 1140 | 1140 | numpy._typing._nbit import time: 6993 | 6993 | numpy._typing._char_codes import time: 2949 | 2949 | numpy._typing._scalars import time: 1509 | 1509 | numpy._typing._shape import time: 12421 | 12421 | numpy._typing._dtype_like import time: 18100 | 18100 | numpy._typing._array_like import time: 3534 | 48575 | numpy._typing import time: 10052 | 86073 | numpy.linalg.linalg import time: 1579 | 87651 | numpy.linalg import time: 2962 | 90613 | numpy.matrixlib.defmatrix import time: 1452 | 92064 | numpy.matrixlib import time: 2518 | 2518 | numpy.lib.histograms import time: 7946 | 10463 | numpy.lib.function_base import time: 3692 | 106218 | numpy.lib.index_tricks import time: 3289 | 3289 | numpy.lib.nanfunctions import time: 3211 | 3211 | numpy.lib.shape_base import time: 4403 | 4403 | numpy.lib.polynomial import time: 6776 | 6776 | textwrap import time: 13075 | 13075 | platform import time: 5047 | 24897 | numpy.lib.utils import time: 2555 | 2555 | numpy.lib.arraysetops import time: 2270 | 2270 | _weakrefset import time: 4568 | 6837 | weakref import time: 2418 | 2418 | numpy.lib.format import time: 3367 | 3367 | numpy.lib._datasource import time: 3468 | 3468 | numpy.lib._iotools import time: 5432 | 21520 | numpy.lib.npyio import time: 1820 | 1820 | numpy.lib.arrayterator import time: 2880 | 2880 | numpy.lib.arraypad import time: 1504 | 1504 | numpy.lib._version import time: 5701 | 186701 | numpy.lib import time: 1546 | 1546 | numpy.fft._pocketfft_internal import time: 4901 | 6446 | numpy.fft._pocketfft import time: 1957 | 1957 | numpy.fft.helper import time: 2149 | 10552 | numpy.fft import time: 2703 | 2703 | numpy.polynomial.polyutils import time: 4440 | 4440 | numpy.polynomial._polybase import time: 4115 | 11257 | numpy.polynomial.polynomial import time: 3273 | 3273 | numpy.polynomial.chebyshev import time: 2490 | 2490 | numpy.polynomial.legendre import time: 2490 | 2490 | numpy.polynomial.hermite import time: 2817 | 2817 | numpy.polynomial.hermite_e import time: 2528 | 2528 | numpy.polynomial.laguerre import time: 2251 | 27102 | numpy.polynomial import time: 768 | 768 | backports_abc import time: 3701 | 4469 | numpy.random._common import time: 460 | 460 | binascii import time: 2461 | 2921 | base64 import time: 15275 | 15275 | _hashlib import time: 463 | 463 | _blake2 import time: 2676 | 3139 | hashlib import time: 2124 | 20537 | hmac import time: 266 | 266 | _bisect import time: 1242 | 1507 | bisect import time: 419 | 419 | _random import time: 539 | 539 | _sha512 import time: 3323 | 5786 | random import time: 1444 | 30686 | secrets import time: 4737 | 4737 | threading import time: 3590 | 43480 | numpy.random.bit_generator import time: 1990 | 1990 | numpy.random._bounded_integers import time: 1707 | 1707 | numpy.random._mt19937 import time: 6682 | 53857 | numpy.random.mtrand import time: 1992 | 1992 | numpy.random._philox import time: 1985 | 1985 | numpy.random._pcg64 import time: 1973 | 1973 | numpy.random._sfc64 import time: 8760 | 8760 | numpy.random._generator import time: 1945 | 70509 | numpy.random._pickle import time: 2002 | 72510 | numpy.random import time: 4935 | 4935 | numpy.ctypeslib import time: 293 | 293 | _opcode import time: 3002 | 3295 | opcode import time: 4982 | 8276 | dis import time: 1477 | 1477 | importlib import time: 971 | 2447 | importlib.machinery import time: 1592 | 1592 | token import time: 6203 | 7794 | tokenize import time: 1517 | 9311 | linecache import time: 11586 | 31619 | inspect import time: 18885 | 50503 | numpy.ma.core import time: 7705 | 7705 | numpy.ma.extras import time: 2566 | 60774 | numpy.ma import time: 19216 | 653261 | numpy import time: 1762 | 1762 | pytz.exceptions import time: 2024 | 2024 | pytz.lazy import time: 1956 | 1956 | pytz.tzinfo import time: 1398 | 1398 | pytz.tzfile import time: 5213 | 12351 | pytz import time: 1208 | 1208 | dateutil._version import time: 1903 | 3110 | dateutil import time: 2926 | 2926 | sysconfig import time: 4412 | 4412 | _sysconfigdata__i386-linux-gnu import time: 4258 | 11595 | pandas.compat._constants import time: 1849 | 1849 | _compression import time: 2313 | 2313 | _bz2 import time: 2338 | 6499 | bz2 import time: 2514 | 2514 | _lzma import time: 3317 | 5831 | lzma import time: 2225 | 14553 | pandas.compat.compressors import time: 1316 | 1316 | pandas.util import time: 16489 | 17804 | pandas.util.version import time: 2592 | 20395 | pandas.compat.numpy import time: 628 | 628 | pyarrow import time: 1287 | 1915 | pandas.compat.pyarrow import time: 2272 | 50728 | pandas.compat import time: 25084 | 25084 | pandas._typing import time: 1409 | 1409 | pandas.util._exceptions import time: 5769 | 32261 | pandas._config.config import time: 2012 | 2012 | pandas._config.dates import time: 559 | 559 | _locale import time: 5221 | 5779 | locale import time: 1790 | 7568 | pandas._config.display import time: 2095 | 43936 | pandas._config import time: 1148 | 1148 | pandas.core import time: 9735 | 10883 | pandas.core.config_init import time: 1611 | 1611 | pandas._libs.pandas_parser import time: 1038 | 1038 | numpy._core import time: 1970 | 3008 | numpy._core._multiarray_umath import time: 1702 | 4709 | pandas._libs.pandas_datetime import time: 1437 | 1437 | pandas._libs.tslibs.ccalendar import time: 2397 | 2397 | pandas._libs.tslibs.np_datetime import time: 8525 | 12358 | pandas._libs.tslibs.dtypes import time: 1338 | 1338 | pandas._libs.tslibs.base import time: 2578 | 2578 | pandas._libs.tslibs.nattype import time: 1258 | 1258 | pandas.compat._optional import time: 1764 | 1764 | zoneinfo._tzpath import time: 1576 | 1576 | zoneinfo._common import time: 1811 | 1811 | _zoneinfo import time: 1885 | 7034 | zoneinfo import time: 1512 | 1512 | importlib._abc import time: 1195 | 2707 | importlib.util import time: 6251 | 8957 | six import time: 434 | 434 | six.moves import time: 1813 | 1813 | dateutil.tz._common import time: 1515 | 1515 | dateutil.tz._factories import time: 343 | 343 | six.moves.winreg import time: 1873 | 2216 | dateutil.tz.win import time: 6103 | 21037 | dateutil.tz.tz import time: 1705 | 22741 | dateutil.tz import time: 3778 | 34811 | pandas._libs.tslibs.timezones import time: 3139 | 3139 | calendar import time: 5788 | 8927 | _strptime import time: 4769 | 4769 | signal import time: 524 | 524 | fcntl import time: 916 | 916 | msvcrt import time: 478 | 478 | _posixsubprocess import time: 593 | 593 | select import time: 3976 | 3976 | selectors import time: 4147 | 15399 | subprocess import time: 1435 | 16834 | pandas._config.localization import time: 3174 | 28933 | pandas._libs.tslibs.fields import time: 6139 | 69881 | pandas._libs.tslibs.timedeltas import time: 2797 | 2797 | pandas._libs.tslibs.tzconversion import time: 678 | 678 | backports_abc import time: 6462 | 82394 | pandas._libs.tslibs.timestamps import time: 691 | 691 | backports_abc import time: 1460 | 1460 | pandas._libs.properties import time: 10123 | 94667 | pandas._libs.tslibs.offsets import time: 670 | 670 | backports_abc import time: 3897 | 3897 | _decimal import time: 1300 | 5197 | decimal import time: 302 | 302 | _string import time: 4291 | 4593 | string import time: 1044 | 1044 | dateutil._common import time: 7131 | 12766 | dateutil.parser._parser import time: 2356 | 2356 | dateutil.parser.isoparser import time: 2186 | 17308 | dateutil.parser import time: 6757 | 6757 | pandas._libs.tslibs.strptime import time: 5500 | 35430 | pandas._libs.tslibs.parsing import time: 4434 | 135868 | pandas._libs.tslibs.conversion import time: 4422 | 4422 | pandas._libs.tslibs.period import time: 3015 | 3015 | pandas._libs.tslibs.vectorized import time: 2064 | 157726 | pandas._libs.tslibs import time: 269 | 157994 | pandas._libs.tslibs.nattype import time: 1363 | 1363 | pandas._libs.ops_dispatch import time: 2898 | 162254 | pandas._libs.missing import time: 10632 | 172885 | pandas._libs.hashtable import time: 3624 | 3624 | pandas._libs.algos import time: 9071 | 185580 | pandas._libs.interval import time: 1463 | 193362 | pandas._libs import time: 1145 | 1145 | pandas.core.dtypes import time: 695 | 695 | pyarrow import time: 6272 | 6967 | pandas._libs.lib import time: 4580 | 4580 | pandas.errors import time: 3562 | 3562 | pandas.core.dtypes.generic import time: 2640 | 6201 | pandas.core.dtypes.base import time: 1368 | 1368 | pandas.core.dtypes.inference import time: 9919 | 30179 | pandas.core.dtypes.dtypes import time: 2265 | 2265 | pandas.core.dtypes.common import time: 2478 | 4743 | pandas.core.dtypes.missing import time: 1741 | 1741 | pandas.util._decorators import time: 1053 | 1053 | pandas.io import time: 2018 | 3071 | pandas.io._util import time: 4326 | 7396 | pandas.core.dtypes.cast import time: 1428 | 1428 | pandas.core.dtypes.astype import time: 1701 | 3128 | pandas.core.dtypes.concat import time: 1100 | 1100 | pandas.core.array_algos import time: 2358 | 2358 | pandas.core.common import time: 2075 | 4433 | pandas.core.construction import time: 2770 | 8301 | pandas.core.array_algos.take import time: 1599 | 1599 | pandas.core.indexers.utils import time: 1354 | 2953 | pandas.core.indexers import time: 4936 | 28453 | pandas.core.algorithms import time: 1931 | 1931 | pandas.core.arrays.arrow.accessors import time: 594 | 594 | unicodedata import time: 1492 | 1492 | pandas.util._validators import time: 3323 | 3323 | pandas.core.missing import time: 2112 | 2112 | pandas._libs.ops import time: 1187 | 1187 | pandas.core.roperator import time: 1066 | 1066 | pandas.core.computation import time: 1901 | 1901 | pandas.core.computation.check import time: 1987 | 3887 | pandas.core.computation.expressions import time: 1060 | 1060 | pandas.core.ops.missing import time: 1254 | 1254 | pandas.core.ops.dispatch import time: 1089 | 1089 | pandas.core.ops.invalid import time: 3118 | 14769 | pandas.core.ops.array_ops import time: 1397 | 1397 | pandas.core.ops.common import time: 1740 | 1740 | pandas.core.ops.docstrings import time: 1307 | 1307 | pandas.core.ops.mask_ops import time: 1698 | 20908 | pandas.core.ops import time: 3236 | 3236 | pandas.core.arraylike import time: 1392 | 1392 | pandas.core.arrays._arrow_string_mixins import time: 1468 | 1468 | pandas.core.arrays._utils import time: 2453 | 2453 | pandas.compat.numpy.function import time: 1245 | 1245 | pandas.core.array_algos.quantile import time: 1796 | 1796 | pandas.core.sorting import time: 5184 | 10676 | pandas.core.arrays.base import time: 7425 | 7425 | pandas.core.nanops import time: 1340 | 1340 | pandas.core.array_algos.masked_accumulations import time: 1402 | 1402 | pandas.core.array_algos.masked_reductions import time: 1082 | 1082 | pandas.core.util import time: 2454 | 2454 | pandas._libs.hashing import time: 2249 | 5783 | pandas.core.util.hashing import time: 6977 | 22925 | pandas.core.arrays.masked import time: 1704 | 1704 | pandas._libs.arrays import time: 1674 | 1674 | pandas.core.arrays.numeric import time: 2220 | 3893 | pandas.core.arrays.floating import time: 2807 | 2807 | pandas.core.arrays.integer import time: 975 | 975 | pandas.core.array_algos.transforms import time: 4017 | 4992 | pandas.core.arrays._mixins import time: 1036 | 1036 | pandas.core.strings import time: 2046 | 2046 | pandas.core.strings.base import time: 3473 | 6554 | pandas.core.strings.object_array import time: 6581 | 18126 | pandas.core.arrays.numpy_ import time: 3785 | 30312 | pandas.core.arrays.string_ import time: 1034 | 1034 | pandas.tseries import time: 3144 | 4177 | pandas.tseries.frequencies import time: 10428 | 110926 | pandas.core.arrays.arrow.array import time: 1701 | 114558 | pandas.core.arrays.arrow import time: 2102 | 2102 | pandas.core.arrays.boolean import time: 439 | 439 | _csv import time: 2613 | 3052 | csv import time: 1733 | 1733 | zlib import time: 5287 | 7019 | shutil import time: 3576 | 3576 | pandas.core.accessor import time: 6350 | 6350 | pandas.core.base import time: 1410 | 1410 | pandas.io.formats import time: 1927 | 1927 | pandas.io.formats.console import time: 7597 | 30927 | pandas.core.arrays.categorical import time: 4839 | 4839 | pandas._libs.tslib import time: 1048 | 1048 | pandas.core.array_algos.datetimelike_accumulations import time: 10546 | 11593 | pandas.core.arrays.datetimelike import time: 1221 | 1221 | pandas.core.arrays._ranges import time: 1278 | 1278 | pandas.tseries.offsets import time: 6593 | 25522 | pandas.core.arrays.datetimes import time: 4086 | 4086 | pandas.core.arrays.timedeltas import time: 9693 | 13779 | pandas.core.arrays.interval import time: 3834 | 3834 | pandas.core.arrays.period import time: 2622 | 2622 | pandas._libs.sparse import time: 2726 | 2726 | pandas.io.formats.printing import time: 5422 | 10769 | pandas.core.arrays.sparse.array import time: 2473 | 13242 | pandas.core.arrays.sparse.accessor import time: 1387 | 14629 | pandas.core.arrays.sparse import time: 3376 | 3376 | pandas.core.arrays.string_arrow import time: 2362 | 211084 | pandas.core.arrays import time: 1414 | 1414 | pandas.core.flags import time: 669 | 669 | backports_abc import time: 4040 | 4709 | pandas._libs.internals import time: 1044 | 1044 | pandas.core._numba import time: 2067 | 3111 | pandas.core._numba.executor import time: 4780 | 12598 | pandas.core.apply import time: 769 | 769 | org import time: 178 | 947 | org.python import time: 191 | 1137 | org.python.core import time: 1558 | 2695 | copy import time: 525 | 525 | gc import time: 1301 | 1301 | _json import time: 3299 | 4599 | json.scanner import time: 3228 | 7827 | json.decoder import time: 3051 | 3051 | json.encoder import time: 1824 | 12701 | json import time: 1313 | 1313 | pandas._libs.indexing import time: 952 | 952 | pandas.core.indexes import time: 809 | 809 | backports_abc import time: 5734 | 6543 | pandas._libs.index import time: 2409 | 2409 | pandas._libs.writers import time: 2613 | 2613 | pandas._libs.join import time: 1257 | 1257 | pandas.core.array_algos.putmask import time: 1507 | 1507 | pandas.core.indexes.frozen import time: 17134 | 17134 | pandas.core.strings.accessor import time: 14753 | 46214 | pandas.core.indexes.base import time: 1807 | 1807 | pandas.core.indexes.extension import time: 2898 | 4705 | pandas.core.indexes.category import time: 4195 | 4195 | pandas.core.indexes.range import time: 1195 | 1195 | pandas.core.tools import time: 2043 | 3237 | pandas.core.tools.timedeltas import time: 6095 | 13526 | pandas.core.indexes.datetimelike import time: 1405 | 1405 | pandas.core.tools.times import time: 7406 | 22336 | pandas.core.indexes.datetimes import time: 9446 | 9446 | pandas.core.indexes.multi import time: 6020 | 6020 | pandas.core.indexes.timedeltas import time: 6388 | 21853 | pandas.core.indexes.interval import time: 3843 | 3843 | pandas.core.indexes.period import time: 2811 | 102711 | pandas.core.indexes.api import time: 10017 | 114040 | pandas.core.indexing import time: 1316 | 1316 | pandas.core.sample import time: 1554 | 1554 | pandas.core.array_algos.replace import time: 5158 | 5158 | pandas.core.internals.blocks import time: 2168 | 7326 | pandas.core.internals.api import time: 1987 | 1987 | pandas.core.internals.base import time: 2646 | 2646 | pandas.core.internals.ops import time: 5589 | 8235 | pandas.core.internals.managers import time: 4763 | 14984 | pandas.core.internals.array_manager import time: 2614 | 2614 | pandas.core.internals.concat import time: 1619 | 26541 | pandas.core.internals import time: 2541 | 2541 | pandas.core.internals.construction import time: 1088 | 1088 | pandas.core.methods import time: 1056 | 1056 | pandas.core.reshape import time: 2812 | 3867 | pandas.core.reshape.concat import time: 4119 | 4119 | dataclasses import time: 3921 | 3921 | gzip import time: 2041 | 2041 | mmap import time: 410 | 410 | pwd import time: 723 | 723 | grp import time: 7535 | 8667 | tarfile import time: 7020 | 7020 | zipfile import time: 1447 | 1447 | pandas.core.shared_docs import time: 11730 | 38942 | pandas.io.common import time: 5780 | 44721 | pandas.io.formats.format import time: 3041 | 52716 | pandas.core.methods.describe import time: 1073 | 1073 | pandas._libs.window import time: 7783 | 8855 | pandas._libs.window.aggregations import time: 2002 | 2002 | pandas._libs.window.indexers import time: 3766 | 5768 | pandas.core.indexers.objects import time: 1382 | 1382 | pandas.core.util.numba_ import time: 1579 | 1579 | pandas.core.window.common import time: 1842 | 1842 | pandas.core.window.doc import time: 1711 | 1711 | pandas.core.window.numba_ import time: 1270 | 1270 | pandas.core.window.online import time: 14734 | 14734 | pandas.core.window.rolling import time: 6794 | 43931 | pandas.core.window.ewm import time: 8831 | 8831 | pandas.core.window.expanding import time: 1554 | 54316 | pandas.core.window import time: 50048 | 318988 | pandas.core.generic import time: 6091 | 6091 | pandas.core.methods.selectn import time: 1229 | 1229 | pandas.core.reshape.util import time: 1501 | 1501 | pandas.core.tools.numeric import time: 2573 | 5302 | pandas.core.reshape.melt import time: 2368 | 2368 | pandas._libs.reshape import time: 5280 | 5280 | pandas.core.indexes.accessors import time: 1215 | 1215 | pandas.arrays import time: 5879 | 7093 | pandas.core.tools.datetimes import time: 6415 | 6415 | pandas.io.formats.info import time: 7014 | 7014 | pandas.plotting._core import time: 1754 | 1754 | pandas.plotting._misc import time: 1620 | 10387 | pandas.plotting import time: 26806 | 58346 | pandas.core.series import time: 56043 | 444768 | pandas.core.frame import time: 7623 | 7623 | pandas.core.groupby.base import time: 796 | 796 | backports_abc import time: 4543 | 5339 | pandas._libs.groupby import time: 1087 | 1087 | pandas.core.groupby.categorical import time: 3269 | 4356 | pandas.core.groupby.grouper import time: 4735 | 14429 | pandas.core.groupby.ops import time: 1364 | 1364 | pandas.core.groupby.numba_ import time: 2375 | 2375 | pandas.core.groupby.indexing import time: 15050 | 18788 | pandas.core.groupby.groupby import time: 14694 | 512897 | pandas.core.groupby.generic import time: 1381 | 514278 | pandas.core.groupby import time: 2502 | 986011 | pandas.core.api import time: 1105 | 1105 | pandas.tseries.api import time: 1267 | 1267 | pandas.core.computation.common import time: 1592 | 2858 | pandas.core.computation.align import time: 2152 | 2152 | pprint import time: 2804 | 4955 | pandas.core.computation.scope import time: 3164 | 8118 | pandas.core.computation.ops import time: 1976 | 12951 | pandas.core.computation.engines import time: 1188 | 1188 | pandas.core.computation.parsing import time: 7429 | 8616 | pandas.core.computation.expr import time: 1630 | 23196 | pandas.core.computation.eval import time: 1127 | 24322 | pandas.core.computation.api import time: 1548 | 1548 | pandas.core.reshape.encoding import time: 1900 | 1900 | _uuid import time: 2879 | 4778 | uuid import time: 6286 | 11064 | pandas.core.reshape.merge import time: 5244 | 5244 | pandas.core.reshape.pivot import time: 1894 | 1894 | pandas.core.reshape.tile import time: 1786 | 21534 | pandas.core.reshape.api import time: 1393 | 1393 | pandas.api.extensions import time: 1210 | 1210 | pandas.api.indexers import time: 1072 | 1072 | pandas.core.interchange import time: 12274 | 13345 | pandas.core.interchange.dataframe_protocol import time: 1396 | 1396 | pandas.core.interchange.utils import time: 2052 | 3447 | pandas.core.interchange.from_dataframe import time: 1304 | 18095 | pandas.api.interchange import time: 1076 | 1076 | pandas.core.dtypes.api import time: 1589 | 2665 | pandas.api.types import time: 50906 | 50906 | pandas.core.resample import time: 1445 | 1445 | pandas._libs.json import time: 1505 | 1505 | pandas.io.json._normalize import time: 1746 | 1746 | pandas.io.json._table_schema import time: 827 | 827 | backports_abc import time: 4204 | 5031 | pandas._libs.parsers import time: 3859 | 3859 | pandas.io.parsers.base_parser import time: 2002 | 5861 | pandas.io.parsers.arrow_parser_wrapper import time: 1897 | 1897 | pandas.io.parsers.c_parser_wrapper import time: 3676 | 3676 | pandas.io.parsers.python_parser import time: 13655 | 30118 | pandas.io.parsers.readers import time: 1360 | 31478 | pandas.io.parsers import time: 273 | 31751 | pandas.io.parsers.readers import time: 7243 | 43688 | pandas.io.json._json import time: 1497 | 45184 | pandas.io.json import time: 283 | 45467 | pandas.io.json._json import time: 9918 | 9918 | pandas.io.stata import time: 1568 | 107857 | pandas.api.typing import time: 2302 | 133520 | pandas.api import time: 3385 | 3385 | tempfile import time: 1946 | 5331 | pandas._testing.contexts import time: 1611 | 6941 | pandas._testing._io import time: 1524 | 1524 | pandas._testing._warnings import time: 465 | 465 | cmath import time: 1808 | 2273 | pandas._libs.testing import time: 3054 | 5326 | pandas._testing.asserters import time: 1324 | 1324 | pandas._testing.compat import time: 3210 | 18324 | pandas._testing import time: 1377 | 19701 | pandas.testing import time: 1465 | 1465 | pandas.util._print_versions import time: 1369 | 1369 | pandas.io.clipboards import time: 1470 | 1470 | pandas.io.excel._util import time: 2144 | 2144 | pandas.io.excel._calamine import time: 2485 | 2485 | pandas.io.excel._odfreader import time: 2799 | 2799 | pandas.io.excel._openpyxl import time: 2462 | 2462 | pandas.io.excel._pyxlsb import time: 2913 | 2913 | pandas.io.excel._xlrd import time: 10887 | 25158 | pandas.io.excel._base import time: 2105 | 2105 | pandas.io.excel._odswriter import time: 2449 | 2449 | pandas.io.excel._xlsxwriter import time: 1844 | 31554 | pandas.io.excel import time: 2763 | 2763 | pandas.io.feather_format import time: 1579 | 1579 | pandas.io.gbq import time: 7067 | 7067 | pandas.io.html import time: 1934 | 1934 | pandas.io.orc import time: 6841 | 6841 | pandas.io.parquet import time: 2089 | 2089 | pandas.compat.pickle_compat import time: 3761 | 5850 | pandas.io.pickle import time: 5196 | 5196 | pandas.core.computation.pytables import time: 14149 | 19345 | pandas.io.pytables import time: 2209 | 2209 | pandas.io.sas.sasreader import time: 1355 | 3563 | pandas.io.sas import time: 1304 | 1304 | pandas.io.spss import time: 5817 | 5817 | pandas.io.sql import time: 5588 | 5588 | pandas.io.xml import time: 2715 | 97281 | pandas.io.api import time: 1346 | 1346 | pandas.util._tester import time: 1266 | 1266 | pandas._version_meson import time: 4712 | 2070152 | pandas
rhshadrach
commented
5 months ago @carbon-starlight - the import times you posted says pandas is importing in 2 seconds.
carbon-starlight
commented
5 months ago That's strange: importing pandas from command line seems to be fast, but running it from inside script is slow
Code:
print('Importing pandas...')
timestamp1 = time.time()
import pandas as pd
print(f"Importing took {time.time() - timestamp1}{' f#cking' if time.time() - timestamp1 > 10 else ''} seconds")Terminal:
Importing pandas...
Importing took 113.06395554542542 f#cking seconds
Pandas version checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this issue exists on the latest version of pandas.
[ ] I have confirmed this issue exists on the main branch of pandas.
Reproducible Example
I did the following steps:
conda create --name py39-repro python=3.9conda activate py39-repropip install ipython; pip install pandasipythonInstalled Versions
Prior Performance
No response