search

python / cpython

The Python programming language
https://www.python.org
Other
63.49k stars 30.41k forks source link

Fatal Python error: Py_Initialize: unable to load the file system codec #77792

Closed c9459d89-998a-4a2a-9d05-c5e376dd295e closed 6 years ago

c9459d89-998a-4a2a-9d05-c5e376dd295e commented 6 years ago
BPO 33611
Nosy @ronaldoussoren, @vstinner, @ned-deily, @methane
Files
  • image.png
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = created_at = labels = ['OS-mac', 'invalid', 'type-crash'] title = 'Fatal Python error: Py_Initialize: unable to load the file system codec' updated_at = user = 'https://bugs.python.org/sm' ``` bugs.python.org fields: ```python activity = actor = 'ned.deily' assignee = 'none' closed = True closed_date = closer = 'vstinner' components = ['macOS'] creation = creator = 'sm' dependencies = [] files = ['47614'] hgrepos = [] issue_num = 33611 keywords = [] message_count = 13.0 messages = ['317361', '317370', '317371', '317469', '317485', '317487', '317488', '317525', '317539', '317573', '317578', '317584', '317585'] nosy_count = 5.0 nosy_names = ['ronaldoussoren', 'vstinner', 'ned.deily', 'methane', 'sm'] pr_nums = [] priority = 'normal' resolution = 'not a bug' stage = 'resolved' status = 'closed' superseder = None type = 'crash' url = 'https://bugs.python.org/issue33611' versions = ['Python 3.6'] ```

    c9459d89-998a-4a2a-9d05-c5e376dd295e commented 6 years ago

    Fatal Python error: Py_Initialize: unable to load the file system codec ModuleNotFoundError: No module named 'encodings' ########################################### Process: Python [1889] Path: /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python Identifier: Python Version: 3.6.5 (3.6.5) Code Type: X86-64 (Native) Parent Process: bash [722] Responsible: Python [1889] User ID: 501

    Date/Time: 2018-05-22 23:52:40.997 -0700 OS Version: Mac OS X 10.13.4 (17E202) Report Version: 12 Anonymous UUID: F667AFCE-E0F9-BF0F-FB74-372904C44DB5

    Time Awake Since Boot: 2000 seconds

    System Integrity Protection: disabled

    Crashed Thread: 0 Dispatch queue: com.apple.main-thread

    Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY

    Application Specific Information: abort() called

    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libsystemkernel.dylib 0x00000001018eeb6e \_pthread_kill + 10 1 libsystem_pthread.dylib 0x0000000101928080 pthread_kill + 333 2 libsystem_c.dylib 0x00000001015c51ae abort + 127 3 org.python.python 0x0000000100877611 Py_FatalError + 65 4 org.python.python 0x00000001008793c1 _Py_InitializeEx_Private + 1281 5 org.python.python 0x0000000100896b40 Py_Main + 1360 6 org.python.python 0x0000000100000dfe 0x100000000 + 3582 7 org.python.python 0x0000000100000c34 0x100000000 + 3124

    Thread 0 crashed with X86 Thread State (64-bit): rax: 0x0000000000000000 rbx: 0x000000010192e380 rcx: 0x00007fff5bffef58 rdx: 0x0000000000000000 rdi: 0x0000000000000307 rsi: 0x0000000000000006 rbp: 0x00007fff5bffef90 rsp: 0x00007fff5bffef58 r8: 0x0000000000000000 r9: 0x0000000000000006 r10: 0x0000000000000000 r11: 0x0000000000000202 r12: 0x0000000000000307 r13: 0x0000000000000002 r14: 0x0000000000000006 r15: 0x000000000000002d rip: 0x00000001018eeb6e rfl: 0x0000000000000202 cr2: 0x0000000101bd7020

    Logical CPU: 0 Error Code: 0x02000148 Trap Number: 133

    Binary Images: 0x100000000 - 0x100000ff7 +org.python.python (3.6.5 - 3.6.5) \<C539EB14-B454-D151-7B93-6A12255582B2> /Library/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python 0x100003000 - 0x10049dff7 com.apple.CoreFoundation (6.9 - 1452.23) \<358C547D-E227-3228-8218-62327F4605C8> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 0x100750000 - 0x100977fef +org.python.python (3.6.5, [c] 2001-2018 Python Software Foundation. - 3.6.5) \<EBF198CD-A37E-A0F1-1E7C-19B9F912D34A> /Library/Frameworks/Python.framework/Versions/3.6/Python 0x100ac0000 - 0x100ac1ffb libSystem.B.dylib (1252.50.4) \<F64430FD-30A6-328D-A476-0AED41D371C6> /usr/lib/libSystem.B.dylib 0x100ac8000 - 0x100ac9ff3 libDiagnosticMessagesClient.dylib (104) \<9712E980-76EE-3A89-AEA6-DF4BAF5C0574> /usr/lib/libDiagnosticMessagesClient.dylib 0x100acf000 - 0x100cf6ffb libicucore.A.dylib (59173.0.1) \<CECAD5E5-3EFD-3AAC-AD9B-E355B2DD5E9C> /usr/lib/libicucore.A.dylib 0x100dc8000 - 0x1011b93b7 libobjc.A.dylib (723) \<BEEA51E2-1C4D-3672-87F6-F267B4AC6221> /usr/lib/libobjc.A.dylib 0x10136a000 - 0x10137cffb libz.1.dylib (70) \<48C67CFC-940D-3857-8DAD-857774605352> /usr/lib/libz.1.dylib 0x101382000 - 0x101386ff7 libcache.dylib (80) \<092479CB-1008-3A83-BECF-E115F24D13C1> /usr/lib/system/libcache.dylib 0x10138c000 - 0x101396ff3 libcommonCrypto.dylib (60118.50.1) \<029F5985-9B6E-3DCB-9B96-FD007678C6A7> /usr/lib/system/libcommonCrypto.dylib 0x1013a4000 - 0x1013abfff libcompiler_rt.dylib (62) \<968B8E3F-3681-3230-9D78-BB8732024F6E> /usr/lib/system/libcompiler_rt.dylib 0x1013b9000 - 0x1013c2ffb libcopyfile.dylib (146.50.5) \<3885083D-50D8-3EEC-B481-B2E605180D7F> /usr/lib/system/libcopyfile.dylib 0x1013c9000 - 0x10144efff libcorecrypto.dylib (562.50.17) \<67007279-24E1-3F30-802D-A55CD5C27946> /usr/lib/system/libcorecrypto.dylib 0x10146b000 - 0x1014a4ff7 libdispatch.dylib (913.50.12) \<848EEE57-4235-3A61-9A52-C0097DD2AB5E> /usr/lib/system/libdispatch.dylib 0x1014df000 - 0x1014fcff7 libdyld.dylib (551.3) \<CF59A5A5-288B-30E6-BD42-9056B4E4139A> /usr/lib/system/libdyld.dylib 0x101517000 - 0x101517ffb libkeymgr.dylib (28) \<E34E283E-90FA-3C59-B48E-1277CDB9CDCE> /usr/lib/system/libkeymgr.dylib 0x10151c000 - 0x10151cff7 liblaunch.dylib (1205.50.76) \<4D52BB64-C568-3A36-8935-2480427EE2A2> /usr/lib/system/liblaunch.dylib 0x101523000 - 0x101527ffb libmacho.dylib (906) \<1902A611-081A-3452-B11E-EBD1B166E831> /usr/lib/system/libmacho.dylib 0x10152e000 - 0x101530ff3 libquarantine.dylib (86) \<26C0BA22-8F93-3A07-9A4E-C8D53D2CE42E> /usr/lib/system/libquarantine.dylib 0x101537000 - 0x101538ff3 libremovefile.dylib (45) \<711E18B2-5BBE-3211-A916-56740C27D17A> /usr/lib/system/libremovefile.dylib 0x10153e000 - 0x101555fff libsystem_asl.dylib (356.50.1) \<3B24F2D1-B578-359D-ADB2-0ED19A364C38> /usr/lib/system/libsystem_asl.dylib 0x101563000 - 0x101563fff libsystem_blocks.dylib (67) \<17303FDF-0D2D-3963-B05E-B4DF63052D47> /usr/lib/system/libsystem_blocks.dylib 0x101568000 - 0x1015f1ff7 libsystem_c.dylib (1244.50.9) \<1187BFE8-4576-3247-8177-481554E1F9E7> /usr/lib/system/libsystem_c.dylib 0x10161a000 - 0x10161dffb libsystem_configuration.dylib (963.50.8) \<DF6B5287-203E-30CB-9947-78DF446C72B8> /usr/lib/system/libsystem_configuration.dylib 0x101624000 - 0x101627ffb libsystem_coreservices.dylib (51) \<486000D3-D8CB-3BE7-8EE5-8BF380DE6DF7> /usr/lib/system/libsystem_coreservices.dylib 0x10162d000 - 0x10162efff libsystem_darwin.dylib (1244.50.9) \<09C21A4A-9EE0-388B-A9D9-DFF8F6758791> /usr/lib/system/libsystem_darwin.dylib 0x101634000 - 0x10163aff7 libsystem_dnssd.dylib (878.50.17) \<9033B909-BCF7-37EB-A040-ADE8081611D6> /usr/lib/system/libsystem_dnssd.dylib 0x101641000 - 0x10168aff7 libsystem_info.dylib (517.30.1) \<AB634A98-B8AA-3804-8436-38261FC8EC4D> /usr/lib/system/libsystem_info.dylib 0x1016a2000 - 0x1016edfcb libsystem_m.dylib (3147.50.1) \<FC2E58BA-E6D5-3D2A-89CA-67F212294136> /usr/lib/system/libsystem_m.dylib 0x1016fc000 - 0x10171bfff libsystem_malloc.dylib (140.50.6) \<7FD43735-9DDD-300E-8C4A-F909A74BDF49> /usr/lib/system/libsystem_malloc.dylib 0x101728000 - 0x101858ff3 libsystem_network.dylib (1229.51.2) \<F65ED095-8ABA-3355-93F8-B9D0505932F4> /usr/lib/system/libsystem_network.dylib 0x101899000 - 0x1018a3ffb libsystem_networkextension.dylib (767.50.25) \<758F1414-796D-3422-83C9-92D2915A06CE> /usr/lib/system/libsystem_networkextension.dylib 0x1018b0000 - 0x1018b9ff3 libsystem_notify.dylib (172) \<08012EC0-2CD2-34BE-BF93-E7F56491299A> /usr/lib/system/libsystem_notify.dylib 0x1018c1000 - 0x1018c4fff libsystem_sandbox.dylib (765.50.51) \<B48D256E-D7DB-3D4F-BC95-34557170C7AA> /usr/lib/system/libsystem_sandbox.dylib 0x1018cb000 - 0x1018ccff3 libsystem_secinit.dylib (30) \<DE8D14E8-A276-3FF8-AE13-77F7040F33C1> /usr/lib/system/libsystem_secinit.dylib 0x1018d2000 - 0x1018f8ff7 libsystem_kernel.dylib (4570.51.2) \<7FF1E390-2FAC-33A7-A545-60D9BB6692D4> /usr/lib/system/libsystem_kernel.dylib 0x101912000 - 0x101919ff7 libsystem_platform.dylib (161.50.1) \<6355EE2D-5456-3CA8-A227-B96E8F1E2AF8> /usr/lib/system/libsystem_platform.dylib 0x101922000 - 0x10192dfff libsystem_pthread.dylib (301.50.1) \<0E51CCBA-91F2-34E1-BF2A-FEEFD3D321E4> /usr/lib/system/libsystem_pthread.dylib 0x10193a000 - 0x101941ff7 libsystem_symptoms.dylib (820.50.37) \<ED7350E0-B68B-374A-9AB7-DC1F195A8327> /usr/lib/system/libsystem_symptoms.dylib 0x101948000 - 0x10195bfff libsystem_trace.dylib (829.50.17) \<6568D68B-1D4C-38EE-90A9-54821D6403C0> /usr/lib/system/libsystem_trace.dylib 0x10196a000 - 0x10196fff7 libunwind.dylib (35.3) \<BEF3FB49-5604-3B5F-82B5-332B80023AC3> /usr/lib/system/libunwind.dylib 0x101977000 - 0x1019a4fff libxpc.dylib (1205.50.76) \<25DB244E-217D-3CE0-A8F2-0C4255783B42> /usr/lib/system/libxpc.dylib 0x1019c7000 - 0x1019faff7 libclosured.dylib (551.3) \<DC3DA678-9C40-339C-A9C6-32AB74FCC682> /usr/lib/closure/libclosured.dylib 0x101a16000 - 0x101a3aff7 libc++abi.dylib (400.8.2) \<EF5E37D7-11D9-3530-BE45-B986612D13E2> /usr/lib/libc++abi.dylib 0x101a4a000 - 0x101aa0fff libc++.1.dylib (400.9) \<7D3DACCC-3804-393C-ABC1-1A580FD00CB6> /usr/lib/libc++.1.dylib 0x7fff61e04000 - 0x7fff61e4e9df dyld (551.3) \<AFAB4EFA-7020-34B1-BBEF-0F26C6D3CA36> /usr/lib/dyld

    External Modification Summary: Calls made by other processes targeting this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by all processes on this machine: task_for_pid: 950 thread_create: 0 thread_set_state: 0

    VM Region Summary: ReadOnly portion of Libraries: Total=22.8M resident=0K(0%) swapped_out_or_unallocated=22.8M(100%) Writable regions: Total=26.4M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=26.4M(100%)

                                VIRTUAL   REGION

    REGION TYPE SIZE COUNT (non-coalesced) \=========== ======= ======= Kernel Alloc Once 8K 2 MALLOC 9.9M 9 MALLOC guard page 16K 5 __DATA 4332K 61 __LINKEDIT 4716K 47 __TEXT 18.2M 47 __UNICODE 560K 2 __UNIXSTACK 16.0M 2 shared memory 8K 3 \=========== ======= ======= TOTAL 53.6M 169

    methane commented 6 years ago

    It seems not a bug of Python. Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

    vstinner commented 6 years ago

    Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

    We should maybe dump sys.path into stderr on the specific "unable to load the file system codec" error.

    c9459d89-998a-4a2a-9d05-c5e376dd295e commented 6 years ago

    if it could be broken installation or configuration (like incorrectly set PYTHONHOME or any other python related environment variables), then this crash should be expected every time I invoke python executable and attempt to run python scripts. However, it is not a case, and some of my scripts are running just fine. I definitely have some bugs in certain scripts (they are accessing sqlite3) and this is when I am seeing this crash of python. ...and it is python, who is crashing: I think whatever bugs in my scripts are - it is not a valid reason for python (which is wrapper around c code and compiles c code on the fly) to crash.

    I have reinstalled python3 on my laptop with brew and for half of the day it was running fine...

    On Wed, May 23, 2018 at 2:59 AM, STINNER Victor \report@bugs.python.org\ wrote:

    STINNER Victor \vstinner@redhat.com\ added the comment:

    > Maybe, your Python installation is broken, or you have wrong PYTHONPATH envvar.

    We should maybe dump sys.path into stderr on the specific "unable to load the file system codec" error.

    ---------- nosy: +vstinner

    Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue33611\

    methane commented 6 years ago

    if it could be broken installation or configuration (like incorrectly set PYTHONHOME or any other python related environment variables), then this crash should be expected every time I invoke python executable and attempt to run python scripts. However, it is not a case, and some of my scripts are running just fine.

    There are many possibilities.

    Anyway, your report doesn't have enough information to specify what's wrong on your environment. Valid Python environment doesn't raise such error. I use Python everyday.

    methane commented 6 years ago

    We should maybe dump sys.path into stderr on the specific "unable to load the file system codec" error.

    In addition to sys.path, I think "What's current python home (prefix and exec-prefix), and where it come from" is very useful.

    I recently fixed Vim's bug. https://github.com/vim/vim/pull/2787

    Vim called Py_SetPythonHome with path of Python when vim is compiled. After reinstalling Python to another directory, vim fails with "no module named encoding" error everytime. This suffered many homebrew users because homebrew use different directory to install Python every revisions.

    Python home will be come from different locations:

    vstinner commented 6 years ago

    I completed the _PyCoreConfig structure to add home, module_search_path_env, and many other options. It might help to guess how the final configuration has been computed. I also reworked the code to clarify when and how sys.path is computed. sys.path value (not the actual sys.path Python list) is now computed earlier.

    c9459d89-998a-4a2a-9d05-c5e376dd295e commented 6 years ago

    Guys, First of all - thank you for accepting it and trying to fix it. I lost two days attempting to resurrect python3 on my laptop and ended up with uninstalling it and restoring python 2.7.10. I agree (and I think it is separate and well known python's problem) - there are several tools/techniques installing python and paths ARE different between OS. this is what I was observing today comparing Mac OS and CentOS 7. and as it seems to me this "complaint" about "encoding" is exactly about it

    Regards, Slava

    On Wed, May 23, 2018 at 6:20 PM, STINNER Victor \report@bugs.python.org\ wrote:

    STINNER Victor \vstinner@redhat.com\ added the comment:

    I completed the _PyCoreConfig structure to add home, module_search_path_env, and many other options. It might help to guess how the final configuration has been computed. I also reworked the code to clarify when and how sys.path is computed. sys.path value (not the actual sys.path Python list) is now computed earlier.

    ----------

    Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue33611\

    methane commented 6 years ago

    I feel your message is just a complaint, not issue report. You didn't report any concrete information about your environment. (e.g. How did you install Python, which bash do you use, env output, which python output, etc...)

    Your message made I feel bad, without any useful information. If you really want to fix your environment, you can use other place (python-list, stackoverflow, or something else).

    I think we can improve error message, but I don't want to keep using this issue. May I close this issue?

    vstinner commented 6 years ago

    msg317525, s (sm): "Guys, (...)"

    In the future please use gender-neutral words such as "folks" and "y'all" instead of "guys". Thanks.

    vstinner commented 6 years ago

    Slava doesn't seem to want to provide any useful information to debug his event, whereas Naoki asked multiple times gently. I also dislike Slava's tone in answers, trolling Python 3 doesn't help to fix your issue...

    There are millions of Python 3 users on Windows, and it's the first time that I see this bug on Windows. So it's likely something very specific to your setup.

    It's very unlikely a bug in the Windows installer of Python, so I close the issue.

    c9459d89-998a-4a2a-9d05-c5e376dd295e commented 6 years ago

    :) sure. if you want to close it - it's up to you. if you however, want to fix it, please tell me what information do you need me to provide. I don't think my tone was bad/offensive. ...and it's still a bug, regardless of my tone or anything else. and by the way, I don't like yours tone.

    I used brew and pip

    bash --version GNU bash, version 4.4.12(1)-release (x86_64-apple-darwin16.3.0) Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later \<http://gnu.org/licenses/gpl.html

    which python /usr/bin/python

    env _system_type=Darwin OPT=/opt LANG=en_US.UTF-8 XPC_FLAGS=0x0 DISPLAY=/private/tmp/com.apple.launchd.XNIGqOXCBz/org.macosforge.xquartz:0 rvm_bin_path=/Users/xxxx/.rvm/bin TERM_SESSION_ID=781A3902-E266-48B2-8BDF-0114DE0CAC97 rvm_version=1.26.11 (latest) RUBY_VERSION=ruby-2.2.3 GEM_HOME=/Users/xxxx/.rvm/gems/ruby-2.2.3 JAVA_HOME=/usr SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.tIfMUtKKFM/Listeners GVM_ROOT=/Users/xxxx/.gvm STDLIB=/usr/local/lib USER=xxxx RBENV_SHELL=bash GOPATH=/Users/xxxx/working/golang/projects PWD=/Users/xxxx/working/python/scripts/passman/db _system_version=10.13 HOME=/Users/xxxx GOROOT=/usr/local/go LC_CTYPE=en_US.UTF-8 _system_name=OSX rvm_user_install_flag=1 TERM_PROGRAM=Apple_Terminal TERM_PROGRAM_VERSION=404 Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.zL9jzzDVzY/Render _system_arch=x86_64 GEM_PATH=/Users/xxxx/.rvm/gems/ruby-2.2.3:/Users/xxxx/.rvm/gems/ruby-2.2.3@global TMPDIR=/var/folders/g8/xflwp75s5nv0lc4wmhz2t2700000gn/T/ rvm_stored_umask=0022 rvm_loaded_flag=1 rvm_path=/Users/xxxx/.rvm XPC_SERVICE_NAME=0 SHELL=/bin/bash TERM=xterm-256color SECURITYSESSIONID=186a7 rvm_prefix=/Users/xxxx SHLVL=2 PYENV_SHELL=bash PYTHONPATH=/usr/local/lib/python2.7/site-packages LOGNAME=xxxx LDFLAGS=/usr/local/opt/bison/lib GOBIN=/usr/local/go/bin STDINCLUDE=/usr/local/include MY_RUBY_HOME=/Users/xxxx/.rvm/rubies/ruby-2.2.3 PATH=/opt/:/usr/local/lib:/usr/local/include:/usr/local/opt/bison/bin:/usr/local/opt/e2fsprogs/sbin:/Users/xxxx/.pyenv/shims:/Users/xxxx/.rvm/gems/ruby-2.2.3/bin:/Users/xxxx/.rvm/gems/ruby-2.2.3@global /bin:/Users/xxxx/.rvm/rubies/ruby-2.2.3/bin:/Users/xxxx/.rbenv/shims:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Applications/Wireshark.app/Contents/MacOS:/Applications/Postgres.app/Contents/Versions/latest/bin/:/Users/xxxx/.rvm/bin:/usr/local/go/bin:/Users/xxxx/working/golang/projects:/usr/local/go/bin:/usr/local/include:/opt:/usr/local/lib:/Users/xxxx/.rvm/bin:/usr/local/go/bin:/usr/local:/opt IRBRC=/Users/xxxx/.rvm/rubies/ruby-2.2.3/.irbrc GOROOTBOOTSTRAP=/usr/local/go \=/usr/bin/env

    On Thu, May 24, 2018 at 7:27 AM, STINNER Victor \report@bugs.python.org\ wrote:

    STINNER Victor \vstinner@redhat.com\ added the comment:

    Slava doesn't seem to want to provide any useful information to debug his event, whereas Naoki asked multiple times gently. I also dislike Slava's tone in answers, trolling Python 3 doesn't help to fix your issue...

    There are millions of Python 3 users on Windows, and it's the first time that I see this bug on Windows. So it's likely something very specific to your setup.

    It's very unlikely a bug in the Windows installer of Python, so I close the issue.

    ---------- resolution: -> not a bug stage: -> resolved status: open -> closed

    Python tracker \report@bugs.python.org\ \https://bugs.python.org/issue33611\

    ned-deily commented 6 years ago

    PYTHONPATH=/usr/local/lib/python2.7/site-packages

    That looks pretty suspicious right there. It's pulling in 2.7 modules. Normally you should never need to set PYTHONPATH in a shell initialization script, like .profile, rarely elsewhere.