python / cpython

The Python programming language
https://www.python.org
Other
63.87k stars 30.57k forks source link

3.3 : test_import.py causes 'make test' to fail #56165

Closed dff06f57-790c-4585-b46f-809245342eab closed 13 years ago

dff06f57-790c-4585-b46f-809245342eab commented 13 years ago
BPO 11956
Nosy @merwok
Files
  • bug_11956.patch: patch to test_import.py to skip unwritable_directory test if root
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields: ```python assignee = None closed_at = created_at = labels = ['type-bug', 'tests'] title = "3.3 : test_import.py causes 'make test' to fail" updated_at = user = 'https://bugs.python.org/JasonVasDias' ``` bugs.python.org fields: ```python activity = actor = 'neologix' assignee = 'none' closed = True closed_date = closer = 'neologix' components = ['Tests'] creation = creator = 'Jason.Vas.Dias' dependencies = [] files = ['21835'] hgrepos = [] issue_num = 11956 keywords = ['patch'] message_count = 8.0 messages = ['134781', '134784', '134787', '134789', '134871', '134875', '144907', '144920'] nosy_count = 3.0 nosy_names = ['eric.araujo', 'python-dev', 'Jason.Vas.Dias'] pr_nums = [] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue11956' versions = ['Python 3.3'] ```

    dff06f57-790c-4585-b46f-809245342eab commented 13 years ago

    Hi - I've been experiencing many errors trying to build any version of Python that will pass its test suite - see issues : bpo-11946 , bpo-11954 - and now I've been advised to raise bugs about each test failure - hence this bug. For details of my config and build procedure, please see : issue bpo-11954 .

    So, running the new ./python fails test_import.py :

    $ LD_LIBRARY_PATH=`pwd` LD_PRELINK=`pwd`/libpython3.3.so.1.0 \
      ./python /usr/src/cpython/Lib/test/test_import.py

    ====================================================================== FAIL: test_unwritable_directory (test.test_import.PycacheTests) ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/usr/src/cpython/Lib/test/test_import.py", line 545, in test_unwritable_directory
        '__pycache__', '{}.{}.pyc'.format(TESTFN, self.tag))))
    AssertionError: True is not false

    Ran 40 tests in 0.851s

    FAILED (failures=1)
    Traceback (most recent call last):
      File "/usr/src/cpython/Lib/test/test_import.py", line 658, in <module>
        test_main()
      File "/usr/src/cpython/Lib/test/test_import.py", line 652, in test_main
        RelativeImportFromImportlibTests)
      File "/usr/src/cpython/Lib/test/support.py", line 1208, in run_unittest
        _run_suite(suite)
      File "/usr/src/cpython/Lib/test/support.py", line 1191, in _run_suite
        raise TestFailed(err)
    test.support.TestFailed: Traceback (most recent call last):
      File "/usr/src/cpython/Lib/test/test_import.py", line 545, in test_unwritable_directory
        '__pycache__', '{}.{}.pyc'.format(TESTFN, self.tag))))
    AssertionError: True is not false

    Thanks for the helpful error backtrace !

    This function is somehow failing (test_import.py @ line 545 ):

    @unittest.skipUnless(os.name == 'posix', "test meaningful only on posix systems") def test_unwritabledirectory(self): # When the umask causes the new \_pycache directory to be # unwritable, the import still succeeds but no .pyc file is written. with tempumask(0o222): \_import(TESTFN) self.assertTrue(os.path.exists('__pycache')) self.assertFalse(os.path.exists(os.path.join( '__pycache', '{}.{}.pyc'.format(TESTFN, self.tag))))

    Running the same command under strace shows what the test is trying to assert shouldn't exist really doesn't : umask(022) = 0222 stat("__pycache", {st_mode=S_IFDIR|S_ISGID|0555, stsize=4096, ...}) = 0 stat("\_pycache/@test_9634_tmp.cpython-32.pyc", {st_mode=S_IFREG|0444, st_size=110, ...}) = 0 unlink("/usr/src/cpython/Lib/test/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory)

    (see full listing below)

    umask(022) = 0222 stat("pycache", {st_mode=S_IFDIR|S_ISGID|0555, st_size=4096, ...}) = 0 stat("pycache/@test_9634_tmp.cpython-32.pyc", {st_mode=S_IFREG|0444, st_size=110, ...}) = 0 unlink("/usr/src/cpython/Lib/test/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/test/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/lib/python33.zip/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/plat-linux2/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/plat-linux2/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/plat-linux2/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/usr/src/cpython/Lib/plat-linux2/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/mnt/sda3/Python-2.7/build/lib.linux-x86_64-3.3/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/mnt/sda3/Python-2.7/build/lib.linux-x86_64-3.3/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/mnt/sda3/Python-2.7/build/lib.linux-x86_64-3.3/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/mnt/sda3/Python-2.7/build/lib.linux-x86_64-3.3/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) unlink("/home/root/.local/lib/python3.3/site-packages/@test_9634_tmp.pyc") = -1 ENOENT (No such file or directory) unlink("/home/root/.local/lib/python3.3/site-packages/@test_9634_tmp.pyo") = -1 ENOENT (No such file or directory) unlink("/home/root/.local/lib/python3.3/site-packages/pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) unlink("/home/root/.local/lib/python3.3/site-packages/pycache/@test_9634_tmp.cpython-32.pyo") = -1 ENOENT (No such file or directory) lstat("pycache", {st_mode=S_IFDIR|S_ISGID|0555, st_size=4096, ...}) = 0 open("pycache", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3 getdents64(3, / 3 entries /, 32768) = 104 getdents64(3, / 0 entries /, 32768) = 0 close(3) = 0 lstat("pycache/@test_9634_tmp.cpython-32.pyc", {st_mode=S_IFREG|0444, st_size=110, ...}) = 0 unlink("pycache/@test_9634_tmp.cpython-32.pyc") = 0 rmdir("pycache") = 0 unlink("@test_9634_tmp.py") = 0 stat("/usr/src/cpython/Lib/test/test_import.py", {st_mode=S_IFREG|0644, st_size=24532, ...}) = 0 open("/usr/src/cpython/Lib/test/test_import.py", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=24532, ...}) = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0x7fff7ef66e98) = -1 ENOTTY (Inappropriate ioctl for device) fstat(3, {st_mode=S_IFREG|0644, st_size=24532, ...}) = 0 lseek(3, 0, SEEKCUR) = 0 read(3, "import builtins\nimport imp\nfrom importlib.test.import import test_relativeimports\nfrom importlib.test.import import util as importlib_util\nimport marshal\nimport os\nimport py_compile\nimport random\nimport stat\nimport sys\nimport unittest\nimport textwrap\n\nfrom test.support import (\n EnvironmentVarGuard, TESTFN, check_warnings, forget, is_jython,\n make_legacy_pyc, rmtree, run_unittest, swap_attr, swap_item, temp_umask,\n unlink, unload)\nfrom test import script_helper\n\n\ndef remove_files(name):\n for f in (name + \".py\",\n name + \".pyc\",\n name + \".pyo\",\n name + \".pyw\",\n name + \"$py.class\"):\n unlink(f)\n rmtree('pycache')\n\n\nclass ImportTests(unittest.TestCase):\n\n def setUp(self):\n remove_files(TESTFN)\n\n def tearDown(self):\n unload(TESTFN)\n\n setUp = tearDown\n\n def test_case_sensitivity(self):\n # Brief digression to test that import is case-sensitive: if we got\n # this far, we know for sure that \"random\" exists.\n with self.assertRaises(ImportError):\n import RAnDoM\n\n def test_double_const(self):\n # Another brief digression to test the accuracy of manifest float\n # constants.\n from test import double_const # don't blink -- that was the test\n\n def test_import(self):\n def test_with_extension(ext):\n # The extension is normally \".py\", perhaps \".pyw\".\n source = TESTFN + ext\n pyo = TESTFN + \".pyo\"\n if is_jython:\n pyc = TESTFN + \"$py.class\"\n else:\n pyc = TESTFN + \".pyc\"\n\n with open(source, \"w\") as f:\n print(\"# This tests Python's ability to import a\",\n ext, \"file.\", file=f)\n a = random.randrange(1000)\n b = random.randrange(1000)\n print(\"a =\", a, file=f)\n print(\"b =\", b, file=f)\n\n if TESTFN in sys.modules:\n del sys.modules[TESTFN]\n try:\n try:\n mod = import(TESTFN)\n except ImportError as err:\n self.fail(\"import from %s failed: %s\" % (ext, err))\n\n self.assertEqual(mod.a, a,\n \"module loaded (%s) but contents invalid\" % mod)\n self.assertEqual(mod.b, b,\n \"module loaded (%s) but contents invalid\" % mod)\n finally:\n forget(TESTFN)\n unlink(source)\n unlink(pyc)\n unlink(pyo)\n\n sys.path.insert(0, os.curdir)\n try:\n test_with_extension(\".py\")\n if sys.platform.startswith(\"win\"):\n for ext in [\".PY\", \".Py\", \".pY\", \".pyw\", \".PYW\", \".pYw\"]:\n test_with_extension(ext)\n finally:\n del sys.path[0]\n\n @unittest.skipUnless(os.name == 'posix',\n \"test meaningful only on posix systems\")\n def test_execute_bit_not_copied(self):\n # bpo-6070: under posix .pyc files got their execute bit set if\n # the .py file had the execute bit set, but they aren't executable.\n with temp_umask(0o022):\n sys.path.insert(0, os.curdir)\n try:\n fname = TESTFN + os.extsep + \"py\"\n open(fname, 'w').close()\n os.chmod(fname, (stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH |\n stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH))\n import(TESTFN)\n fn = imp.cache_from_source(fname)\n if not os.path.exists(fn):\n self.fail(\"import did not result in creation of \"\n \"either a .pyc or .pyo file\")\n s = os.stat(fn)\n self.assertEqual(\n stat.S_IMODE(s.st_mode),\n stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)\n finally:\n del sys.path[0]\n remove_files(TESTFN)\n u", 4096) = 4096 lseek(3, 0, SEEK_CUR) = 4096 read(3, "nload(TESTFN)\n\n def test_imp_module(self):\n # Verify that the imp module can correctly load and find .py files\n # XXX (ncoghlan): It would be nice to use support.CleanImport\n # here, but that breaks because the os module registers some\n # handlers in copy_reg on import. Since CleanImport doesn't\n # revert that registration, the module is left in a broken\n # state after reversion. Reinitialising the module contents\n # and just reverting os.environ to its previous state is an OK\n # workaround\n orig_path = os.path\n orig_getenv = os.getenv\n with EnvironmentVarGuard():\n x = imp.find_module(\"os\")\n self.addCleanup(x[0].close)\n new_os = imp.load_module(\"os\", *x)\n self.assertIs(os, new_os)\n self.assertIs(orig_path, new_os.path)\n self.assertIsNot(orig_getenv, new_os.getenv)\n\n def test_module_with_large_stack(self, module='longlist'):\n # Regression test for http://bugs.python.org/issue561858.\n filename = module + '.py'\n\n # Create a file with a list of 65000 elements.\n with open(filename, 'w') as f:\n f.write('d = [\n')\n for i in range(65000):\n f.write('\"\",\n')\n f.write(']')\n\n try:\n # Compile & remove .py file; we only need .pyc (or .pyo).\n # Bytecode must be relocated from the PEP-3147 bytecode-only location.\n py_compile.compile(filename)\n finally:\n unlink(filename)\n\n # Need to be able to load from current dir.\n sys.path.append('')\n\n try:\n make_legacy_pyc(filename)\n # This used to crash.\n exec('import ' + module)\n finally:\n # Cleanup.\n del sys.path[-1]\n unlink(filename + 'c')\n unlink(filename + 'o')\n\n def test_failing_import_sticks(self):\n source = TESTFN + \".py\"\n with open(source, \"w\") as f:\n print(\"a = 1/0\", file=f)\n\n # New in 2.4, we shouldn't be able to import that no matter how often\n # we try.\n sys.path.insert(0, os.curdir)\n if TESTFN in sys.modules:\n del sys.modules[TESTFN]\n try:\n for i in [1, 2, 3]:\n self.assertRaises(ZeroDivisionError, import, TESTFN)\n self.assertNotIn(TESTFN, sys.modules,\n \"damaged module in sys.modules on %i try\" % i)\n finally:\n del sys.path[0]\n remove_files(TESTFN)\n\n def test_import_name_binding(self):\n # import x.y.z binds x in the current namespace\n import test as x\n import test.support\n self.assertTrue(x is test, x.name)\n self.assertTrue(hasattr(test.support, \"file\"))\n\n # import x.y.z as w binds z as w\n import test.support as y\n self.assertTrue(y is test.support, y.name)\n\n def test_failing_reload(self):\n # A failing reload should leave the module object in sys.modules.\n source = TESTFN + os.extsep + \"py\"\n with open(source, \"w\") as f:\n f.write(\"a = 1\nb=2\n\")\n\n sys.path.insert(0, os.curdir)\n try:\n mod = import(TESTFN)\n self.assertIn(TESTFN, sys.modules)\n self.assertEqual(mod.a, 1, \"module has wrong attribute values\")\n self.assertEqual(mod.b, 2, \"module has wrong attribute values\")\n\n # On WinXP, just replacing the .py file wasn't enough to\n # convince reload() to reparse it. Maybe the timestamp didn't\n # move enough. We force it to get reparsed by removing the\n # compiled file too.\n remove_files(TESTFN)\n\n # Now damage the module.\n with open(source, \"w\") as f:\n f.write(\"a = 10\nb=20//0\n\")\n\n self.assertRaises(ZeroDivisionError, imp.reload, mod)\n # But we still expect the module to be in sys.modules.\n mod = sys.modules.get(TESTFN)\n self.assertIsNot", 4096) = 4096 read(3, "(mod, None, \"expected module to be in sys.modules\")\n\n # We should have replaced a w/ 10, but the old b value should\n # stick.\n self.assertEqual(mod.a, 10, \"module has wrong attribute values\")\n self.assertEqual(mod.b, 2, \"module has wrong attribute values\")\n\n finally:\n del sys.path[0]\n remove_files(TESTFN)\n unload(TESTFN)\n\n def test_file_to_source(self):\n # check if file points to the source file where available\n source = TESTFN + \".py\"\n with open(source, \"w\") as f:\n f.write(\"test = None\n\")\n\n sys.path.insert(0, os.curdir)\n try:\n mod = import(TESTFN)\n self.assertTrue(mod.file.endswith('.py'))\n os.remove(source)\n del sys.modules[TESTFN]\n make_legacy_pyc(source)\n mod = import(TESTFN)\n base, ext = os.path.splitext(mod.file)\n self.assertIn(ext, ('.pyc', '.pyo'))\n finally:\n del sys.path[0]\n remove_files(TESTFN)\n if TESTFN in sys.modules:\n del sys.modules[TESTFN]\n\n def test_import_name_binding(self):\n # import x.y.z binds x in the current namespace.\n import test as x\n import test.support\n self.assertIs(x, test, x.name)\n self.assertTrue(hasattr(test.support, \"file\"))\n\n # import x.y.z as w binds z as w.\n import test.support as y\n self.assertIs(y, test.support, y.name)\n\n def test_import_initless_directory_warning(self):\n with check_warnings(('', ImportWarning)):\n # Just a random non-package directory we always expect to be\n # somewhere in sys.path...\n self.assertRaises(ImportError, import, \"site-packages\")\n\n def test_import_by_filename(self):\n path = os.path.abspath(TESTFN)\n encoding = sys.getfilesystemencoding()\n try:\n path.encode(encoding)\n except UnicodeEncodeError:\n self.skipTest('path is not encodable to {}'.format(encoding))\n with self.assertRaises(ImportError) as c:\n import(path)\n\n def test_import_in_del_does_not_crash(self):\n # bpo-4236\n testfn = script_helper.make_script('', TESTFN, textwrap.dedent(\"\"\"\\n import sys\n class C:\n def del(self):\n import imp\n sys.argv.insert(0, C())\n \"\"\"))\n script_helper.assert_python_ok(testfn)\n\n\nclass PycRewritingTests(unittest.TestCase):\n # Test that the co_filename attribute on code objects always points\n # to the right file, even when various things happen (e.g. both the .py\n # and the .pyc file are renamed).\n\n module_name = \"unlikely_module_name\"\n module_source = \"\"\"\nimport sys\ncode_filename = sys._getframe().f_code.co_filename\nmodule_filename = file\nconstant = 1\ndef func():\n pass\nfunc_filename = func.code.co_filename\n\"\"\"\n dir_name = os.path.abspath(TESTFN)\n file_name = os.path.join(dir_name, module_name) + os.extsep + \"py\"\n compiled_name = imp.cache_from_source(file_name)\n\n def setUp(self):\n self.sys_path = sys.path[:]\n self.orig_module = sys.modules.pop(self.module_name, None)\n os.mkdir(self.dir_name)\n with open(self.file_name, \"w\") as f:\n f.write(self.module_source)\n sys.path.insert(0, self.dir_name)\n\n def tearDown(self):\n sys.path[:] = self.sys_path\n if self.orig_module is not None:\n sys.modules[self.module_name] = self.orig_module\n else:\n unload(self.module_name)\n unlink(self.file_name)\n unlink(self.compiled_name)\n rmtree(self.dir_name)\n\n def import_module(self):\n ns = globals()\n import(self.module_name, ns, ns)\n return sys.modules[self.module_name]\n\n def test_basics(self):\n mod = self.import_module()\n self.assertEqual(mod.module_filename, self.file_name)\n self.assertEqual(mod.code_filename, self.fil", 4096) = 4096 read(3, "e_name)\n self.assertEqual(mod.func_filename, self.file_name)\n del sys.modules[self.module_name]\n mod = self.import_module()\n self.assertEqual(mod.module_filename, self.file_name)\n self.assertEqual(mod.code_filename, self.file_name)\n self.assertEqual(mod.func_filename, self.file_name)\n\n def test_incorrect_code_name(self):\n py_compile.compile(self.file_name, dfile=\"another_module.py\")\n mod = self.import_module()\n self.assertEqual(mod.module_filename, self.file_name)\n self.assertEqual(mod.code_filename, self.file_name)\n self.assertEqual(mod.func_filename, self.file_name)\n\n def test_module_without_source(self):\n target = \"another_module.py\"\n py_compile.compile(self.file_name, dfile=target)\n os.remove(self.file_name)\n pyc_file = make_legacy_pyc(self.file_name)\n mod = self.import_module()\n self.assertEqual(mod.module_filename, pyc_file)\n self.assertEqual(mod.code_filename, target)\n self.assertEqual(mod.func_filename, target)\n\n def test_foreign_code(self):\n py_compile.compile(self.file_name)\n with open(self.compiled_name, \"rb\") as f:\n header = f.read(8)\n code = marshal.load(f)\n constants = list(code.co_consts)\n foreign_code = test_main.code\n pos = constants.index(1)\n constants[pos] = foreign_code\n code = type(code)(code.co_argcount, code.co_kwonlyargcount,\n code.co_nlocals, code.co_stacksize,\n code.co_flags, code.co_code, tuple(constants),\n code.co_names, code.co_varnames, code.co_filename,\n code.co_name, code.co_firstlineno, code.co_lnotab,\n code.co_freevars, code.co_cellvars)\n with open(self.compiled_name, \"wb\") as f:\n f.write(header)\n marshal.dump(code, f)\n mod = self.import_module()\n self.assertEqual(mod.constant.co_filename, foreign_code.co_filename)\n\n\nclass PathsTests(unittest.TestCase):\n SAMPLES = ('test', 'test\u00e4\u00f6\u00fc\u00df', 'test\u00e9\u00e8',\n 'test\u00b0\u00b3\u00b2')\n path = TESTFN\n\n def setUp(self):\n os.mkdir(self.path)\n self.syspath = sys.path[:]\n\n def tearDown(self):\n rmtree(self.path)\n sys.path[:] = self.syspath\n\n # Regression test for http://bugs.python.org/issue1293.\n def test_trailing_slash(self):\n with open(os.path.join(self.path, 'test_trailing_slash.py'), 'w') as f:\n f.write(\"testdata = 'test_trailing_slash'\")\n sys.path.append(self.path+'/')\n mod = import(\"test_trailing_slash\")\n self.assertEqual(mod.testdata, 'test_trailing_slash')\n unload(\"test_trailing_slash\")\n\n # Regression test for http://bugs.python.org/issue3677.\n def _test_UNC_path(self):\n with open(os.path.join(self.path, 'test_trailing_slash.py'), 'w') as f:\n f.write(\"testdata = 'test_trailing_slash'\")\n # Create the UNC path, like \\myhost\c$\foo\bar.\n path = os.path.abspath(self.path)\n import socket\n hn = socket.gethostname()\n drive = path[0]\n unc = \"\\\\%s\\%s$\"%(hn, drive)\n unc += path[2:]\n sys.path.append(path)\n mod = import(\"test_trailing_slash\")\n self.assertEqual(mod.testdata, 'test_trailing_slash')\n unload(\"test_trailing_slash\")\n\n if sys.platform == \"win32\":\n test_UNC_path = _test_UNC_path\n\n\nclass RelativeImportTests(unittest.TestCase):\n\n def tearDown(self):\n unload(\"test.relimport\")\n setUp = tearDown\n\n def test_relimport_star(self):\n # This will import from .test_import.\n from . import relimport\n self.assertTrue(hasattr(relimport, \"RelativeImportTests\"))\n\n def test_issue3221(self):\n # Note for mergers: the 'absolute' tests from the 2.x branch\n # are missing in Py3k because implicit relative imports are\n # a thing of the past\n #\n # Regression test for htt", 4096) = 4096 read(3, "p://bugs.python.org/issue3221.\n def check_relative():\n exec(\"from . import relimport\", ns)\n\n # Check relative import OK with package and name correct\n ns = dict(package='test', name='test.notarealmodule')\n check_relative()\n\n # Check relative import OK with only name wrong\n ns = dict(package='test', name='notarealpkg.notarealmodule')\n check_relative()\n\n # Check relative import fails with only package wrong\n ns = dict(package='foo', name='test.notarealmodule')\n self.assertRaises(SystemError, check_relative)\n\n # Check relative import fails with package and name wrong\n ns = dict(package='foo', name='notarealpkg.notarealmodule')\n self.assertRaises(SystemError, check_relative)\n\n # Check relative import fails with package set to a non-string\n ns = dict(package=object())\n self.assertRaises(ValueError, check_relative)\n\n def test_absolute_import_without_future(self):\n # If explicit relative import syntax is used, then do not try\n # to perform an absolute import in the face of failure.\n # Issue bpo-7902.\n with self.assertRaises(ImportError):\n from .os import sep\n self.fail(\"explicit relative import triggered an \"\n \"implicit absolute import\")\n\n\nclass OverridingImportBuiltinTests(unittest.TestCase):\n def test_override_builtin(self):\n # Test that overriding builtins.import can bypass sys.modules.\n import os\n\n def foo():\n import os\n return os\n self.assertEqual(foo(), os) # Quick sanity check.\n\n with swap_attr(builtins, \"import\", lambda x: 5):\n self.assertEqual(foo(), 5)\n\n # Test what happens when we shadow import in globals(); this\n # currently does not impact the import process, but if this changes,\n # other code will need to change, so keep this test as a tripwire.\n with swap_item(globals(), \"import\", lambda *x: 5):\n self.assertEqual(foo(), os)\n\n\nclass PycacheTests(unittest.TestCase):\n # Test the various PEP-3147 related behaviors.\n\n tag = imp.get_tag()\n\n def _clean(self):\n forget(TESTFN)\n rmtree('pycache')\n unlink(self.source)\n\n def setUp(self):\n self.source = TESTFN + '.py'\n self._clean()\n with open(self.source, 'w') as fp:\n print('# This is a test file written by test_import.py', file=fp)\n sys.path.insert(0, os.curdir)\n\n def tearDown(self):\n assert sys.path[0] == os.curdir, 'Unexpected sys.path[0]'\n del sys.path[0]\n self._clean()\n\n def test_import_pyc_path(self):\n self.assertFalse(os.path.exists('pycache'))\n import(TESTFN)\n self.assertTrue(os.path.exists('pycache'))\n self.assertTrue(os.path.exists(os.path.join(\n 'pycache', '{}.{}.py{}'.format(\n TESTFN, self.tag, debug and 'c' or 'o'))))\n\n @unittest.skipUnless(os.name == 'posix',\n \"test meaningful only on posix systems\")\n def test_unwritable_directory(self):\n # When the umask causes the new pycache directory to be\n # unwritable, the import still succeeds but no .pyc file is written.\n with temp_umask(0o222):\n import(TESTFN)\n self.assertTrue(os.path.exists('pycache'))\n self.assertFalse(os.path.exists(os.path.join(\n 'pycache', '{}.{}.pyc'.format(TESTFN, self.tag))))\n\n def test_missing_source(self):\n # With PEP-3147 cache layout, removing the source but leaving the pyc\n # file does not satisfy the import.\n import(TESTFN)\n pyc_file = imp.cache_from_source(self.source)\n self.assertTrue(os.path.exists(pyc_file))\n os.remove(self.source)\n forget(TESTFN)\n self.assertRaises(ImportError, import, TESTFN)\n\n def test_missing_source_legacy(self):\n # Like test_missing_source() ", 4096) = 4096 read(3, "except that for backward compatibility,\n # when the pyc file lives where the py file would have been (and named\n # without the tag), it is importable. The file of the imported\n # module is the pyc location.\n import(TESTFN)\n # pyc_file gets removed in _clean() via tearDown().\n pyc_file = make_legacy_pyc(self.source)\n os.remove(self.source)\n unload(TESTFN)\n m = import(TESTFN)\n self.assertEqual(m.file,\n os.path.join(os.curdir, os.path.relpath(pyc_file)))\n\n def test_cached(self):\n # Modules now also have an cached that points to the pyc file.\n m = import__(TESTFN)\n pyc_file = imp.cache_from_source(TESTFN + '.py')\n self.assertEqual(m.cached, os.path.join(os.curdir, pycfile))\n\n def testcached_legacy_pyc(self):\n # Like test_cached() except that for backward compatibility,\n # when the pyc file lives where the py file would have been (and named\n # without the tag), it is importable. The cached of the imported\n # module is the pyc location.\n import(TESTFN)\n # pyc_file gets removed in _clean() via tearDown().\n pyc_file = make_legacy_pyc(self.source)\n os.remove(self.source)\n unload(TESTFN)\n m = import(TESTFN)\n self.assertEqual(m.cached,\n os.path.join(os.curdir, os.path.relpath(pyc_file)))\n\n def test_packagecached(self):\n # Like testcached but for packages.\n def cleanup():\n rmtree('PEP-3147')\n os.mkdir('PEP-3147')\n self.addCleanup(cleanup)\n # Touch the init.py\n with open(os.path.join('PEP-3147', 'init.py'), 'w'):\n pass\n with open(os.path.join('PEP-3147', 'foo.py'), 'w'):\n pass\n unload('PEP-3147.foo')\n unload('PEP-3147')\n m = import('PEP-3147.foo')\n init_pyc = imp.cache_from_source(\n os.path.join('PEP-3147', 'init.py'))\n self.assertEqual(m.cached, os.path.join(os.curdir, init_pyc))\n foo_pyc = imp.cache_from_source(os.path.join('PEP-3147', 'foo.py'))\n self.assertEqual(sys.modules['PEP-3147.foo'].cached,\n os.path.join(os.curdir, foo_pyc))\n\n def testpackagecached_from_pyc(self):\n # Like test_cached but ensuring cached when imported from a\n # PEP-3147 pyc file.\n def cleanup():\n rmtree('PEP-3147')\n os.mkdir('PEP-3147')\n self.addCleanup(cleanup)\n unload('PEP-3147.foo')\n unload('PEP-3147')\n # Touch the init.py\n with open(os.path.join('PEP-3147', 'init.py'), 'w'):\n pass\n with open(os.path.join('PEP-3147', 'foo.py'), 'w'):\n pass\n m = import('PEP-3147.foo')\n unload('PEP-3147.foo')\n unload('PEP-3147')\n m = import('PEP-3147.foo')\n init_pyc = imp.cache_from_source(\n os.path.join('PEP-3147', 'init.py'))\n self.assertEqual(m.cached, os.path.join(os.curdir, init_pyc))\n foo_pyc = imp.cache_from_source(os.path.join('PEP-3147', 'foo.py'))\n self.assertEqual(sys.modules['PEP-3147.foo'].cached,\n os.path.join(os.curdir, foo_pyc))\n\n\nclass RelativeImportFromImportlibTests(test_relative_imports.RelativeImports):\n\n def setUp(self):\n self._importlib_util_flag = importlibutil.usingimport\n importlibutil.usingimport = True\n\n def tearDown(self):\n importlibutil.usingimport__ = self._importlib_util_flag\n\n\ndef test_main(verbose=None):\n run_unittest(ImportTests, PycacheTests,\n PycRewritingTests, PathsTests, RelativeImportTests,\n OverridingImportBuiltinTests,\n RelativeImportFromImportlibTests)\n\n\nif name == 'main':\n # Test needs to be a package, so we can do relative imports.\n from test.test_import import test_main\n test_main()\n", 4096) = 4052 read(3, "", 4096) = 0 close(3) = 0 write(1, "FAIL\ntest_basics (test.test_import.PycRewritingTests) ... ", 58FAIL

    dff06f57-790c-4585-b46f-809245342eab commented 13 years ago

    So, in the statement that fails :

    self.assertFalse(os.path.exists(os.path.join(...))) .

    either self.assertFalse is failing or os.path.exists is failing or os.path.join is failing.

    The fact that the error message is 'AssertionError: True is not false' suggests that os.path.exists is returning True .

    So how can this be if we see only 'ENOENT' errors for every file that is a target of unlink() between the umask() and the write of "FAIL\n..." ?

    I think that this suggests that os.path.exists is failing for this build.

    dff06f57-790c-4585-b46f-809245342eab commented 13 years ago

    oops, no sorry it was this bit from the strace log :

    umask(0222) = 022 stat("./@test_9634_tmp", 0x7fff7ef64130) = -1 ENOENT (No such file or directory) open("./@test_9634_tmp.cpython-33m.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmpmodule.cpython-33m.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmp.abi3.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmpmodule.abi3.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmp.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmpmodule.so", O_RDONLY) = -1 ENOENT (No such file or directory) open("./@test_9634_tmp.py", O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=48, ...}) = 0 open("./pycache/@test_9634_tmp.cpython-32.pyc", O_RDONLY) = -1 ENOENT (No such file or directory) fstat(3, {st_mode=S_IFREG|0644, st_size=48, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd261c4a000 read(3, "# This is a test file written by test_import.py\n", 4096) = 48 read(3, "", 4096) = 0 mkdir("./pycache", 0100777) = 0 unlink("./pycache/@test_9634_tmp.cpython-32.pyc") = -1 ENOENT (No such file or directory) open("./pycache/@test_9634_tmp.cpython-32.pyc", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0100644) = 4 fcntl(4, F_GETFL) = 0x8001 (flags O_WRONLY|O_LARGEFILE) fstat(4, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd261c49000 lseek(4, 0, SEEK_CUR) = 0 write(4, "l\f\r\n\0\0\0\0c\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0@\0\0\0s\4\0\0\0d\0\0S(\1\0\0\0N(\0\0\0\0(\0\0\0\0(\0\0\0\0(\0\0\0\0u\23\0\0\0./@test_9634_tmp.pyu\10\0\0\0\<module>\1\0\0\0s\0\0\0\0", 110) = 110 lseek(4, 4, SEEKSET) = 4 write(4, "?\342\272M", 4) = 4 close(4) = 0 munmap(0x7fd261c49000, 4096) = 0 close(3) = 0 munmap(0x7fd261c4a000, 4096) = 0 umask(022) = 0222 stat("\_pycache", {st_mode=S_IFDIR|S_ISGID|0555, stsize=4096, ...}) = 0 stat("\_pycache/@test_9634_tmp.cpython-32.pyc", {st_mode=S_IFREG|0444, st_size=110, ...}) = 0

    So the test succeeds ; ie. is correctly detecting being able to create a file in a directory with umask 0222 ; I think if you are the super-user (root) - and I am - w bits on directories owned by root are ignored - double-checking this now.

    So I think the fix is simply to say : if I am root user, skip this test, because it will always fail.

    dff06f57-790c-4585-b46f-809245342eab commented 13 years ago

    Aha ! Yes, the test DOES succeed as a non-root user , and yes, if you are super user you can override any write bits in directory permissions if you own the directory.

    So the fix ? : skip 'unwritable_directory' test if you are root - here's the patch

    merwok commented 13 years ago

    It seems strange to build and test Python as root.

    merwok commented 13 years ago

    Disregard my remark; David in msg134804 expressed support for this, and given that there are Python programs run as root for sysadmin tasks, it makes sense to make sure the code is tested when run by root.

    1762cc99-3127-4a62-9baf-30c3d0f51ef7 commented 13 years ago

    New changeset 7697223df6df by Charles-François Natali in branch '3.2': Issue bpo-11956: Skip test_import.test_unwritable_directory on FreeBSD when run as http://hg.python.org/cpython/rev/7697223df6df

    New changeset 58870fe9a604 by Charles-François Natali in branch 'default': Issue bpo-11956: Skip test_import.test_unwritable_directory on FreeBSD when run as http://hg.python.org/cpython/rev/58870fe9a604

    1762cc99-3127-4a62-9baf-30c3d0f51ef7 commented 13 years ago

    New changeset cbda512c6d7f by Charles-François Natali in branch '3.2': Issue bpo-11956: Always skip test_import.test_unwritable_directory when run as http://hg.python.org/cpython/rev/cbda512c6d7f

    New changeset 971093a75613 by Charles-François Natali in branch 'default': Issue bpo-11956: Always skip test_import.test_unwritable_directory when run as http://hg.python.org/cpython/rev/971093a75613