Open andyk opened 2 years ago
Our windows tests started failing when the SB3 repo (the agentos freeze test uses) added a symlink (on June 22 2022) to their repo: https://github.com/mgbellemare/Arcade-Learning-Environment/blob/master/src/python/roms/md5.txt
agentos freeze
The file is getting created in my windows 11 filesystem but it is not a valid symlink
The error is in Dulwich's porcelain.py file: https://github.com/jelmer/dulwich/blob/bbf100b63fb00ebb4a1d0ca698b1abe337611bae/dulwich/porcelain.py#L230 in path_to_tree_path() in a call to Path.resolve() on the symlinked file.
path_to_tree_path()
Path.resolve()
The full failed test stack trace is:
def test_cli_freeze(cli_runner, tmpdir): run_args = ["sb3_agent", "-f"] run_kwargs = {"--registry-file": str(SB3_AGENT_DIR / "components.yaml")} > run_test_command( cli_runner, freeze, cli_args=run_args, cli_kwargs=run_kwargs ) C:\Users\andyk\Development\agentos\tests\test_core.py:57: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ C:\Users\andyk\Development\agentos\tests\utils.py:32: in run_test_command result = cli_runner.invoke( C:\Users\andyk\Development\agentos\tests\conftest.py:26: in wrapper result = f(*args, **kwargs) C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\testing.py:408: in invoke return_value = cli.main(args=args or (), prog_name=prog_name, **extra) C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:1062: in main rv = self.invoke(ctx) C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:1404: in invoke return ctx.invoke(self.callback, **ctx.params) C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\click\core.py:763: in invoke return __callback(*args, **kwargs) C:\Users\andyk\Development\agentos\agentos\cli.py:269: in freeze frozen_reg = module.freeze(force=force).to_registry() C:\Users\andyk\Development\agentos\pcs\instance_manager.py:46: in freeze find_and_replace_leaves( C:\Users\andyk\Development\agentos\pcs\utils.py:220: in find_and_replace_leaves leaf_replace(data_struct, leaf_list, replace_fn) C:\Users\andyk\Development\agentos\pcs\utils.py:156: in leaf_replace data_struct[leaf_list[0]] = replacement_fn(next_inner) C:\Users\andyk\Development\agentos\pcs\instance_manager.py:49: in <lambda> lambda x: x.freeze(force), C:\Users\andyk\Development\agentos\pcs\class_manager.py:86: in freeze self_copy.module = self.module.freeze(force) C:\Users\andyk\Development\agentos\pcs\module_manager.py:184: in freeze return self.to_versioned_module(force) C:\Users\andyk\Development\agentos\pcs\module_manager.py:156: in to_versioned_module repo_url, version = self.repo.get_version_from_git( C:\Users\andyk\Development\agentos\pcs\repo.py:73: in get_version_from_git full_path = self.get_local_file_path(file_path, version) C:\Users\andyk\Development\agentos\pcs\repo.py:123: in get_local_file_path local_repo_path = self.get_local_repo_dir(version) C:\Users\andyk\Development\agentos\pcs\repo.py:115: in get_local_repo_dir local_repo_path = self.GIT.clone_repo( C:\Users\andyk\Development\agentos\pcs\git_manager.py:295: in clone_repo self._clone_repo( C:\Users\andyk\Development\agentos\pcs\git_manager.py:317: in _clone_repo self._checkout_version(clone_destination, version) C:\Users\andyk\Development\agentos\pcs\git_manager.py:331: in _checkout_version self._checkout(repo=repo, target=treeish, force=True) C:\Users\andyk\Development\agentos\pcs\git_manager.py:473: in _checkout for file in porcelain.get_untracked_paths( C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\dulwich\porcelain.py:1283: in get_untracked_paths ip = path_to_tree_path(basepath, ap) C:\Users\andyk\Development\agentos\windows_dev\lib\site-packages\dulwich\porcelain.py:233: in path_to_tree_path resolved_path = path.resolve() C:\Python39\lib\pathlib.py:1204: in resolve s = self._flavour.resolve(self, strict=strict) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <pathlib._WindowsFlavour object at 0x0000018C2CDBFA30> path = WindowsPath('C:/Users/andyk/.agentos/cache/repos_cache/mgbellemare/Arcade-Learning-Environment/db3728264f382402120913d76c4fa0dc320ef59f/src/python/roms/md5.txt') strict = False def resolve(self, path, strict=False): s = str(path) if not s: return os.getcwd() previous_s = None if _getfinalpathname is not None: if strict: return self._ext_to_normal(_getfinalpathname(s)) else: tail_parts = [] # End of the path after the first one not found while True: try: > s = self._ext_to_normal(_getfinalpathname(s)) E OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'C:\\Users\\andyk\\.agentos\\cache\\repos_cache\\mgbellemare\\Arcade-Learning-Environment\\db3728264f382402120913d76c4fa0dc320ef59f\\src\\python\\roms\\md5.txt' C:\Python39\lib\pathlib.py:205: OSError
Our windows tests started failing when the SB3 repo (the
agentos freeze
test uses) added a symlink (on June 22 2022) to their repo: https://github.com/mgbellemare/Arcade-Learning-Environment/blob/master/src/python/roms/md5.txtThe file is getting created in my windows 11 filesystem but it is not a valid symlink![invalid_file_screenshot](https://user-images.githubusercontent.com/228998/176287021-55414b02-af1d-49c3-a359-bec57ce8e6dc.jpg)
The error is in Dulwich's porcelain.py file: https://github.com/jelmer/dulwich/blob/bbf100b63fb00ebb4a1d0ca698b1abe337611bae/dulwich/porcelain.py#L230 in
path_to_tree_path()
in a call toPath.resolve()
on the symlinked file.The full failed test stack trace is: