Manifest upload fails - Githubissues

Description

I encountered an issue while trying to use push_to_git in create_manifest.

Error Details

Command Executed:


from impresso_essentials.versioning.compute_manifest import create_manifest

config_dict_entities = {
    "data_stage": "entities",
    "output_bucket": "42-processed-data-final/entities/entities-bert-historic-multilingual_v1-0-3",
    "input_bucket": "22-rebuilt-final",
    "git_repository": "../impresso-semantic-enrichment-deployment/",
    "temp_directory": "../impresso-semantic-enrichment-deployment/temp",
    "previous_mft_s3_path": "",
    "is_staging": True,
    "is_patch": False,
    "patched_fields": [],
    "push_to_git": True,
    "file_extensions": "jsonl.bz2",
    "log_file": "/local/path/to/log_file.log",
    "notes": """First NER/EL models: 2024-02-01, 
        - NER: stacked-2-bert-medium-historic-multilingual-v3-base (https://github.com/impresso/impresso-named-entity-recognition/tree/main/stacked-ner)
        - EL: mGenre finetuned on (all with Qids) HIPE data (https://github.com/impresso/impresso-named-entity-linking/tree/main/genre)
        Patch with names, functions, and titles + removal of duplicates: 2024-08-01,
    """,
}

create_manifest(config_dict_entities)

Output

Error while cloning the git repository impresso/impresso-data-release using ssh, trying with https. 
Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --branch=staging -- git@github.com:impresso/impresso-data-release.git ../impresso-semantic-enrichment-deployment/temp/impresso-data-release
  stderr: 'Cloning into '../impresso-semantic-enrichment-deployment/temp/impresso-data-release'...
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
'
Username for 'https://github.com': emanuelaboros
Password for 'https://emanuelaboros@github.com': 
Error while cloning the git repository impresso/impresso-data-release, it was not possible to clone it with ssh or https. 
Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --branch=staging -- https://github.com/impresso/impresso-data-release.git ../impresso-semantic-enrichment-deployment/temp/impresso-data-release
  stderr: 'Cloning into '../impresso-semantic-enrichment-deployment/temp/impresso-data-release'...
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/impresso/impresso-data-release.git/'
'
Traceback (most recent call last):
  File "/home/eboros/scratch/eboros/impresso-semantic-enrichment-deployment/generate_manifest.py", line 63, in <module>
    create_manifest(config_dict_entities)
  File "/home/eboros/.local/lib/python3.11/site-packages/impresso_essentials/versioning/compute_manifest.py", line 306, in create_manifest
    manifest.compute(export_to_git_and_s3=True)
  File "/home/eboros/.local/lib/python3.11/site-packages/impresso_essentials/versioning/data_manifest.py", line 1098, in compute
    success = self.validate_and_export_manifest(push, commit_msg)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eboros/.local/lib/python3.11/site-packages/impresso_essentials/versioning/data_manifest.py", line 402, in validate_and_export_manifest
    out_repo = clone_git_repo(self.temp_dir, branch=self.branch)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eboros/.local/lib/python3.11/site-packages/impresso_essentials/versioning/helpers.py", line 469, in clone_git_repo
    raise e
  File "/home/eboros/.local/lib/python3.11/site-packages/impresso_essentials/versioning/helpers.py", line 461, in clone_git_repo
    return git.Repo.clone_from(repo_https_url, repo_path, branch=branch)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eboros/.conda/envs/myenv/lib/python3.11/site-packages/git/repo/base.py", line 1525, in clone_from
    return cls._clone(
           ^^^^^^^^^^^
  File "/home/eboros/.conda/envs/myenv/lib/python3.11/site-packages/git/repo/base.py", line 1396, in _clone
    finalize_process(proc, stderr=stderr)
  File "/home/eboros/.conda/envs/myenv/lib/python3.11/site-packages/git/util.py", line 504, in finalize_process
    proc.wait(**kwargs)
  File "/home/eboros/.conda/envs/myenv/lib/python3.11/site-packages/git/cmd.py", line 834, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --branch=staging -- https://github.com/impresso/impresso-data-release.git ../impresso-semantic-enrichment-deployment/temp/impresso-data-release
  stderr: 'Cloning into '../impresso-semantic-enrichment-deployment/temp/impresso-data-release'...
remote: Support for password authentication was removed on August 13, 2021.
remote: Please see https://docs.github.com/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls for information on currently recommended modes of authentication.
fatal: Authentication failed for 'https://github.com/impresso/impresso-data-release.git/'

impresso / impresso-essentials

Manifest upload fails #6

Description

Error Details

Output