search

tbepler / topaz

Pipeline for particle picking in cryo-electron microscopy images using convolutional neural networks trained from positive and unlabeled examples. Also featuring micrograph and tomogram denoising with DNNs.
GNU General Public License v3.0
170 stars 62 forks source link

Git checkout errors #143

Closed bu-bgregor closed 2 years ago

bu-bgregor commented 2 years ago

I am getting errors when running git clone https://github.com/tbepler/topaz : 

Cloning into 'topaz'...
remote: Enumerating objects: 1698, done.
remote: Counting objects: 100% (450/450), done.
remote: Compressing objects: 100% (335/335), done.
remote: Total 1698 (delta 182), reused 263 (delta 77), pack-reused 1248
Receiving objects: 100% (1698/1698), 110.32 MiB | 42.17 MiB/s, done.
Resolving deltas: 100% (1010/1010), done.
Checking out files: 100% (217/217), done.
Downloading test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00004hl_00004es_c.mrc (228 MB)
Error downloading object: test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00004hl_00004es_c.mrc (b62b164): Smudge error: Error downloading test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00004hl_00004es_c.mrc (b62b164287625f96ae58fa2614be6db8f260cb69f7d0d86c139c24f2ce3f9946): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.

Errors logged to /share/pkg.7/topaz/0.2.5/src/topaz/.git/lfs/logs/20220329T114107.633975757.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00004hl_00004es_c.mrc: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

Running git status in the freshly checked out directory:

On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    deleted:    .dockerignore
    deleted:    .gitattributes
    deleted:    .github/actions/Dockerfile
    deleted:    .github/actions/LICENSE
    deleted:    .github/actions/README.md
    deleted:    .github/actions/action.yml
    deleted:    .github/actions/entrypoint.sh
    deleted:    .github/workflows/ci.yml
    deleted:    .github/workflows/publish_conda.yml
    deleted:    .github/workflows/publish_pypi.yml
    deleted:    .gitignore
    deleted:    .readthedocs.yaml
    deleted:    Dockerfile
    deleted:    LICENSE
    deleted:    MANIFEST.in
    deleted:    README.md
    deleted:    Singularity
    deleted:    conda-build/conda_build_config.yml
    deleted:    conda-build/conda_publish_env.yaml
    deleted:    conda-build/meta-v0.0.0.yaml
    deleted:    conda-build/meta-v0.1.0.yaml
    deleted:    conda-build/meta-v0.2.0.yaml
    deleted:    conda-build/meta-v0.2.1.yaml
    deleted:    conda-build/meta-v0.2.2.yaml
    deleted:    conda-build/meta-v0.2.3.yaml
    deleted:    conda-build/meta-v0.2.4.yaml
    deleted:    conda-build/meta.yaml
    deleted:    docs/Makefile
    deleted:    docs/make.bat
    deleted:    docs/requirements.txt
    deleted:    docs/source/algorithms.md
    deleted:    docs/source/commands.md
    deleted:    docs/source/commands/convert.md
    deleted:    docs/source/commands/denoise.md
    deleted:    docs/source/commands/denoise3d.md
    deleted:    docs/source/commands/downsample.md
    deleted:    docs/source/commands/extract.md
    deleted:    docs/source/commands/file_utilities.md
    deleted:    docs/source/commands/gui.md
    deleted:    docs/source/commands/image_processing.md
    deleted:    docs/source/commands/normalize.md
    deleted:    docs/source/commands/particle_picking.md
    deleted:    docs/source/commands/particle_stack.md
    deleted:    docs/source/commands/precision_recall.md
    deleted:    docs/source/commands/preprocess.md
    deleted:    docs/source/commands/segment.md
    deleted:    docs/source/commands/split.md
    deleted:    docs/source/commands/train.md
    deleted:    docs/source/commands/train_test_split.md
    deleted:    docs/source/conf.py
    deleted:    docs/source/cryosparc.md
    deleted:    docs/source/denoising.md
    deleted:    docs/source/description.md
    deleted:    docs/source/index.md
    deleted:    docs/source/installation.md
    deleted:    docs/source/installation/conda_install.md
    deleted:    docs/source/installation/docker_install.md
    deleted:    docs/source/installation/pip_install.md
    deleted:    docs/source/installation/singularity.md
    deleted:    docs/source/installation/source_install.md
    deleted:    docs/source/methods.md
    deleted:    docs/source/metrics_stats.md
    deleted:    docs/source/models.md
    deleted:    docs/source/relion.md
    deleted:    docs/source/scripts.md
    deleted:    docs/source/transforms.md
    deleted:    docs/source/tutorial.md
    deleted:    docs/source/utils.md
    deleted:    images/anoble.png
    deleted:    images/tbepler.png
    deleted:    relion_run_topaz/LICENSE
    deleted:    relion_run_topaz/README.md
    deleted:    relion_run_topaz/run_topaz_denoise.py
    deleted:    relion_run_topaz/run_topaz_pick.py
    deleted:    relion_run_topaz/run_topaz_train.py
    deleted:    relion_run_topaz/run_topaz_train_denoise.py
    deleted:    requirements.txt
    deleted:    scripts/filter_images.py
    deleted:    scripts/generate_image_list.py
    deleted:    scripts/match_extracted_particles.py
    deleted:    scripts/subsample_particles.py
    deleted:    scripts/train_test_split.py
    deleted:    setup.py
    deleted:    test/data_utils/test_coordinates.py
    deleted:    test/data_utils/test_loader.py
    deleted:    test/data_utils/test_partition.py
    deleted:    test/data_utils/test_sampler.py
    deleted:    test/models/test__model_utils.py
    deleted:    test/models/test_basic.py
    deleted:    test/models/test_classifier.py
    deleted:    test/models/test_densenet.py
    deleted:    test/models/test_factory.py
    deleted:    test/models/test_generative.py
    deleted:    test/models/test_msnet.py
    deleted:    test/models/test_resnet.py
    deleted:    test/test_data/EMPIAR-10025/rawdata/ctf_params.star
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00004hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00005hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00007hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00014sq_00011hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00015sq_00015hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00018sq_00008hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00018sq_00010hl_00005es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00020sq_00011hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00003gr_00020sq_00011hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00031sq_00002hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00031sq_00005hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00031sq_00010hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00007hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00010hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00029hl_00005es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00031hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00033hl_00005es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00037hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00037hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00040hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00040hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00004gr_00032sq_00041hl_00005es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00004hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00005hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00006hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00008hl_00003es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00008hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00009hl_00002es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00009hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/micrographs/14sep05c_c_00007gr_00013sq_00014hl_00004es_c.mrc
    deleted:    test/test_data/EMPIAR-10025/rawdata/particles.star
    deleted:    test/test_data/EMPIAR-10025/rawdata/particles.txt
    deleted:    test/test_example.py
    deleted:    test/topaz/test_algorithms.py
    deleted:    test/topaz/test_denoise.py
    deleted:    test/topaz/test_main.py
    deleted:    test/topaz/test_methods.py
    deleted:    test/topaz/test_metrics.py
    deleted:    test/topaz/test_mrc.py
    deleted:    test/topaz/test_predict.py
    deleted:    test/topaz/test_stats.py
    deleted:    test/topaz_utils/test_conversions.py
    deleted:    test/topaz_utils/test_files.py
    deleted:    test/topaz_utils/test_image.py
    deleted:    test/topaz_utils/test_picks.py
    deleted:    test/topaz_utils/test_star.py
    deleted:    test/transforms/test_scaled_gaussian_mixture.py
    deleted:    topaz/__init__.py
    deleted:    topaz/_version.py
    deleted:    topaz/algorithms.py
    deleted:    topaz/commands/__init__.py
    deleted:    topaz/commands/boxes_to_coordinates.py
    deleted:    topaz/commands/convert.py
    deleted:    topaz/commands/coordinates_to_boxes.py
    deleted:    topaz/commands/coordinates_to_eman2_json.py
    deleted:    topaz/commands/coordinates_to_star.py
    deleted:    topaz/commands/denoise.py
    deleted:    topaz/commands/denoise3d.py
    deleted:    topaz/commands/downsample.py
    deleted:    topaz/commands/extract.py
    deleted:    topaz/commands/gui.py
    deleted:    topaz/commands/normalize.py
    deleted:    topaz/commands/particle_stack.py
    deleted:    topaz/commands/precision_recall_curve.py
    deleted:    topaz/commands/preprocess.py
    deleted:    topaz/commands/scale_coordinates.py
    deleted:    topaz/commands/segment.py
    deleted:    topaz/commands/split.py
    deleted:    topaz/commands/star_particles_threshold.py
    deleted:    topaz/commands/star_to_coordinates.py
    deleted:    topaz/commands/train.py
    deleted:    topaz/commands/train_test_split.py
    deleted:    topaz/cuda.py
    deleted:    topaz/denoise.py
    deleted:    topaz/gui/topaz.html
    deleted:    topaz/main.py
    deleted:    topaz/methods.py
    deleted:    topaz/metrics.py
    deleted:    topaz/model/__init__.py
    deleted:    topaz/model/classifier.py
    deleted:    topaz/model/factory.py
    deleted:    topaz/model/features/__init__.py
    deleted:    topaz/model/features/basic.py
    deleted:    topaz/model/features/densenet.py
    deleted:    topaz/model/features/msnet.py
    deleted:    topaz/model/features/resnet.py
    deleted:    topaz/model/generative.py
    deleted:    topaz/model/utils.py
    deleted:    topaz/mrc.py
    deleted:    topaz/predict.py
    deleted:    topaz/pretrained/denoise/affine_L1_v0.2.2.sav
    deleted:    topaz/pretrained/denoise/fcnn_L1_v0.2.2.sav
    deleted:    topaz/pretrained/denoise/unet-3d-10a-v0.2.4.sav
    deleted:    topaz/pretrained/denoise/unet-3d-20a-v0.2.4.sav
    deleted:    topaz/pretrained/denoise/unet_L2_v0.2.1.sav
    deleted:    topaz/pretrained/denoise/unet_L2_v0.2.2.sav
    deleted:    topaz/pretrained/denoise/unet_small_L1_v0.2.2.sav
    deleted:    topaz/pretrained/detector/resnet16_u32.sav
    deleted:    topaz/pretrained/detector/resnet16_u64.sav
    deleted:    topaz/pretrained/detector/resnet8_u32.sav
    deleted:    topaz/pretrained/detector/resnet8_u64.sav
    deleted:    topaz/stats.py
    deleted:    topaz/torch.py
    deleted:    topaz/transform/__init__.py
    deleted:    topaz/transform/scaled_gaussian_mixture.py
    deleted:    topaz/utils/__init__.py
    deleted:    topaz/utils/conversions.py
    deleted:    topaz/utils/data/__init__.py
    deleted:    topaz/utils/data/coordinates.py
    deleted:    topaz/utils/data/loader.py
    deleted:    topaz/utils/data/partition.py
    deleted:    topaz/utils/data/sampler.py
    deleted:    topaz/utils/files.py
    deleted:    topaz/utils/image.py
    deleted:    topaz/utils/picks.py
    deleted:    topaz/utils/printing.py
    deleted:    topaz/utils/star.py
    deleted:    tutorial/01_quick_start_guide.ipynb
    deleted:    tutorial/02_walkthrough.ipynb
    deleted:    tutorial/03_cross_validation.ipynb
    deleted:    tutorial/04_denoising.ipynb

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    .dockerignore
    .gitattributes
    .github/
    .gitignore
    .readthedocs.yaml
    Dockerfile
    LICENSE
    MANIFEST.in
    README.md
    Singularity
    conda-build/
    docs/
    images/
    relion_run_topaz/
    requirements.txt
    scripts/
    setup.py
    test/
    topaz/
    tutorial/
DarnellGranberry commented 2 years ago

Hi there. It looks like the issue is that Git LFS has run out of free bandwidth for downloading the micrographs I included for the test suite. I'll look into alternatives to it, but for now I'll most likely remove those files.

Did you try other installation methods, like using conda or pip? I would expect the same issue to arise, but they may be worth a try.

DarnellGranberry commented 2 years ago

I've removed those test micrographs because they aren't being used yet, and it seems like cloning the code to another machine worked normally. I'll close this for now, but feel free to reopen it if you have more issues.