search

snakemake / snakemake

This is the development home of the workflow management system Snakemake. For general information, see
https://snakemake.github.io
MIT License
2.25k stars 547 forks source link

snakemake.remote.GS glob_wildcards doesn't seem to work #256

Open tmbdev opened 4 years ago

tmbdev commented 4 years ago

Snakemake version 5.10.0

Describe the bug

snakemake.remote.GS.RemoteProvider doesn't seem to expand glob_wildcards

Minimal example

from snakemake.remote.GS import RemoteProvider as GSRemoteProvider
bucket = "some-bucket-you-can-write"
os.system(f"for i in 1 2 3; do echo $i | gsutil cp - gs://{bucket}/$i.test; done")
GS = GSRemoteProvider(stay_on_remote=True)
IDS, = GS.glob_wildcards(bucket+"/{id}.text")
print(IDS)
IDS, = GS.glob_wildcards("gs://"+bucket+"/{id}.text")
print(IDS)

This should print three ids, but it pritns an empty list.

Additional context

If I generate the list of ids manually in my jobs, they run, so file access itself works.

dcroote commented 4 years ago

You have a typo- the files you are uploading with gsutil are .test while the glob patterns are .text. Fixing that still returned empty lists for me until after stay_on_remote=True was removed from GSRemoteProvider(), then both were successful.