add new dataloaders - Githubissues

openproblems-bio / openproblems

Formalizing and benchmarking open problems in single-cell genomics

MIT License

314 stars 78 forks source link

add new dataloaders #792

Closed danielStrobl closed 1 year ago

danielStrobl commented 1 year ago

Submission type

[x] This submission adds a new dataset
[ ] This submission adds a new method
[ ] This submission adds a new metric
[ ] This submission adds a new task
[ ] This submission adds a new Docker image
[ ] This submission fixes a bug (link to related issue: )
[ ] This submission adds a new feature not listed above

Testing

[x] This submission was written on a forked copy of openproblems
[ ] Nextflow test pipeline is passing on this base branch of this pull request (include link to passed test on NF Tower found in GitHub Actions summary: )
[x] If this pull request is not ready for review (including passing the Nextflow test pipeline), I will open this PR as a draft (click on the down arrow next to the "Create Pull Request" button)

Submission guidelines

[ ] This submission follows the guidelines in our Contributing document
[x] I have checked to ensure there aren't other open Pull Requests for the same update/change

PR review checklist

This PR will be evaluated on the basis of the following checks:

[ ] The task addresses a valid open problem in single-cell analysis
[ ] The latest version of master is merged and tested
[ ] The methods/metrics are imported to __init__.py and were tested in the pipeline
[ ] Method and metric decorators are annotated with paper title, year, author, code version, and date
[ ] The README gives an outline of the methods, metrics and datasets in the folder
[ ] The README provides a satisfactory task explanation (for new tasks)
[ ] The sample test data is appropriate to test implementation of all methods and metrics (for new tasks)

scottgigante-immunai commented 1 year ago

Not yet passing tests, converting back to draft

scottgigante-immunai commented 1 year ago

Tests passing: https://tower.nf/orgs/openproblems-bio/workspaces/openproblems-bio/watch/247vkt93iKumfu

LuckyMD commented 1 year ago

@danielStrobl why is it called "Lung (by batch)"? And still missing a note for the other data loader that batch variable already correctly named...

github-actions[bot] commented 1 year ago

danielStrobl commented 1 year ago

That was from another dataloader that I copied, this is already changed though with the most recent commit, same for the comment for the batch column

danielStrobl commented 1 year ago

@scottgigante-immunai scprep can't download files >3 GB, I think this could be fixed by updating urllib3>=1.26.11, see here

scottgigante-immunai commented 1 year ago

Working on this now

scottgigante-immunai commented 1 year ago

@danielStrobl doesn't look like that worked?

danielStrobl commented 1 year ago

still the same error. do you think it makes sense to split up this PR and leave out the immune cell human mouse dataset for now?

scottgigante-immunai commented 1 year ago

Works for me

scottgigante-immunai commented 1 year ago

Tests at https://tower.nf/orgs/openproblems-bio/workspaces/openproblems-bio/watch/2NnhA8I7V2Xqbi