mosaicml / streaming

A Data Streaming Library for Efficient Neural Network Training
https://streaming.docs.mosaicml.com
Apache License 2.0
1.09k stars 136 forks source link

fix: expand user path for Writer's output directory. #694

Closed huxuan closed 3 months ago

huxuan commented 3 months ago

Description of changes:

When setting the out argument of Writer (e.g. MDSWriter) to something like ~/awesome-streamdataset, it will create a directory in current workdir named with "~" which is quite strange.

Issue #, if available:

Merge Checklist:

Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

Tests

BTW, it seems that pre-commit run -a and pytest -vv -s . does not work well on the latest main branch.

snarayan21 commented 3 months ago

BTW, it seems that pre-commit run -a and pytest -vv -s . does not work well on the latest main branch.

@huxuan Mind elaborating on what went wrong here?