Closed snarayan21 closed 4 months ago
Can we please add a unit test for the same? :D
added unit test @karan6181
@karan6181 @XiaohanZhangCMU I was trying to fix the test that I wrote, but I realized there's also a test_existing_local_raises_exception in test_stream.py that tests this already. Are we good with merging the PR without me adding an additional test then?
@karan6181 @XiaohanZhangCMU I was trying to fix the test that I wrote, but I realized there's also a test_existing_local_raises_exception in test_stream.py that tests this already. Are we good with merging the PR without me adding an additional test then?
Yes, it's okay to merge since the unit test is already in place. To make CI pass, you might want to change the exception type from ValueError
to FileExistsError
in test_existing_local_raises_exception()
.
Description of changes:
Improves the local temp directory error when only a
remote
path is specified. We want to make sure that iflocal
is unspecified, we still create an entirely new temporary directory to hold the shard files, but if a user wants to reuse their locally cached dataset, they should explicitly specify thelocal
argument.Issue #, if available:
Merge Checklist:
Put an
x
without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
pre-commit
on my change. (check out thepre-commit
section of prerequisites)