search

rstojnic / lazydata

Lazydata: Scalable data dependencies for Python projects
Apache License 2.0
624 stars 23 forks source link

Added New remote component: local storage #12

Open lastmeta opened 6 years ago

lastmeta commented 6 years ago

added local storage option for linux and windows for use, most often, in backing data to a mounted shared drive. Also added purge command that deletes the lazydata.yml and the .aws and .lazydata.

rstojnic commented 6 years ago

Thanks very much for this PR - this is amazing work!

I would like to suggest three things:

  1. Would you mind renaming the purge command into something a bit more drastic, like remove-all-tracking or something like that. I would like to reserve the purge command for something less drastic eg. purge only the local cache in ~/.lazydata for those files that are already stored remotely.
  2. Can you resolve the conflict with the current master?
  3. Would it be possible to add some tests for this new local storage? I hope this shouldn't be too much work - probably just adding a couple of lines to tests/test_local_project.py.
lastmeta commented 6 years ago

Thanks very much for this PR - this is amazing work!

I would like to suggest three things:

  1. Would you mind renaming the purge command into something a bit more drastic, like remove-all-tracking or something like that. I would like to reserve the purge command for something less drastic eg. purge only the local cache in ~/.lazydata for those files that are already stored remotely.
  2. Can you resolve the conflict with the current master?
  3. Would it be possible to add some tests for this new local storage? I hope this shouldn't be too much work - probably just adding a couple of lines to tests/test_local_project.py.

Yes, I'll try to get to that, but it might not be soon. turns out the use case I created all this code for isn't actually going to materialize, so I'm working on other stuff now. But yes, I think your suggestions are a good idea.

Avsecz commented 6 years ago

the new track command is exactly what I specified in https://github.com/rstojnic/lazydata/issues/15. So the issue can be closed. Is it possible to make it work with multiple files? This would allow to use bash expansion: lazydata track *.ext

rstojnic commented 6 years ago

Continuing work in the PR12 branch