riga / law

Build large-scale task workflows: luigi + job submission + remote targets + environment sandboxing using Docker/Singularity
http://law.readthedocs.io
BSD 3-Clause "New" or "Revised" License
98 stars 41 forks source link

[target] add optional file size information to representation #114

Closed pfackeldey closed 2 years ago

pfackeldey commented 2 years ago

Hi @riga ,

This small PR adds the possibility to display the file size for any FileSystemTarget-type target class.

This looks as follows:

In [1]: import law

In [2]: f = law.LocalFileTarget("MANIFEST.in")

# standard repr
In [3]: f.__repr__()
Out[3]: 'LocalFileTarget(path=/Users/peter/repos/law/MANIFEST.in)'

In [4]: law.Config.instance().set("target", "filesize_repr", "True")

# repr also including file size
In [5]: f.__repr__()
Out[5]: 'LocalFileTarget(path=/Users/peter/repos/law/MANIFEST.in, 156.0 bytes)'

This is not (yet) implemented for collections as this might be expensive information to collect, especially for many remote targets.

Best, Peter

riga commented 2 years ago

Nice! 🎉

I added the handling of cases where the file does not exist (+ reducing requests for remote targets), and also made it a named pair so in your example it would be

LocalFileTarget(path=/Users/peter/repos/law/MANIFEST.in, size=156.0bytes)

Fyi, one can also use law.config.set("target", "filesize_repr", "True") without the need to get the singleton first :)