harvardnlp / namedtensor

Named Tensor implementation for Torch
http://nlp.seas.harvard.edu/NamedTensor
MIT License
443 stars 42 forks source link

Feature Request: Tensor Abbreviations #103

Open EndingCredits opened 5 years ago

EndingCredits commented 5 years ago

This is actually more of a discussion thread since I have an implementation here, but I wanted to discuss the general idea before submitting a PR.

Essentially, I propose that we allow to access a tensor dimension by a one-character abbreviation as well as its name. This would make working with namedtensors a lot less verbose, however this might go against the general principles of namedtensor as ensuring greater safety.

Example: Given tensor t = ntorch.ones(2, 3, 5, names=('batch', 'width', 'height')) then we can refer to these dimensions just by their first letter sot[{'b': 0}] is equivalent to t[{'batch': 0}]

If there's ambiguity in abbreviations, we allow creation of the tensor:

t = ntorch.ones(2, 3, 5, names=('axis1', 'axis2', 'axis3')) #valid

But we throw an ambiguity error if we try to access in an ambiguous fashion:

t[{'axis1': 0}] #valid
t[{'a': 0}] #invalid, error thrown

To enable abbreviations to still be used with ambiguous names, we allow setting the abbreviation manually with a:name:

t = ntorch.ones(2, 3, 5, names=('axis1', 'axis2', 'c:axis3')) #valid
t[{'axis1': 0}] #valid
t[{'a': 0}] #invalid
t[{'c': 0}] #valid
t = ntorch.ones(2, 3, 5, names=('axis1', 'b:axis2', 'c:axis3')) #valid
t[{'a': 0}] # now valid as only one axis has abbreviation 'a'

Unfortunately this required a lot of code rewriting and adds a little bit more complexity,although the result is arguably more elegant, see the comparison. However I believe this is worthwhile, especially as it makes it easier to add in other features such as dimension labels.

srush commented 5 years ago

This is neat, but I don't think I will add it. I think this type of feature should be done with variables and not shorter strings.

EndingCredits commented 5 years ago

Do you want me to do a PR anyway, but with the abbreviations removed? It should be transparent above the schema level.