search

NVlabs / timeloop

Timeloop performs modeling, mapping and code-generation for tensor algebra workloads on various accelerator architectures.
https://timeloop.csail.mit.edu/
BSD 3-Clause "New" or "Revised" License
303 stars 99 forks source link

Query on using Timeloop and Accelergy #240

Open Yukiona1996 opened 4 months ago

Yukiona1996 commented 4 months ago

Hi,

Just wanted to check in if there's a way for us to load the weight of pre-trained models onto one of the yaml files and use them to perform the necessary analysis on the architecture and mapping?

If yes, could you please redirect me to the relevant documents?

Thanks!

angshuman-parashar commented 4 months ago

Are you trying to model a dense or sparse (compressed) configuration? If it's dense then the model weights are not relevant because they do not change the behavior of the workload on the hardware -- at least not at the level of fidelity at which Timeloop works. All that Timeloop needs to know are the dimensions.

Yukiona1996 commented 4 months ago

How would it be with respect to a sparse architecture?

angshuman-parashar commented 4 months ago

@tanner-andrulis Do you know anything about the status of the real-data density model?

tanner-andrulis commented 4 months ago

That's out of my expertise... pinging Fisher. It's him or Nellie

Yukiona1996 commented 4 months ago

Hi, Do we have any updates on this?

Yukiona1996 commented 4 months ago

@angshuman-parashar Could you please confirm if is there a way for us interface a routing and data compression logic in say a python to timeloop? I was considering having the algorithm output spare data, and feeling that to our timeloop model or mapper.

angshuman-parashar commented 4 months ago

Real data density model is a WIP feature and is not exposed yet.

I am not sure what you mean by "routing and data compression logic". If this is an different question then I suggest creating a new issue for it.

Yukiona1996 commented 4 months ago

By that I meant, writting an algorithm to sparsify the data in python externally and then just pass the data alone as input to the dense architecture. Would I be allowed to interface this kind of a logic into Timeloop?

Yukiona1996 commented 4 months ago

Hi @angshuman-parashar , do you have any inputs on this?

angshuman-parashar commented 4 months ago

Dense architectures do not care about data. The model's output only depends on the tensor shapes.

Instead, if you want Timeloop (Sparseloop) to model an architecture that takes advantage of the sparsity to improve performance or energy efficiency, then you will have to extract statistical properties from your sparsified data and represent the distribution using one of the supported density models (e.g., hypergeometric distribution).

Yukiona1996 commented 4 months ago

Thank you, that helps!