search

tenstorrent / tt-umd

User-Mode Driver for Tenstorrent hardware
Apache License 2.0
9 stars 5 forks source link

reading/writing more than 1KB of data from second chip on N300 fails #77

Open tt-vjovanovic opened 1 month ago

tt-vjovanovic commented 1 month ago

We are using UMD in tt-debuda and having trouble when reading/writing more than 1KB of data from second chp on N300. Either our initialization of UMD is bad or there is a bug in UMD. In function tt_SiliconDevice::read_from_non_mmio_device there is condition use_dram = size_in_bytes > 1024;... We are using tt_SiliconDevice::read_from_device and tt_SiliconDevice::write_to_device...

broskoTT commented 1 month ago

A good candidate to write a test for.

Not urgent since there is a workaround of splitting the data into chunks up to 1KB.

joelsmithTT commented 1 month ago

@tt-vjovanovic can you share a git commit hash of your repo that demonstrates the problem?

broskoTT commented 1 month ago

Metal has this defined const uint32_t MAX_BLOCK_SIZE = (0x1 << 10); // Max 1024 bytes

tt-vjovanovic commented 1 month ago

@tt-vjovanovic can you share a git commit hash of your repo that demonstrates the problem?

3bff9d422e35b8ff2e28f99d6ec63eb31d00cf1e third_party/umd

tt-vjovanovic commented 1 month ago

I'm not saying this commit caused the problem, but that is where our submodule points to :)

broskoTT commented 2 weeks ago

Related, to be fixed after this issue is resolved: https://github.com/tenstorrent/tt-debuda/issues/124