tenstorrent / tt-metal

:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
Apache License 2.0
433 stars 62 forks source link

clean up active eth memory map and linker scripts #12979

Open pgkeller opened 3 weeks ago

pgkeller commented 3 weeks ago

Make it work like the other memory maps, remove unused portions Currently kernels that statically initialize globals are broken

Steps: 1) Remove dead portions of the l1_address_map 2) Define a magic base address where the metal memory map can begin 3) Move the metal required portions of the memory map into dev_mem_map following the naming conventions there 4) Unify linker scripts w/ tensix/idle eth linker scripts 5) Use linker script to place globals into local memory, add copy at FW and kernel init 6) Enable FD test TestEthInitLocalMemory

pgkeller commented 1 week ago

@ubcheema can you update this issue w/ the address ranges that are safe for metal to use?

ubcheema commented 1 week ago

On an Active ethernet core we run base firmware that performs two functions:

  1. ethernet link training and management
  2. Implement a custom Layer 3 routing network that enables host access to remote chips, which are only accessible over ethernet

Due to these two functions, some regions of 256 KB ethernet core L1 are already in use when the system boot up. Runtime firmware and kernel have to be loaded to regions that will not collide with or corrupt base firmware L1 regions.

Base FW: 0x0 - 0x903F: Firmware text and data sections. 0x11000 - 0x17FFF: Layer 3 Routing network queues

Runtime FW/Kernel: 0x9040 - 0x10FFF (32 KB) : runtime loaded fw/kernel text/data

Unreserved: 0x18000 - 3FFFF (160 KB): Unreserved

Base FW regions should be treated as dead portions of l1_address_map.

pgkeller commented 1 week ago

For start, we could put the RingBuffer in the 32K hole, then the erisc FW and mailboxes at the start of unreserved. 32K may not be enough to double buffer large kernels, but we can re-evaluate that later (and swap the above if needed)