currently enclave/normal world memory management is fragile. for example just calling the libc functions is breaking the memory layout (following log, libc tests works before callling normal_world()).
Also it doesn't let proper debugging via openocd & jtag. we need a better mem allocators at least.
`
spike --ic=16:1:8 --dc=16:1:8 --l2=128:1:8 -p1 --enclave=1 -m1024 --l2_partitioning=1 hello
spike.cc: creating vector with 2 elements.
spike.cc: read management binary with 2344 amount of Bytes, ferror: 0, feof: 1
spike.cc: making working memory at base: 0x80000000
spike.cc: Initializing partitioned cache.
sim.cc: Constructing simulator with 3 processors and 2 enclaves.
spike.cc: starting simulation.
sim.cc: running htif.
warning: tohost and fromhost symbols not in ELF; can't communicate with target
sim.cc: Adding boot rom with start_pc 0000000004000000
sim.cc: reset vector contents.
sim.cc: 0x00000297,
sim.cc: 0x02028593,
sim.cc: 0xf1402573,
sim.cc: 0x0182b283,
sim.cc: 0x00028067,
sim.cc: 0x00000000,
sim.cc: 0x04000000,
sim.cc: 0x00000000,
sim.cc: 0x00000002,
sim.cc: 0x00000001,
sim.cc: 0x00000002,
processor.cc: Enclave ID on core 1 changed to 0xfffffffffffffffd
processor.cc: Enclave ID on core 2 changed to 0xfffffffffffffffc
processor.cc: Enclave ID on core 1 changed to 0xfffffffffffffffe
In management enclave.
processor.cc: Currently tagging pages outside of DRAM is not supported 0x4000000
processor.cc: Currently tagging pages outside of DRAM is not supported 0x4001000
processor.cc: Currently tagging pages outside of DRAM is not supported 0x4002000
Encl 0
this is from src!Before memcpy dest = this is from src!
heyyyy this is from src! and 1234
hey again1234
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000000
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000000
t 0
Received create enclave message.
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000010
processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000010
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000017
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000017
t 7
Received set argument message.
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000007
processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000007
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800030004
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800030004
t 4
Received donate page enclave message.
Denying load access to enclave 18446744073709551614, virtual address 0x80003000, physical address 0x80003000, number of pages 262144, page size 0x1000
processor.cc: Changing page 3 to tag: 1
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800040004
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800040004
t 4
Received donate page enclave message.
Denying load access to enclave 18446744073709551614, virtual address 0x80004000, physical address 0x80004000, number of pages 262144, page size 0x1000
processor.cc: Changing page 4 to tag: 1
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800050004
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800050004
t 4
Received donate page enclave message.
Denying load access to enclave 18446744073709551614, virtual address 0x80005000, physical address 0x80005000, number of pages 262144, page size 0x1000
processor.cc: Changing page 5 to tag: 1
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004
processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000015
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000015
t 5
Received switch enclave message.
processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0xfffffffffffffffc, content 0x0000000000000015
processor.cc: core 2 at pc 0x00000000040006f4 found messages in box 1, with message: source 0xfffffffffffffffe, destination 0xfffffffffffffffc, content 0x0000000000000015
processor.cc: core 2 sending mailbox message: source 0xfffffffffffffffc, destination 0xfffffffffffffffe, content 0x0000000000000015
processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 2, with message: source 0xfffffffffffffffc, destination 0xfffffffffffffffe, content 0x0000000000000015
processor.cc: Enclave ID on core 2 changed to 0x1
Denying load access to enclave 1, virtual address 0x8000cbf0, physical address 0x8000cbf0, number of pages 262144, page size 0x1000
���processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000005
��1��Denying load access to enclave 1, virtual address 0x8000cc00, physical address 0x8000cc00, number of pages 262144, page size 0x1000
Denying load access to enclave 1, virtual address 0x8000cc08, physical address 0x8000cc08, number of pages 262144, page size 0x1000
Denying load access to enclave 1, virtual address 0x8000cc10, physical address 0x8000cc10, number of pages 262144, page size 0x1000
sim.cc: ERROR returning NULL for address to memory.
Makefile:78: recipe for target 'run' failed
make: *** [run] Error 255
This is a pretty generic todo. @zatkh would it be ok if we close this TODO and add a more specific TODO. For example: add support for XV6 to add and keep track of enclave pages.
currently enclave/normal world memory management is fragile. for example just calling the libc functions is breaking the memory layout (following log, libc tests works before callling normal_world()). Also it doesn't let proper debugging via openocd & jtag. we need a better mem allocators at least. ` spike --ic=16:1:8 --dc=16:1:8 --l2=128:1:8 -p1 --enclave=1 -m1024 --l2_partitioning=1 hello spike.cc: creating vector with 2 elements. spike.cc: read management binary with 2344 amount of Bytes, ferror: 0, feof: 1 spike.cc: making working memory at base: 0x80000000 spike.cc: Initializing partitioned cache. sim.cc: Constructing simulator with 3 processors and 2 enclaves. spike.cc: starting simulation. sim.cc: running htif. warning: tohost and fromhost symbols not in ELF; can't communicate with target sim.cc: Adding boot rom with start_pc 0000000004000000 sim.cc: reset vector contents. sim.cc: 0x00000297, sim.cc: 0x02028593, sim.cc: 0xf1402573, sim.cc: 0x0182b283, sim.cc: 0x00028067, sim.cc: 0x00000000, sim.cc: 0x04000000, sim.cc: 0x00000000, sim.cc: 0x00000002, sim.cc: 0x00000001, sim.cc: 0x00000002, processor.cc: Enclave ID on core 1 changed to 0xfffffffffffffffd processor.cc: Enclave ID on core 2 changed to 0xfffffffffffffffc processor.cc: Enclave ID on core 1 changed to 0xfffffffffffffffe In management enclave. processor.cc: Currently tagging pages outside of DRAM is not supported 0x4000000 processor.cc: Currently tagging pages outside of DRAM is not supported 0x4001000 processor.cc: Currently tagging pages outside of DRAM is not supported 0x4002000 Encl 0 this is from src!Before memcpy dest = this is from src! heyyyy this is from src! and 1234 hey again1234 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000000 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000000 t 0 Received create enclave message. processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000010 processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000010 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000017 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000017 t 7 Received set argument message. processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000007 processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000007 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800030004 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800030004 t 4 Received donate page enclave message. Denying load access to enclave 18446744073709551614, virtual address 0x80003000, physical address 0x80003000, number of pages 262144, page size 0x1000 processor.cc: Changing page 3 to tag: 1 processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800040004 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800040004 t 4 Received donate page enclave message. Denying load access to enclave 18446744073709551614, virtual address 0x80004000, physical address 0x80004000, number of pages 262144, page size 0x1000 processor.cc: Changing page 4 to tag: 1 processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800050004 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000800050004 t 4 Received donate page enclave message. Denying load access to enclave 18446744073709551614, virtual address 0x80005000, physical address 0x80005000, number of pages 262144, page size 0x1000 processor.cc: Changing page 5 to tag: 1 processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 at pc 0x00000000800001de found messages in box 1, with message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000004 processor.cc: core 0 sending mailbox message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000015 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 0, with message: source 0x0000000000000000, destination 0xfffffffffffffffe, content 0x0000000000000015 t 5 Received switch enclave message. processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0xfffffffffffffffc, content 0x0000000000000015 processor.cc: core 2 at pc 0x00000000040006f4 found messages in box 1, with message: source 0xfffffffffffffffe, destination 0xfffffffffffffffc, content 0x0000000000000015 processor.cc: core 2 sending mailbox message: source 0xfffffffffffffffc, destination 0xfffffffffffffffe, content 0x0000000000000015 processor.cc: core 1 at pc 0x00000000040006f4 found messages in box 2, with message: source 0xfffffffffffffffc, destination 0xfffffffffffffffe, content 0x0000000000000015 processor.cc: Enclave ID on core 2 changed to 0x1 Denying load access to enclave 1, virtual address 0x8000cbf0, physical address 0x8000cbf0, number of pages 262144, page size 0x1000 ���processor.cc: core 1 sending mailbox message: source 0xfffffffffffffffe, destination 0x0000000000000000, content 0x0000000000000005 ��1��Denying load access to enclave 1, virtual address 0x8000cc00, physical address 0x8000cc00, number of pages 262144, page size 0x1000 Denying load access to enclave 1, virtual address 0x8000cc08, physical address 0x8000cc08, number of pages 262144, page size 0x1000 Denying load access to enclave 1, virtual address 0x8000cc10, physical address 0x8000cc10, number of pages 262144, page size 0x1000 sim.cc: ERROR returning NULL for address to memory. Makefile:78: recipe for target 'run' failed make: *** [run] Error 255
`