Ares Weekly Meeting Notes

[jalehman]

~2024.5.29

Agenda

• Developer Alpha
  • [x] Release comms @jalehman
  • [x] Release notes @matthew-levan
  • [x] Reviewed by @joemfb
  • [x] Reviewed by @eamsden
  • Release this week? Yes or No
• Status of SKA @joemfb @eamsden
• Status of Vere Compatibility @matthew-levan
• Status of PMA memory corruption bug & fuzz testing @barter-simsum
• Status of PMA garbage collection @barter-simsum
• Status of interpreter optimizations @eamsden

Additional Notes

• Release
  • @eamsden to review release notes by EOD today
  • Only supported on Linux, sometimes(?) it works on MacOS — unsure whether it's M* only or supports older architectures: https://github.com/urbit/ares/issues/234
• Instructions for running Ares can be found in DEVELOPERS.md
  • @matthew-levan to add link to heading in README
• SKA
  • In context of memoization, we're emitting incorrect labels
  • 120 call sites with incorrect/missing labels in analysis of hoon.hoon out of ~20k total call sites
  • we have extensive debug information
  • appears we're matching properly, but not normalizing/propagating constraints properly
  • @joemfb has some questions for @eamsden that they need to run through together
  • @eamsden is mostly free on Friday
• Vere Compatibility
  • Making modifications to vere PR that allows replay in a subprocess. This will allow vere to call whichever serf is linked to perform replay.
  • Started on reading events, i.e. reading a vere pier from Ares. WIP
• PMA
  • Memory corruption bug is believed to be solved
  • Chaos monkey testing is being started soon
  • @barter-simsum is clearing out some smaller known PMA bugs
  • Discussion on garbage collection between @barter-simsum @eamsden @joemfb happening later today
  • Garbage collection & testing can run in parallel
• Interpreter optimizations
  • Blocked on completion of SKA
  • Possible to optimize the treewalking interpreter some, gains unclear w/ SKA
  • Initial line of work: modify linearizer so it emits noun code, output is contiguously numerically indexed so it can be efficiently referenced from arrays
  • Not blocked on SKA
• @matthew-levan has anyone spoken with @ashelkovnykov about picking up work on this again?
  • @belisarius222 talked to Alex this morning, currently has no availability to work on ares
  • He seems interested in doing some work on this again when he's back from Japan, earliest June 4th

[eamsden]

@jalehman requests:

• close previous week's notes-issue when opening new notes issue
• link to previous week's notes-issue in current week's
• label meeting notes issues with "meeting notes" label as I have done for this one

[jalehman]

@eamsden My plan was to pin this issue and take all meeting notes here as comments — significantly less meeting note administration that way, and less clicking needed to establish context from past meetings. If you'd prefer it as outlined above, happy to do it that way.

[eamsden]

@jalehman that works too, I just don't want open meeting notes issues to proliferate or old notes to be hard to find

[jalehman]

~2024.6.5

Agenda

• Status of SKA @joemfb @eamsden
• Status of Vere Compatibility @matthew-levan
• Status of PMA smaller bugs @barter-simsum
• Status of PMA chaos monkey testing @barter-simsum
• Status of PMA garbage collection @barter-simsum
• Status of interpreter optimizations @eamsden

Notes

• Every discussion is the overlay namespace discussion
• Status of SKA
  • PR is up! Tricky cases abound in mutual recursion memoization
  • Has improved debug tooling, can get performance traces out of it, output is cleaned up
  • How far do we chase these last cases of recursive analysis? What do we decide to punt on?
  • The cases:
  • Punt all of them: analysis gets faster (details missing)
  • If you find that prior memoization of heuristic recursive analysis turns out to be wrong, turn that and everything above it indirect
  • Tangent: was not previously logging indirect calls in debug output, but this is what you care about as a consumer more than everything else
    • The paradigmatic case of an indirect call: (trip (?:(s tod:po tos:po) q))
  • Current state is subtly incorrect in small number of cases. It's not ok to be incorrect at all.
• Status of Vere Compatibility
  • PR is up! It doesn't work yet.
  • You can boot boot a fresh fakezod with vere 3.0 and an ares serf on a slim pill into the dojo, exit gracefully, and reboot
  • Currently trying to make it work without graceful exit it will restart and replay missing events in log
  • Focusing on replay, making use of code that @ashelkovnykov wrote last year
  • Slapping stuff together to get it to work ASAP, but at some point the proper design will need to be evaluated after it works to make it right
  • Debating on whether to load snapshot before or after staging mars/urth split
  • Goal: proper replay by EOW
• Status of PMA smaller bugs
  • Delay due to unexpected move
  • Smaller bugs should be closed out tomorrow
  • what are the smaller bugs?
• Status of PMA chaos monkey testing
  • Moving onto this post bugs, should result in more bugs being found
  • Will need to rent some space in the cloud with lots of storage to test at capacity
  • We could also add measurements of performance under real usage during this stage
• Status of PMA garbage collection (important discussion happened here)
  • will happen in parallel with chaos monkey
  • met with joe and edward on this last week
  • came away with multiple approaches
  • "stop the world" garbage collector
  • manually invocation of pack and meld routines
  • purpose isn't to free unneeded objects, as for instance the case in traditional GC, but to free up disk space
  • @belisarius222 most allocations are single page allocations
    • @eamsden that is not correct. when done with an event, we sum up necessary size of noun structure not already in PMA, and allocate in a single chunk.
    • initial implementation doesn't correctly allocate multi-page atoms
    • so, you have big page size allocations
  • @belisarius222 back to the original point: if you have the situation with lots of single-page allocs, and only big alloc (in terms of # of pages) is large indirect atoms, you'll end up with mostly 1-2 page free space holes in middle of PMA, new events will use one page that will fill those holes, implication being fairly good allocation with the exception of very large atoms/nouns spanning multiple pages
    • for example, movie file won't fit in the middle and PMA will almost certainly grow
    • big perf problem with GC in general is doing lots of disk copying between different parts of disk
    • if you don't do that, no matter how small your nouns get and how much you trim, your big media file will be a big anchor at the end of the pma keeping it very large
    • implications: running GC on the PMA probably won't do much if we don't move the files around, and if we do, we're doing lots of write amplification
    • solutions:
    • use a sparse file representation, so OS knows if middle pages are 0s not to represent on disk
    • move large allocations into their own files, but is more complex and has synchronization problems
    • @eamsden there's a secret third thing: pma itself is designed to allow for noncontiguous mapping of memory to disk, so this is more complex allocation logic, but that's it
      • heuristics: start grabbing stuff on top of the disk freelist from earlier to later until the sum of those thigns is size of alloc, then just make btree entries that are virtual memory contiguous, but point to different places on disk, as long as GC can understand that those are one object (more sophistication required) that's fine
      • it's like a block-based FS
      • sparse files are out — don't do this, it's unusable
      • @belisarius222 if you do it this way, would you still do multi-page allocations? @eamsden no, you'd treat them as blocks
    • @belisarius222 do you actually need GC? @eamsden yes, actually
    • @eamsden nouns and pages are semi-coupled in this design
      • when there are no more nouns in a page, you free it
      • you need GC at the noun level
    • It's still extent-based, but you want the ability to express multiple ranges for a given object
    • Takeaway: a single allocation can be multiple noncontiguous regions; you can break up large allocations into multiple regions. A very naive copying GC is basically fine. It's fairly straightforward to make it less naive and probably sufficient for a while. An updated cell representation for cells in the PMA that allows you to move cells a page at a time is good.
    • @belisarius222 it would be great to have a writeup of this at some point, since it's hard to hold in the head.
• Status of interpreter optimizations
  • No further update from last week.
  • This could be done in a huddle-up at lake summit with the devs on this call.

[jalehman]

~2024.6.12

Agenda

• Status of SKA @joemfb @eamsden
• Status of Vere Compatibility @matthew-levan
• Status of PMA smaller bugs @barter-simsum
• Status of PMA chaos monkey testing @barter-simsum
• Status of PMA garbage collection @barter-simsum
• Status of interpreter optimizations @eamsden

Notes

• SKA
  • It's done (holy shit)
  • Analysis is done, has no known issues, takes 1.5 minutes on hoon.hoon
  • No known errors, completes in reasonable amount of time
  • Two big wins
    1. since beginning, wrong layer of memoization. context was not flowing back upstream to impact subsequent checks ("am I the same as this other thing"). Fix looks like it should be much slower (?), biggest perf win is not entirely understood
       • @eamsden would a jet for hub still speed things up significantly? unknown
  • What's next?
  • put pretty printing compute behind a flag and see how much further that speeds things up
  • either or both of boot a ship from the slim pill, try to analyze more things like arvo, vanes, userspace stuff, the analysis itself
  • no known bugs, memoization may be blocking future analysis, memoization is an unbounded space leak, overall lifecycle of analyzed code is incomplete
  • analyzing other vanes would be to find more bugs
  • @eamsden since we're still in alpha, merge condition for PR: successful boot of ship using slim pill. we should attempt this.
    • it's not difficult to do, nothing is preventing us from just doing this.
  • @matthew-levan to do this, just needs the jamfile, let's have @joemfb work with matt on it so he can learn too
  • unblocks the noun code — the interpreter optimizations
    • tree-walking interpreter is separate from the codegen work
    • there's an interpreter that takes nouns in format that linearizer produces, and executes those directly. those nouns are lists of instructions, so the optimization is to introduce a level of indirection in those nouns and use that to allow the rust to optimize those nouns by turning one level of them into arrays when it sees them. that turns a hamt lookup into an array lookup, and makes direct jet matching more real. @matthew-levan wrote most of the original codegen interpreter and is probably the right person to work on this.
  • lifecycle stuff that joe mentioned: needs to have a resolution for beta. doesn't block work that lets us show wins. it may blow up memory, but that's worth it for showing nock going fast.
    • you want subsequent rounds of analysis to further optimize
  • we can do this today
  • @eamsden @joemfb to commit the jamfile as an artifact into the repo
  • it is pushed
• Vere compatibility
  • got replay working last week with nasty, stupid code (bad code! bad!)
  • @matthew-levan needs to spend more time on it this afternoon before pushing
  • needs advice on whether or not to restage mars/urth split or vere snapshot loads in ares
  • @eamsden a libvere crate for rust sounds cool
    • two approaches:
      1. link c code from vere into rust, and in rust just write methods that marshal vere nouns to ares nounds
      2. write pure rust code that can read the snapshot bins directly by mmapping them and doing pointer offsets. upside: no extra c dependencies required in this case. downside is that it has to match what vere expects exactly in read-only form.
    • option two is favored
    • write rust code that's able to mmap north.bin and south.bin and do pointer offsets into them, and migrate cold state
    • these are new lines of work in addition to basic replay
    • let's push this until after slim pill boot and linearizer optimizations (specifically: basic block coalescing, register minimization, some other things related to recursion that @eamsden will later enumerate)
• @joemfb we're nearing the point of being able to begin the bytecode interpreter
  • @frodwith isn't interested in beginning this until he knows that the ground won't shift
  • noun code is a prototype of the bytecode
  • if you have the noun code, you're as near as you're going to get to a spec for the bytecode
  • current IR leaves too much open
  • we need a separate time set aside for discussing lifecycle stuff in detail
  • this is a good time for @matthew-levan to reengage with codegen interpreter
  • @matthew-levan will get current replay work finished and PR'd today
  • depending on what's found during slimpill boot, codegen may be ready for merge into status branch
• happy path plan:
  • replay wrapped up today
  • slimpill boot works, SKA gets merged to status
  • do some benchmarks
  • spend week of lake summit writing interpreter optimizations as a group
• Status of PMA smaller bugs
  • cleared out most of the ones being worked on
  • new one popped up about ephemeral structure restoration
  • cause seems to be marking pages as dirty that shouldn't be marked dirty
  • believed to be fixed
  • should be complete by EOD today
  • includes testing and validating ephemeral structure restoration, fixes known issues with free that enable GC, fixes restore of PMA, these should be mergeable into alpha
  • there'll be several PRs forthcoming later today
  • @matthew-levan wants to get a rundown on @barter-simsum 's code next week
• Status of PMA chaos monkey testing
  • this starts next (tomorrow)
  • should surface more bugs and/or metrics
• Status of PMA garbage collection
  • no update over last week
• Status of interpreter optimizations
  • see above discussion in and following SKA
• Other discussion
  • if slim pill boots, solid pill should as well
  • if it doesn't, we'll need a slimmer pill

[jalehman]

~2024.6.26

Agenda

• Status of nouncode @matthew-levan @joemfb
• Status of Vere compat @matthew-levan
• Status of PMA @barter-simsum

Notes

• Nouncode status
  • Has not advanced past lake summit
  • Started speccing what needed to be done
  • Pushed WIP commit
• Vere compat status
  • Vere side is done and merged
  • Ares side needs review but is working
  • Design review for correctness is needed beyond simple code review @eamsden
  • @joemfb best bet may be to go a step further towards mars/urth style replay
  • Need to be more explicit about where read/write responsibilities live between vere/ares
• How can we advance the nouncode work while matt is out?
  • We need to be able to print the linearized output
  • This would help us troubleshoot faster
  • @eamsden has an unpushed WIP on nouncode that he hacked on that will be helpful, but probably cannot take it all the way to completion
• Is Paul still lined up to do bytecode?
  • We need to talk to him and ensure that he's actually still on deck to do this work
  • @joemfb to talk to @frodwith this week to determine if/when he can begin on this
  • What more would you want to see to get started?
  • Prerequisites:
  • Lifecycle stuff
    • @belisarius222 doesn't think this blocks starting bytecode
  • Working Nouncode
  • Optimized Nouncode
• @barter-simsum was sick last two days
  • solved an issue with dirty bitmaps being accidentally persisted ahead of his talk
  • matt found a bug with fourth GB allocation? 3 gigs works, 4 gigs fails
  • what's the expression to reproduce? @belisarius222 to provide, it's a bex one-liner
    • Write a C test that does the allocation
  • one failing test (probably bad test logic) needs to be fixed
  • what do tests prove?
  • restoration
  • malloc of contiguous space
  • structural equality of ephemeral data structures upon restore
  • freeing data
  • freelists are being properly coalesced
  • close and reopen b tree, partitions are where they're expected to be
  • need a demo video that shows loading of 1 TB of data into an Urbit