wjhun
commented
5 years ago I'm able to get a ruby hello world using the munmap branch. This branch isn't quite ready for merging - the implementation is kind of a hack to just get ruby unstuck. Because of some idiosyncrasies with the bitmap allocator, the existing code could be potentially unsafe in certain situations. I have a plan for fixing the bitmap allocator logic (also allowing unaligned allocations - would be good for tfs) to make these unmaps more straightforward. Prob 1-2 add'l days to fix.
A few notes about this ruby/sinatra test:
- mkfs is really, really slow with the large image and large number of small files...there is some low-hanging fruit that can speed this up (avoiding a copy step, unrolling runtime_memcpy() and doing word copies - it got reverted to a sad byte copy at some point and never got fixed)
- the request takes a long time to serve - why? Because, for some reason, ruby or a library is repeatedly mmap()ing large areas of memory and immediately unmapping them again. Given that we map everything and don't fill pages on demand, we're also zeroing the whole damn allocation...and of course the zero() implementation sucks too.
Possibilities for the short term:
- run with kvm (still with user mode networking, not bridging) to greatly speed up emulation speed
- fix runtime_memcpy() and zero() to be halfway decent
- figure out why such large mappings are occurring and try to eliminate them. They seem to occur just after a getcwd()...which appears to succeed, but it's not clear to me yet what it is doing.
For the longer term:
- crude demand paging for anonymous, zero-filled pages only...we actually kind of have enough structure to do it
$ ops load ruby_2.3.1 -p 4567 -c config.json
Extracting /home/wjhun/.ops/packages/ruby_2.3.1.tar.gz to /home/wjhun/.ops/.staging/ruby_2.3.1
[ruby myapp.rb -o 0.0.0.0]
booting /home/wjhun/.ops/images/ruby.img ...
assigned: 10.0.2.15
[2019-03-10 06:48:50] INFO WEBrick 1.3.1
[2019-03-10 06:48:50] INFO ruby 2.3.1 (2016-04-26) [x86_64-linux-gnu]
== Sinatra (v2.0.5) has taken the stage on 4567 for development with backup from WEBrick
[2019-03-10 06:48:50] INFO WEBrick::HTTPServer#start: pid=1 port=4567
10.0.2.2 - - [10/Mar/2019:06:49:20 +0000] "GET / HTTP/1.1" 200 12 14.7117
10.0.2.2 - - [10/Mar/2019:06:49:05 UTC] "GET / HTTP/1.1" 200 12
- -> /
10.0.2.2 - - [10/Mar/2019:07:04:08 +0000] "GET / HTTP/1.1" 200 12 14.0485
10.0.2.2 - - [10/Mar/2019:07:03:54 UTC] "GET / HTTP/1.1" 200 12
- -> /
eyberg
ruby is doing a lot of mmaping of large spaces, only to immediately unmap them entirely or at least partially. Since we don't implement munmap(2) and are doing physical allocations with each mmap (no demand paging), we eventually run out of physical pages to dole out (at least in the sizes requested).
There are a number of issues here to deal with over the longer term, but for today it may suffice to implement munmap and remove such mappings. If we really want to be lazy, we can just unmap the space, return the physical allocation and leave the virtual / rangemap allocation in place rather than try to undo it. (I'm going to start here to see if it's good enough for a ruby demo.)
Of course, we might just run out of virtual address space then. To do the right thing:
Although we'd still need to do #2 for physical id heap deallocations...
We could also discuss doing a sort of poor man's demand paging just for anonymous, private mappings. This is possible now that we have the vmaps structure in place.