search

samoht / mirage-seal

Sealing local directories into Mirage Unikernels
27 stars 10 forks source link

Generating large site results in Stack overflow exception #22

Open CumpsD opened 9 years ago

CumpsD commented 9 years ago

I tried to use mirage-seal on a 50MB site, which resulting in a stack overflow exception on cmx static2.ml

When I deleted most of the files (all images) till I had a site of about 3 MB everything went ok.

Any ideas on how to deal with this?

CumpsD commented 9 years ago

For information, the 50MB of files resulted in a 154MB static2.ml crunch file

samoht commented 9 years ago

Hum, indeed, the compiler is raising Stack Overflow. That's annoying. An option would be to generate a block device that you can attach on start-up but I don't know how well it works on EC2.

samoht commented 9 years ago

Seems that the compilation of list is not tail-recursive (I've repro the issue with a list of 100_000 1s). @yallop any idea why and if this is already a known issue?

yallop commented 9 years ago

Several parts of the compiler are not tail-recursive. I wouldn't be surprised if compilation of large literals caused a stack overflow. It's best to report the issue on Mantis if it occurs with the latest release of OCaml.

Which OCaml version are you using? I can reproduce the Stack_overflow issue with 4.01.0, but my test case compiles fine with 4.02.3.

CumpsD commented 9 years ago

It was 4.01.0 indeed. Installing 4.02.3 now and trying again.

... building, fingers crossed, ... 10 minutes later it is still compiling on it. Might be worth researching time to generate a block device :)

After 20 minutes it sadly enough crashed on stack overflow again

CumpsD commented 9 years ago

Managed to get it to work on my local Xen by using a FAT system for the static content (left the SSL keys in the crunch FS). Not sure yet on how to get it onto EC2.

jonludlam commented 8 years ago

Perhaps the iso9660 route might work well (http://lists.xenproject.org/archives/html/mirageos-devel/2015-09/msg00043.html). We'd just need to extend the mirage tool to support the iso library.

samoht commented 8 years ago

@jonludlam that'd be indeed ideal! Maybe a good opportunity to test the upcoming new release with Functoria :p

jonludlam commented 8 years ago

Yup, I was holding off doing any integration until Functoria had landed, so I can start looking at that again.

CumpsD commented 8 years ago

Perhaps a silly question, but is Functoria the name of an upcoming release or something else?

mor1 commented 8 years ago

@CumpsD It's the library that forms the basis rewrite of the Mirage CLI tool by @Drup -- it makes for a much nicer, more easily extensible Mirage tool :)

https://github.com/mirage/functoria

amirmc commented 8 years ago

@CumpsD You can read the post (as it currently stands) about functoria over at https://github.com/mirage/mirage-www/pull/396 - comments/feedback welcome :)

CumpsD commented 8 years ago

Seems to be all done to me besides releasing it ;) Some big changes, looking forward to trying it out

CumpsD commented 8 years ago

Any updates on the Functoria plan? :)