[Proposal] 'Memory Contract' for JerryScript structs

martijnthe commented 7 years ago

Background

Pebble App Runtime

At Pebble, the watches run 3rd party applications inside an "application runtime" (JerryScript is part of this). We shipped new firmware updates every 6 weeks or so, which often included fixes and optimization, potentially affecting that application runtime. While making changes to the runtime, we always tried our best to avoid breaking any existing applications. It's worth mentioning, that those 3rd party applications had an update cycle that was decoupled and was often way slower, e.g. once a year, when a new HW model came out.

I suspect that most JerryScript-based projects (IoT.js, Zephyr.js, mbed.js, etc.) do not have to deal with decoupled release cycles between 1st party "application runtime" and 3rd party .js application code. So the issue I'm about to explain is probably a none-issue for those projects.

"Memory Contract"

One of the biggest challenges we had to deal with, is what we call the "Memory Contract": the idea that the memory footprint (esp. heap usage) underneath an API, must never grow over time. Not obeying this "contract" meant potential trouble for existing applications. Even growing an internal struct by one byte from one firmware version to the next, could mean that a (3rd party) application that ran fine on the earlier firmware version, may run out of memory on the updated firmware, if the app used an API that allocated that bigger struct behind the scenes and it was at the edge of running out of memory. And this actually happened in practice early on, before we formally defined a "Memory Contract".

Note that in our case, the applications are written by 3rd parties over which Pebble's firmware developers had no direct control. We did try to motivate them to test their applications by releasing a beta version of our firmware ahead of the public release, in order to find out memory contract issues and hopefully giving us a chance to fix problems before the public release. But having to prod developers to fix apps for every system software release is no fun. Having applications "rot" away and having to remove them because of this is also no fun.

The "solution" we used was quite simple. By padding out structs up to the contract-defined size per type (a few extra bytes), we effectively bought ourself a bit of leeway to add fields in the future, at the cost of wasting those unused padding bytes, for example:

typdef struct {
  int foo;
  int bar;
  uint8_t _padding[8];
} SomeDatastructureInternalToAPIImplementation;

_Static_assert(sizeof(SomeDatastructureInternalToAPIImplementation) == 16,
               "Whoops, larger than allowed!");

I realize many types of refactorings affect memory usage (both the heap as well as stack) in ways that will not be covered or caught by this system. That said, it did cover many refactorings that we did and thereby effectively slowed down "application rot".

Memory Contract for JerryScript

When we added JerryScript in the mix at Pebble, we ignored to define a "Memory Contract" for the JerryScript code itself. We felt it was a topic that needed to addressed in JerryScript itself, but never really got to addressing this.

Our proposal is based on pre-processor macros, to support semi-automatic padding of structs.

In creating them, we have 2 design goals:

for clients of JerryScript that do not share our concern, there should be no impact on the RAM footprint.
the "baseline" for how much padding should be added should be decided by the client project, based on how large the unpadded structs are when starting the project and also based on a guesstimate of how long into the future they want to be able support that contract (the more padding you add, the more wiggle room you buy to allow for future expansions).

Sketch:


// --- For each struct declaration in JerryScript:

JERRY_TYPEDEF_STRUCT({
 // ... all the fields here ...
}, ecma_object_t);

// ... etc ...

// --- default-padding.h (part of JerryScript)

// By default, don't add padding:
#ifndef JERRY_TYPEDEF_STRUCT
#  define JERRY_TYPEDEF_STRUCT(c_type, ...) \
 typedef struct { \
   __VA_ARGS__ \
 } c_type;
#endif

// --- my-project-padding.h (passed into gcc using -include)

// Max struct sizes ("Memory Contract"):
#define EXPECTED_SIZE_ecma_object_t             8
#define EXPECTED_SIZE_ecma_built_in_props_t     16
// ... etc ...

// Padding size controls:
#define PADDING_SIZE_ecma_object_t         4
#define PADDING_SIZE_ecma_built_in_props_t 4
// ... etc ...

#define JERRY_TYPEDEF_STRUCT(c_type, ...) \
 typedef struct { \
   __VA_ARGS__ \
   uint8_t _padding[PADDING_SIZE_##c_type]; \
 } c_type; \
 _Static_assert(sizeof(c_type) == EXPECTED_SIZE_##c_type, \
                "struct 'TODO' too large!");

// TODO: Need a macro for structs with flexible arrays at the end.

Because of the large alignment size of the JerryScript heap, this would probably waste too much space for most users, hence the "fallback" default-padding.h that does not add any padding at at.

I'm looking forward to feedback on this idea. Has anyone else run into these or similar concerns?

jiangzidong commented 7 years ago

I think it is a good idea.

Although it cannot deal with the memory growth brought by process/logic changes, it can cover the major cases. (according to pebble's survey, right? )
Firmware developers can choose whether to enable this feature or not.

Any other feedbacks?

zherczeg commented 7 years ago

I feel this is quite use case specific. When a person adds a new structure how can he evaluate the right contract size? Different projects probably require different sizes. I would like to see how this works in practice, how much extra burden is put on the developers before making any decision.

zherczeg commented 7 years ago

I also realized that medium scale reworks, when structures disappear and new ones introduced could also be difficult to judge from this contract perspective.

I have been thinking about this and I have an alternative proposal:

When a system needs such backward compatibility, it should reserve some memory for that purpose. E.g. if Jerry could have 64K RAM, give it only 56K. After an update old applications (and only the old ones) could use the entire 64K RAM.

The effect would be the same (we have extra space), but this padding is generic, not tied to specific structures. In Jerry the focus is reducing memory consumption, not increasing it, so 15% extra space should be enough for compatibility.

jiangzidong commented 7 years ago

@zherczeg Actually I thought of a similar idea before, but there is another problem: we cannot track "how many memory are increased because of the refactoring", so we don't know how many extra memory will assign to the old app after this refactoring. In your example, the entire memory is assigned to the app after one update, then no extra would give after the second update.

The original proposal don't know the exact memory growth either, it can know the growth part of the struct memory. But as you said, the struct could disappear/appear during refactor

zherczeg commented 7 years ago

I don't think the extra padding could solve the cumulative update problem. When we have a very old app, and we have 4-5 jerry updates since it created, padding could not help in the same way as my proposal.

To tell you the truth I don't think there is a perfect solution for this problem. Too much uncertainty. That is why we need to consider the gain/burden ratio. There is also another problem: what happens if the padding space is not enough for a change? Shall we immediately reject that patch?

martijnthe commented 7 years ago

cc @HBehrens

guilt commented 7 years ago

Actually, padding should be such that most struct members can be accessed in one cycle. That means if you end up mixing chars and ints, the ints should not get affected.

I appreciate the padding idea only for the above reason.

Compilers allow us to specify the alignment/padding automatically based on the width of the word. But increasing this just to make future ABI compatibility happen is an exercise that will never end, will waste memory till the future happens.

guilt commented 7 years ago

You could have a log/build artifact generated post the runtime build, on what the sizes of each of these structs are, so any developer who is concerned on memory could plan on optimizing the code accordingly.

However, increasing the struct size is still something I am generally concerned. I looked at JerryScript primarily because of the memory footprint. :)

rerobika commented 5 years ago

There is no conclusion after so many time, and the issue is inactive more than 2,5 years so IMO we can close this issue, please reopen it if needed.

jerryscript-project / jerryscript