32-bit Serialized Blocks Can't Be Materialized in 64-bits

[seandenigris]

I reported the following problem on Pharo-Dev:

1. Serialized a block in #60540 32-bit
2. Materialized it in same version 64-bit and it was "broken", i.e.: a. It can't be serialized again as indicated above b. Strangely, its print string in the GT Inspector is the entire source code of #outerContext instead of the source code of just the block

aBlock:

• The following instVars are the same in both images: startup = 56, numArgs = 1
• outerContext also seems the same, with the only obvious exception that the

  method bytecodes are not all the same

  • sender = nil
  • pc = 45
  • method = (SmallBaselineLoadScript>>#descriptionConflictBlock)
  • closureOrNil = nil
  • receiver = aSmallBaselineLoadScript

Eliot Miranda answered:

Fuel is not adjusting the pcs in Contexts and BlockClosures when loading something saved in a different word size. CompiledCode (CompiledMethod & CompiledBlock) is a hybrid object, the first part being object references (the method’s header followed by its literals), the second part being bytes (the bytecodes for the method and any additional info encoded in trailing bytes). So in 32 bits a pc is (4*numLiterals+1) less than it is in 64 bits, and Fuel and other code must adjust things accordingly. See CompiledCode>>#initialPC

[eliotmiranda]

I said "Looks like..", not "Is" :-). This has to be verified. I could easily be wrong.

[tinchodias]

Hi. Some years ago we would solve this issue by signaling a fuel descriptive error. I mean, we would not add support to transform objects between 32 bits from/to 64 bits: "too smart" for Fuel, which was intended to just materialize what was serialized without conversions.

What do you think?

[eliotmiranda]

Hi Martin,

On Oct 10, 2018, at 12:09 PM, Martín Dias notifications@github.com wrote:

Hi. Some years ago we would solve this issue by signaling a fuel descriptive error. I mean, we would not add support to transform objects between 32 bits from/to 64 bits: "too smart" for Fuel, which was intended to just materialize what was serialized without conversions.

What do you think?

I know it is easy to convert between the two when deserializing. Squeak image segment support has already been updated to allow free interchange between 64 & 32 bits. And image segments are “physical”; tied to the heap representation. Fuel should be even easier, being a fully virtual format.

There are a few things to take care of:

• PCs need mapping in BlockClosure and Context. See CompiledCode>>#initialPC for the computation.
• if floats are materialized using BoxedFloat64 one can multiply by 1.0 to derive the SmallFloat immediate instance on 64 bits
• SmallInteger, LargePositiveInteger & LargeNegativeInteger must be handled specially. If SmallInteger is written differently to LargePositiveInteger & LargeNegativeInteger then materializing 61 bit immediate SmallIntegers may yield LargePositiveIntegers & LargeNegativeIntegers on 32 bits. And when materializing LargePositiveIntegers & LargeNegativeIntegers written on 32 bits on a 64 bit system, the values may be representable as SmallIntegers.

Other than that, things tend to “just work” :-)

,,,^..^,,, (phone)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[theseion]

This should now work (as of 3.0.3) but we'll have to test it. I don't have the means to do that at the moment.

[seandenigris]

Great! Let me see if I can find the data that first generated the error to test...

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will remain open but will probably not come into focus. If you still think this should receive some attention, leave a comment. Thank you for your contributions.

[theseion]

Still needs to be tested.

[seandenigris]

Seems to work (Ubuntu 16.04.3, Pharo 8.0):

Loaded Fuel into both 32 and 64 bit images via:

Metacello new
    repository: 'github://theseion/Fuel:3.0.2/repository';
    baseline: 'Fuel';
    load.

Then serialized in 32-bit image:

FLSerializer serialize: [ :ex | ex asNumber + 1 ] toFileNamed: '~/Documents/block-32-64/32/block32.fuel'

Then materialized in 64-bit image:

block := FLMaterializer materializeFromFileNamed: '~/Documents/block-32-64/32/block32.fuel'.
block value: '4' "=5"

I also did a second test returning the block from a method (instead of Playground) and it also worked as long as the method was in the target image as well.

Thanks!

[theseion]

Great! Thanks a lot!