milibopp / jelly

Abstraction layer for running numerical hydrodynamics software
1 stars 0 forks source link

Big endian will make problems #40

Open Moredread opened 10 years ago

Moredread commented 10 years ago

Currently the code (esp. the header generation) will produce garbage on big endian systems. Either we should fix this or fail on such systems.

milibopp commented 10 years ago

Is this because Arepo uses native endianness instead of something machine-independent? Is there a flag in Arepo to override such behaviour?

Moredread commented 10 years ago

I have to look at the source. There is switiching of endiness in some parts I have seen, so it might be, that little endiness is the default for output files. In that case we should also make sure to be machine independend.

milibopp commented 10 years ago

The current code is machine-independent as per the Python specs for binary data formatting. We should just make sure to either make this a flag (perhaps detecting and suggesting native endianness) or suggest in the docs to configure Arepo to always read in little endian.

Moredread commented 10 years ago

No, atm we are not machine independent, as the splitting of the long long ntotal fields needs to be aware of endiness.

milibopp commented 10 years ago

Oh, you're right… it's jelly's fault. So, we could use a fixed endianness explicitly. Is this what Arepo does by default? From what you said it should be little-endian, so we could simply prefix all our format strings with a <, right?

Moredread commented 10 years ago

It seems like Arepo uses native endiness, but allow to convert IC files automatically with the option AUTO_SWAP_ENDIAN_READIC, but there doesn't seem to be an equivalent option for writing snapshots and restartfiles. So we need to handle it when parsing files.

Moredread commented 10 years ago

For now we should document this and only write little endiness (with <). If someone needs big endiness we can add it later.

Moredread commented 10 years ago

Or we should make the choice explicit, not implict when run on a big endian machine.

We can warn in that case though.