SDK 2.0 compatibility - Githubissues

tjkirch commented 1 year ago

Panic sent out this email about the upcoming SDK 2.0 update, which links to this detail page.

I tested my simple crankstart-based game with SDK 2.0 - it ran fine in the simulator, but it crashes on my Playdate, saying CAPI handler function wasn't located in loaded data. I'm wondering what's required for compatibility.

Here are some random thoughts:

Build requirements: pdc now needs to know about ELF relocations via a pdex.elf file. I don't see it generating one automatically. I'm not knowledgeable in this area; is this a concern, or is Rust doing automatically what Panic suggests in their updated compiler/linker flags? I saw someone mention emit-relocs for tock.
Similar for updated linker requirements.
Once those basics are figured out: SDK 2.0 games aren't compatible with 1.x Playdates, and SDK 1.x games aren't compatible with 2.0 Playdates. Can it just be up to the crankstart user to pick an SDK and crankstart will work (possibly via detection) or will there need to be a flag? Maybe 1.x compatibility just won't matter?

rtsuk commented 1 year ago

@boozook tells me he's going to take a look at the build and linking issues.

On the subject of picking an SDK, I'd recommend just switching to 2.0 to start, and back-fit a 1.0 switch if there's anyone ends up needing it.

boozook commented 1 year ago

Same Issue for me. We're unable to test on the device because latest released OS version is 1.13.7 without new loader. We just can hope :)) official note They will change elf loader in the future OS 2.0.

Playdates on OS 1.x will not be able to launch your 2.0 build, but will display a dialog asking the user to upgrade their Playdate’s OS. If OS 2.0 is not available to the public, the user will not be able to run your game.

I'll try to get "normal execution" of new build (for new loader with relloc support) on the rev1 device with OS 1.13.7 on board to get message like "The Playdate OS is too old to run this game". I'm working on it...

boozook commented 1 year ago

Update. Actually nothing interesting. I've build a correct binary (as I suppose) with rust and rust-lld, but after pdc there's nothing (only 130 bytes of nothing).

My test "foo" binary contains the following...

I'm sure it should work because now in linker-map `eventHandlerShim` as `ENTRY` explicitly. And I have one exactly as needed, same as in the setup.c. ```text __bss_end__ __bss_start__ __data_end__ __data_start__ __etext eventHandlerShim foo: file format elf32-littlearm Program Header: LOAD off 0x00000080 vaddr 0x00000000 paddr 0x00000000 align 2**4 filesz 0x00001774 memsz 0x00001778 flags rwx STACK off 0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64 filesz 0x00000000 memsz 0x00000000 flags rw- Dynamic Section: Sections: Idx Name Size VMA LMA Type 0 00000000 00000000 00000000 1 .text 00001774 00000000 00000000 TEXT 2 .data 00000000 00001774 00001774 TEXT 3 .bss 00000004 00001774 00001774 BSS 4 .rel.text 00000598 00000000 00000000 5 .ARM.attributes 00000047 00000000 00000000 6 .comment 00000013 00000000 00000000 7 .symtab 000007a0 00000000 00000000 8 .shstrtab 0000004f 00000000 00000000 9 .strtab 00000f71 00000000 00000000 ``` 🤷‍♂️

boozook commented 1 year ago

One more update. I've compiled one of examples in the SDK/C_API/Examples... Just to compare sections in the resulting elf. And you know what? When I run it on the device, I see "no capi handler"! Well, I'm not sure that its possible right now if I can't run example built with pdc2.0-beta on my hardware with OSv1.13.7. All ideas are welcome.

tjkirch commented 1 year ago

One more update. I've compiled one of examples in the SDK/C_API/Examples... Just to compare sections in the resulting elf. And you know what? When I run it on the device, I see "no capi handler"! Well, I'm not sure that its possible right now if I can't run example built with pdc2.0-beta on my hardware with OSv1.13.7. All ideas are welcome.

@boozook Interesting - I was able to get the expected "Update Needed" message on my 1.13.7 device. I just unpacked the 2.0 beta2 SDK to my $PLAYDATE_SDK_PATH, did cd $PLAYDATE_SDK_PATH/C_API/Examples/Hello World, then make all, and copied HelloWorld.pdx to my Playdate. Is there any chance you had the path variable pointing to a 1.x SDK, or something like that?

Two things surprised me, though - the pdx had no ELF relocations file, and it ran on both 1.x and 2.x simulators, which I didn't think was supposed to work. I'd have to guess that the simulator is special-cased to run 1.x and 2.x games.

boozook commented 1 year ago

@tjkirch I've reported about it on the dev forum. There's video with reproduction. Link

tjkirch commented 1 year ago

@boozook Do you think there's any way others could help with your investigation? I'm not experienced in this area, but I'm worried about the SDK 2 release obsoleting all crankstart games, so I'd like to do what I can!

tjkirch commented 1 year ago

Panic just announced that SDK 2.0 is going live in a few days, on June 12th, 2023. They make it easy to update, so most users will probably do so quickly, and then crankstart games will no longer work. We definitely need to figure this out.

rtsuk commented 1 year ago

Has anyone shipped a game using crankstart?

tjkirch commented 1 year ago

Has anyone shipped a game using crankstart?

I'm not sure, but I'd like to. I've spent a long time on mine, and I'd be quite sad to have to rebuild it in Lua.

boozook commented 1 year ago

I have an MVP-like build script "rust build executable bin then give it to pdc". I not sure that it works, so I just need to test it with latest beta. Or better test on real hardware, I have one, but so have to wait OS 2.0 release. 🤷🏻‍♂️

tjkirch commented 1 year ago

I have an MVP-like build script "rust build executable bin then give it to pdc". I not sure that it works, so I just need to test it with latest beta. Or better test on real hardware, I have one, but so have to wait OS 2.0 release. 🤷🏻‍♂️

Nice, thank you!!

If you're interested, they requested beta testers for new releases, and you obviously have a very good case. You might be able to try 2.0 on-device sooner. https://devforum.play.date/t/now-accepting-applications-for-beta-testers/11123

boozook commented 1 year ago

@tjkirch thank you, I didn’t know that.

Playdate OS/SDK Beta Tester Sign Up Your response has been recorded

boozook commented 1 year ago

I'm working on it... 😐

scratchminer commented 1 year ago

Here's what I've figured out so far in terms of the pdex.bin format:

0x0: constant Playdate PDX 0xC: flags (all 0) 0x10: some kind of MD5 hash? 0x20: BSS start address 0x24: BSS end address 0x28 - 0x30: unknown, but clearly two uint32s

(everything after this is compressed)

.text section contents .data section contents relocation table: offsets are uint32 and relative to the compression boundary

boozook commented 1 year ago

Sad news for now. I tried really many options and didn't understand where I was wrong. So I have no idea for now.

There is dummy isolated example (/playground).

@scratchminer, @tjkirch, do you have any ideas. Seems to I need help.

boozook commented 1 year ago

Hm... Probably I have another option. Tomorrow I'll try to use only gcc linking. Will see...

boozook commented 1 year ago

Okay. Almost all my previous solutions did not works because of bug in the pdc. Now building works fine. Testing relocations and fix will be here soon.

boozook commented 1 year ago

Status update:

Simple build lib with rustc+arm-gcc works
I have some trouble with alloc, just strange cstrings behaviour, solvable. Sims I close to finish.

UPD: For 2. - I've just forgot to build alloc, so -Z build-std=core,alloc solve this problem. ❤️‍🔥

boozook commented 1 year ago

Probably I'm done. Please test and review.

boozook commented 1 year ago

Also we're need some validity tests for produced output to detect regression on future changes.

boozook commented 1 year ago

I need feedback to merge crank#42.

boozook commented 1 year ago

I would close this because there's a latest fix in #48.

pd-rs / crankstart

SDK 2.0 compatibility #45