search

debrouxl / hplp

Windows / MacOS X / Linux third-party connectivity kit library / linking software for the newest HP calculators, targeting the Prime at first. It is strongly derived from libti*, https://github.com/debrouxl/tilibs , which implements communication with all TI graphing calculator families.
43 stars 8 forks source link

Q: Is this still an active project to be used with HP Prime G2? #6

Open gecko65 opened 2 years ago

gecko65 commented 2 years ago

Sorry more a question then a error report. Is that still an active project I? I would try to build it on a current Linux dist.

BeatSkip commented 2 years ago

Might I suggest moving this discussion over to the discussion board on my repo? As I do like the ideas and input. And I think the issue here has pretty much been solved right? Haha

https://github.com/BeatSkip/PrimeDev/discussions/2

BeatSkip commented 2 years ago

Although @debrouxl do you happen to know what's happening with the screenshot png header bug in the communication? From the de compilation of the connectivity kit I've extracted debug strings noting there is a bug and it seems the connectivity kit does something to fix the header (and colors). But do you know what?

debrouxl commented 2 years ago

All I might have known about screenshot-related weirdness is in libhpcalcs/src/prime_cmd.c::calc_prime_r_recv_screen() :)

Cyrille-de-Brebisson commented 2 years ago

Hello guys,

It looks like you are having some great fun :-) but you might save yourself some frustration getting info directly from the horse's mouth, so to speak!

I have not spent the time reading all the communication going on in the various threads, but if you have specific questions, I would be more than willing to help you answer them.

Cyrille

BeatSkip commented 2 years ago

Aha! great to have you here @Cyrille-de-Brebisson! Well, i happen to have a couple questions for you.

When i request a backup Command 0xF9, i seem to receive a zlib encoded file containing a summary of the entire contents. but only when i restart my program and reinitiate the connection i seem to get a contstant stream of file packets into my program. Oddly enough in the V1 protocol while i was working in the V2 protocol before restart. correctly ACK the last packet of the summary file (i think).

what is the proper order to request list of files at the beginning? currently i'm using: 1. initiate connection. 2. request check ready command 3. request calculator info ~~4. check build, if build >10500, ~~ 5. send protocol change V2 command, note the last bit of the protocol change to 0x00, so V2 not V3 ~~ (or new protocol that just seems to limit all the packet lengths to below 100 from the calculator somehow)~~ 6. request backup. 0xF9 wrapped in V2 packet. 7. receive multi-packt message with summary.

but somehow all the actual files only come after restarting the program. but also when i restart the program ~~wand i don't want the files. as that sudden mountain of packets on restart before initialization f's with the ~~ program (although to be fair, that is my library just not handling it correctly yet)~~

edit: this turned out to be a bug in my back and forth acking with the protocol. now it dumps out everything as expected!

edit2: I can't seem to get the compression right. anything special going on? decompression of data dumps works fine, but actual compression of content doesn't seem to go happily. i'm using regular zlib deflate/inflate (with zlib header) to compress/decompress.

Note this is all with G2 calculator on build 14603

all the code is at: https://github.com/BeatSkip/PrimeDev

note: still very much a mix and match between different libraries and sources, still dilligently rebuilding everything into a neat state, so code readability is horrible and jumping around between classes is still very much a thing, As I've only been working on this project for the last two weeks or so outside my day job as mech. engineer.

Cyrille-de-Brebisson commented 2 years ago

Hello,

This is all VERY old to me, so please accept my excuses if I mess up!

When you request a screenshot. The colors are a mess, even when i litterally resend the same commands as the connectivity kit does. I presume the connectivity kit does some magic that corrects the header?

Well, there are a LOT of issues in the whole screen send. But most of the issues come from the fact that multiple color schemes are used by various part of the SW. Some time it is RGB, sometimes BGR... Depending on the header type, it might be one or the other! sorry about that.

I attched some code for you to look at... screen decoce.TXT

when the conection is going i keep getting heartbeats (at least i presume) starting with FE 01 FF. I decode this as:
FE -> out of band packet
01 -> ACK

FF -> should be acking the sequence number but FF is not valid sequence number.

This is the structure of said packet. Hope that this helps. apparently, the 3rd byte has no meaning on an ack... struct TAckPacket { u8 Byte254; // Packet ID: out of bounds: 254 u8 NewProtocolCommand; // command at 0: nack, 1: ack u8 SequenceToResend; // no meaning for ack... u8 unused_0; u32 BlockPosition; // Position associated with first bit in this packet u32 IOMessageID; // id of the IO Message beeing acked here... void inline init(u8 Sequence, u32 position, u32 IOMessageID) { Byte254=254; NewProtocolCommand= 0; SequenceToResend= Sequence; unused_0= 0; BlockPosition= position; this->IOMessageID= IOMessageID; } };

When i request a backup Command 0xF9, i seem to receive a zlib encoded file containing a summary of the entire contents. but only when i restart my program and reinitiate the connection i seem to get a contstant stream of file packets into my program. Oddly enough in the V1 protocol while i was working in the V2 protocol before restart. correctly ACK the last packet of the summary file (i think).

Here is the code that is a responce to the 0xf9 command: bool CCalc::SendWholeCalc(u32 epid) { Calc->SaveCalcData(); //save our current stuff before sending, else

// send config settings

IOSendFile(epid, CALCSettings, OTNone, false, false); IOSendFile(epid, CALCsettings, OTNone, false, false); IOSendFile(epid, CASSettings, OTNone, false, false); ///send home variables IOSendFile(epid, CALChpvars, OTNone, false, false);

wchar_t name[MaxAppNameSize]= L"\0\0";

//// Send List name[0]=L'L'; name[2]=L'\0'; for (name[1]= L'0'; name[1]<=L'9'; name[1]++) IOSendFile(epid, name,OTLists,false,false);

//// Send Matrix name[0]=L'M'; name[2]=L'\0'; for (name[1]= L'0'; name[1]<=L'9'; name[1]++) IOSendFile(epid, name,OTMatrices,false,false);

//// Send Programs for (Int i=0; i<Programs.Size; i++) IOSendFile(epid, Programs[i]->GetFileName(),OTPrograms,false,false); //use this to send whole content

//// Send Notes for (Int i=0; i<NoteList.Size; i++) IOSendFile(epid, NoteList[i].name,OTNotes,false,false);

// send Test Mode configurations wchar_t n[CTestMode::MAX_SIZE]; for(Int i = 0; i < Exams.Size; i++) IOSendFile(epid, Exams[i]->get_name(n), OTExamModes, false, false); //send the exam mode configs

/// Send Apps for(Int i=0; i<Apps.Size; i++) IOSendFile(epid, Calc->Apps[i]->GetFileName(), OTApps,false,false);

//// termination - this let's the CK know "end of data" u8 b[6]= { IOPROT_GETCALC, GetIoProtVersionNum(), 0, 0, 0, 0 }; IOManager.PostMessage(TIOMessage::New(6, b), epid); return true; }

Strangely enough, I thought that it was changed to send a zip at some point... but I do not remember...

I've seen a new command pass by in my communication dumps and can't seem to find out what it does. 0xFD, with a content length of 7 that always is: 0xFD 0x03 0x00 0x00 0x00 0x01 0x03

define IOPROT_GETIO_VERSION 253 // handle a return of multi byte IO protocol version packet information (switch to newer protocol or similar)

here is how the calc responds to it... Here it basically tells you: I can handle protocol version 3... to which the calc will respond I will switch to version ? (with the last byte being the version number) if(cmd==IOPROT_GETIO_VERSION) //receiving response regarding IO protocol version { if(m->data()[0]<=GetIoProtVersionNum()) return true; // not capable of higher then we are set to at this moment? //if we are currently using slower protocol, switch to highest supported u8 ver= ::min(m->data()[0],(u8)IOPROTOCOL_VERSION_NUM); TProtocolVersionRequestResponce p(ver); ep->PostMessage(TIOMessage::New(sizeof(p), (u8*)&p, 0)); SetIoProtVersionNum(ver); //set version number on this connection return true; }

what is the proper order to request list of files at the beginning? currently i'm using:

initiate connection.
request check ready command
request calculator info
check build, if build > ~10500,
send protocol change V2 command, note the last bit of the protocol change to 0x00, so V2 not V3 (or new protocol v2 as that just seems to limit all the packet lengths to below 100 from the calculator somehow)
request backup. 0xF9 wrapped in V2 packet.
receive multi-packt message with summary.

I do not remember, sorry!

but somehow all the actual files only come after restarting the program. but also when i restart the program and i don't want the files. as that sudden mountain of packets on restart before initialization f's with the program (although to be fair, that is my library just not handling it correctly yet)

I do have a flush on first connection from memory, specifically to handle that type of issues...

Keep the questions going, I will try to keep answering them...

Cyrille

BeatSkip commented 2 years ago

@Cyrille-de-Brebisson thanks for the info! Sure helps, if there is more I'll let you know. One question, do yo happen to know anything on the current state of development on the prime? Is there still active development going on? And is the python functionality going to be extended? (LVGL compiled into the system? This would be sooo nice(!) as it would allow full freedom in app development while still keeping the examn mode intact and secure)

edit: oh yeah, and the source code for the backup did help nicely in removing some guesswork when creating the backup procedure. image nicely working as you can see. the library switches all received files to a separate HpBackup class containing everything when it is in backup mode.

the backup mode now starts when it receives the actual ACK from the backup request and stops when the last backup done message comes in, just to make sure the backup is nice and complete.

BeatSkip commented 2 years ago

@Cyrille-de-Brebisson I've got all the communication working now and the code editor is implemented. I'm just having trouble decoding the apps. I've managed to reverse engineer where all the sections in the apps that are sent over are sliced. So I have separate files. But I'm struggling with the header contents of the 'hpapp' file and related specific files. Where the lengths of the filenames inside the programs are encoded etc. And what exactly I need to re-encode in the headers when I want to send back a modified app

Cyrille-de-Brebisson commented 2 years ago

Hello,

Well... problem here... Basically, I do NOT know what the format is, because there is not really one... Let me explain, file load/save is done through an abstraction system which "describes" an in memory structure as a series of items of various types, located at various places in memory, with various IDs. It uses this to generate an output file which will have the id/type for each item. Then, the loading "attempts" do to the reverse, potentially transforming input data from one format (in is 16 bit, but 32 bit is needed in RAM? no problem, it handles it for you!)... Although this is great for my as it makes save/load much easier and allows updating from V1 to V2 easly... but not easy for someone like you who wants to reverse engineer the format! Note that programs now use the same system to save all the programs data....

But if you want, indeed, to decode apps, it will be a lot of work!

For example, this describes the persistance stuff for the finance app... if starts with a 32 bit number which is our flags. id=1 then a vector of 12HP_Reals (id= 2)... etc... now, when reading the file, the data can be in any order (thanks to the id), not everything has to be present. if id 2 in the input is a batch of 5 u32, they will be converted to 5 HP_Reals, and the others will we left to default...

const DATATYPE::Numb::TThisPersistedType DATATYPE::Numb::persisted_type = { PersistedType3(Numb, 0, 0, TPersistedTypeBase::f_UsePointerProvided, NULL, NULL, NULL), { Persisted(u32, 1, Numb, flags, 1), Persisted(HP_Real, 2, Numb, TVM, 12), Persisted(HP_Real, 3, Numb, IConv, 3), Persisted(HP_Real, 4, Numb, DDAYS, 3), PersistedS(MObj, 5, Numb::TCashFlow::persisted_type, Numb, CashFlow, 1), // cf! Persisted(HP_Real, 6, Numb, DEPR, 6), Persisted(HP_Real, 7, Numb, BRKEV, 5), Persisted(HP_Real, 8, Numb, PChange, 8), Persisted(HP_Real, 9, Numb, BOND, 9), Persisted(HP_Real, 10, Numb, BS, 8), }, { &Numb::TCashFlow::persisted_type.base } };

So, each "item" in the file starts with a 32 bit stuff which has that format... // This first u32 is what actually gets saved in the file!!!! u32 Type: 4, // Matches one of the TTypes above TypeModifiers: 2, // See modifiers above Compressed: 1, // set if data compressed TypeId: 15, // Either matches the current structure TypeId, OR matches a sub-structure thereof. A sub structure will be loaded and placed as needed. // Can also be set to 0 for facility. At this point, it is assumed to be current structure TypeId MemberId: 10; // An id for yourself. Assign one per saved data member of your structure. Do not reuse if member is removed. This does provide an indirection from file to data structure so that // you can load old/new files (what is not recognized does not get loaded)... // DO NOT USE 0, THIS IS A PROHIBITED NUMBER!!! (they get ignored in order to make sure that if you mess up and do not define all the members, you do not end up with a number of 0 members at the end of your definition!)

Cyrille