debrouxl / hplp

Windows / MacOS X / Linux third-party connectivity kit library / linking software for the newest HP calculators, targeting the Prime at first. It is strongly derived from libti*, https://github.com/debrouxl/tilibs , which implements communication with all TI graphing calculator families.
42 stars 8 forks source link

Q: Is this still an active project to be used with HP Prime G2? #6

Open gecko65 opened 2 years ago

gecko65 commented 2 years ago

Sorry more a question then a error report. Is that still an active project I? I would try to build it on a current Linux dist.

debrouxl commented 2 years ago

You could indeed try to build it on a current Linux distro, the code isn't supposed to contain weird constructs which ought to break with newer toolchains. However, I haven't worked on it or used it for years. I don't know whether the Prime G2 still supports the old protocol implemented by hplp.

gecko65 commented 2 years ago

So give me some time to check that. I will come back here and report, once I got that working. Thanks for your answer. Have a nice Advent season.

BeatSkip commented 2 years ago

@gecko65 any updates? Is it still working? I'm till looking for something to base my own library on

debrouxl commented 2 years ago

Your own use case might be different, e.g. embedded software / platform, but FWIW, nowadays, in order to maximize the usability for most end users, the best way to build software for [high-powered computer in the general sense] <- USB -> [graphing calculator, and many other USB devices] communication is to produce something based on WebUSB. The TI graphing calculators community has started doing that.

Beyond a bit of glue code, WebUSB-based software needn't be written in JS / TS: various other languages can be targeted to WebAssembly :)

BeatSkip commented 2 years ago

Well, so funnily enough, that's what I'm trying to achieve (an actually usable ppl and python IDE in c# Blazor webassembly with webusb).but to do that I need a reliable baseline communication library that I can copy the protocol/details off of as reverse engineering the entire workings is just too enthusiastic of a goal haha

Atm I have basic communication working (I can send a message to my prime) . But a working library will make life a lot easier. As even simple stuff like "how do I get a list of applications on the prime?" seems not too widely documented and any documentation spans over 8 years without any details on current status in modern firmware. (is it me or has the community on the prime died a bit? Such potential in this device, especially the G2)

gecko65 commented 2 years ago

@BeatSkip Unfortunately so far I did not get it working. I could built it. But once I try to connect the calculator with my linux laptop teh program does not find any valid connection.

gecko65 commented 2 years ago

I could see the hp prime G2 in the list of usb devices but hplp seams not to catch it. In the past I have had a sync problem with Palm and Linux. The solution was that first a initial transfer was needed with the Windows program and then the linux sync was able to communicate with the device. I know this is completely something different but maybe the hp prime also needs such an initialization for the communication?

BeatSkip commented 2 years ago

@gecko65 As i indicated before, i'm actually looking to bypass platforms entirely and go directly to Webassembly/Js with WebHID. Initial tests seem promising, as soon as i've got a basic proof of concept working i'll put up a public repo and leave a link here. you can keep an eye out for that if you're interested. currently i'm just using this code as a rough baseline indication and use PrimeComm as the actual basis to work off of as that seems to mostly be working with the G2 ( at least my tests for now seem to work). It's just that functionality in PrimeComm is very limited.

debrouxl commented 2 years ago

See also #5 (new USB PID) and #4 (newer variant of the protocol).

gecko65 commented 2 years ago

@BeatSkip Yes I am in deed interested in the results of your future repo.

gecko65 commented 2 years ago

@debrouxl Should I start an issue with the error/logging message of test_hpcalcs?

metawops commented 2 years ago

I, too, experimented with this in 2020 because my plan was to directly connect a Prime via cable to an Apple iPad Pro's USB-C port. I wanted to make an app that was able to take screenshots from the Prime's screen into the iCloud Photo Library. – Never was successful with that. However, Cyrille de Brebisson – who was an HP Prime team member – was very open about technical details regarding the USB protocol and supplied me with details in emails. This was in March 2020. I'm quite sure he would be supportive today, too. Just wanted to drop this in, in case somebody needs help. Besides from that I'd very much be interested in testing any efforts made here. My platform is macOS & most of the time Safari (which doesn't yet support WebUSB). But I have Chrome & Firefox, too. => I'll watch this space or any folloing/spawned repositories. 😉

debrouxl commented 2 years ago

@gecko65 : for the benefit of others, indeed, but first of all, try modifying the code ( libhpcalcs/src/hplibs.h & libhpcalcs/src/link_prime_hid.c ) to look for the USB PID indicated in the lsusb output on your computer, in addition to (or instead of, but that's less good) the existing ones. In #5, @metawops mentioned a 0x2441 USB PID.

adriweb commented 2 years ago

@metawops: would you be able to share the/some information from Cyrille? I don't know how easy it would be to reach him nowadays considering HP Prime development got transferred out of HP.

Although maybe Lionel got some from him already, but I'm not so sure. And that would have been much older than 2020.

BeatSkip commented 2 years ago

@metawops that would definitely be great! I'm almost done setting up my base repo. Hope to get it live tonight with a working simple communication message from the browser and I'm planning on taking it from there. But an open source up-to-date communication platform is definitely something the prime needs to get it's community a bit more lively with more people. And input from the devs would definitely be appreciated

gecko65 commented 2 years ago

@debrouxl Yes I also have a 0x2441 USB PID. I changed that in the code (thanks for you deatilled hint). The behaviour already changed but I still do not get something like a menu. I am still checking if I missed something.

gecko65 commented 2 years ago

@debrouxl Ok got it working - at least now as root (1st step) I get the menu. Now testing the test functionality :)

debrouxl commented 2 years ago

Good to hear that you can at least get the menu :)

BeatSkip commented 2 years ago

for those interested, i've got the repo up https://github.com/BeatSkip/PrimeDev now working to integrate my local test to work with the WebHID library instead of the Windows HidLibrary. But anyone interested should feel free to engange and pitch in :)

metawops commented 2 years ago

Let me try to reach Cyrille first (already sent out a ping) and check if it's okay to publish his infos here. When I asked him for those infos I promised I wouldn't publish it anywhere ... But I'll check old eMails, too, and see if we talked about this, too. Maybe he mentioned that it would be fine to publish it and I forgot about that. I'll report back with news.

gecko65 commented 2 years ago

@debrouxl now sending a chat (Hello World) works. Receiving a chat works in the DUMP. I get a 00 F2 01 00 00 00 <LEN> <CHAR1LOW> <CHAR1HIGH> ....

BeatSkip commented 2 years ago

@gecko65 FYI, I managed to get the initial message to my prime working! What's your status? Any updates? And @metawops did you hear back from cyrille? Any more info on the protocol would now be appreciated as I've got the initial communication working in the browser.

gecko65 commented 2 years ago

@BeatSkip : Yes as I already wrote I was able to send "chat" messages to the Prime. Receiving chat messages work only as a DUMP (need to check that).

metawops commented 2 years ago

@metawops did you hear back from cyrille?

Not yet, unfortunately, no.

BeatSkip commented 2 years ago

@gecko65 getting the messages back is pretty simple in principle. Not sure how this couples into hplp but in my tool I've got it working byparsing the return data as: Data[0] = 0x00 Data[1] = 0xF2 (data packet identifier, 0xF2 is 'chat message') Data[2] = 0x03 (docs say count of something, but primecomm and hplp are looking for this as fixed 0x01, but with modern g2 appears to be always 0x03) Data[3..6] = message length in bytes Data[7.. +7] = utf-16 message bytes that you can easily parse as a string (little endian)

It seems to me that the packet buffer is always 1024 bytes and any data after your message is just old data that hasn't been overwritten. So you can just discard that in your parsing. Still checking about any crc stuff that should be in the message.

Edit: what seems weird to me is that your dump has the third byte as 0x01 but for me it is always 0x03, unsure what that is about

gecko65 commented 2 years ago

@BeatSkip I also did identify Data[0] Data[1] as &00F2 and Chat identifier Data[2] == 0x03 I will check that on my device Data[3] Data[4] I also have seen that as message size (2 bytes per UTF16-character") Data[nn] Data[nn+1] is the UTF16-character

So far it seams my messages are cut, if the datagram exceeds 64 bytes. The longest messagte I was able to SEND as paylaod was 30 characters (in UTF-3 This are 60bytes).

gecko65 commented 2 years ago

For a received message "789456123" I get 00 F2 01 00 00 00 14 37 00 38 00 39 00 34 00 35 00 36 00 31 00 32 00 33 00 00 00 00 88 00 00 00

BeatSkip commented 2 years ago

@gecko65 I have no such limitation, i can send a full string as long as i want. just tried to send over 2000 characters. the first 508 are sent fine to the calc. and after 508 characters [note: that's around 1024-ish bytes of the max packet size] the text gets garbled due to a bug in my chunk splitting code (but that's just an error in my code).

image

and for return i can receive long strings fine too.

image

gecko65 commented 2 years ago

Get Info from prime has the following format 00 FA 01 00 00 00 6C 00 00 00 00 00 00 00 00 00

Sending time and date has the following format 00 00 E7 01 00 00 00 0A 00 00 54 1E 16 02 0C 16

Get a screenshot begins with 00 FC 01 00 00 46 29 9C 9A 08 FF FF FF FF 89 50 Fails, because of expected 17942 bytes but only got 1134 bytes, output corrupted

Get a backup 00 F7 01 00 00 27 28 00 1E 05 AB 63 00 61 00 6C Fails: packet out of sequence, got 0, expected 10

Example for send Key 00 00 EC 01 00 00 00 01 17 Need to check for Key-IDs ;-) Maybe in HPs Prime documenation

gecko65 commented 2 years ago

@BeatSkip Which firmware version do you have? My is 2.1.14538 (2020 12 17) Hardwareversion: D OS: V2.060.650

BeatSkip commented 2 years ago

you're out-of-date it seems image

debrouxl commented 2 years ago

FWIW, back in the day, I could reliably transfer multi-kilobyte-sized variables. For a long time, TILP has unconditionally redirected terminal output to a file on Windows for the same reason as hplp, namely the slowness of the classic Windows terminal. With TI graphing calculators in USB vendor-specific class using USB bulk transfers, terminal slowness does not adversely affect transfer reliability; with the Prime in HID mode using USB interrupt transfers, it clearly did...

You can see occurrences of that 0x03 byte value in PR #4 .

BeatSkip commented 2 years ago

@debrouxl aha, so hplp does account for the 0x03, then it's just PrimeComm that's looking for the 0x01. Currently i'm watching through a couple conference video's that reveal some source code of the Prime. and i'm starting to create some up-to-date documentation of the prime protocol. as there's nothing up-to-date with more than the basics it seems.

but it seems that apart from my bug in packet splitting, i can just transfer multi-kilobyte packets without issue.

adriweb commented 2 years ago

Even though it's very old, maybe https://tiplanet.org/hpwiki/index.php?title=HP_Prime/Linking_Protocol could be updated?

BeatSkip commented 2 years ago

@adriweb yeah, from what I gathered around there's a tonne different with the modern protocol. That document is last updated in 2013

metawops commented 2 years ago

Even though it's very old, maybe https://tiplanet.org/hpwiki/index.php?title=HP_Prime/Linking_Protocol could be updated?

Oh, come on, please not on a TI website! 🤦‍♂️ There‘s plenty of HP calculators dedicated websites like hpcalc.org or hpmuseum.org (has a forum) or the Wiki at www.wiki4hp.com/doku.php?id=prime:start, to name just a few.

adriweb commented 2 years ago

Oh, come on, please not on a TI website! 🤦‍♂️ There‘s plenty of HP calculators dedicated websites like hpcalc.org or hpmuseum.org (has a forum) or the Wiki at www.wiki4hp.com/doku.php?id=prime:start, to name just a few.

When this was created, there was nowhere else to actively talk about the Prime, much less in a good wiki (mediawiki-based, anyway) way. And it's not like the Prime community is very active, it's like 0.5% of what the TI community does, so.... But regardless, TI-Planet is not TI-only-focused, we literally deal with all brands, and the HP Prime won twice the back-to-school comparison guides, if you needed some more proof that it's not biased towards TI.

BeatSkip commented 2 years ago

@adriweb you are totally right, it seems to me that that little bit of community that the hp prime had is slowly bleeding to death. And honestly I think that is at least partially due to the fact there is so little and vague documentation and absolutely no good tooling to make the prime do what it's capable of. That's what I'm trying to fix with at least proper tooling. Hopefully it'll live up a bit when more people have easy access to their calculator. Just sad that the 'risk of influencing the examn mode' chokehold is limiting any progress, while I'm just toying with my hardware as an engineer.

debrouxl commented 2 years ago

Besides, that very wiki4hp page links to the TI-Planet hpwiki , both the root page and the USB protocol page :)

Your goal is laudable, but you probably know that you're facing a huge uphill battle creating significant activity for the Prime series' superior hardware and good, in some aspect superior software ;)

BeatSkip commented 2 years ago

@debrouxl oh yes, I know it'll be pretty much impossible to bring life into it. but als long as i see it as a fun and interesting challenge to make this tooling and some people appreciate the result. that's good enough for me :) even if it's just me appreciating a better tool, that's also fine haha

BeatSkip commented 2 years ago

@gecko65 what's your current status? Just got multi-part compressed messages via the newer protocol working! The newer protocol is mostly a wrapper around the old protocol with some additional checking and acknowledgements. Now working on the actual commands and features running on the protocol.

BeatSkip commented 2 years ago

@debrouxl if you are interested in maybe adding some newer features to your library, I can give you some explanations about the newer protocol. Still working out some final details on the CRC checking and I'll eventually document it. But that may take some time.

metawops commented 2 years ago

Guys, I was thinking ... wouldn't it be way cooler to have a VSCode Extension that allows us to freely write code for the HP Prime (and maybe even later other HP calcs, like the 48/49/50 series) with syntax highlighting and transferring it to/from the physical calculator? With enough effort I think even the full functionality of the HP Connectivity Kit could be integrated into VSCode. Just a proposal ...

BeatSkip commented 2 years ago

@metawops well, so that's pretty much what I'm going for. But then without vs code, no installation, no drivers, in your browser. Same text editor as vs code (Monaco editor also has a Web based version). Basically connectivity kit + ide with just opening a link. And native install with one click possible as pwa :)

metawops commented 2 years ago

@metawops well, so that's pretty much what I'm going for. But then without vs code, no installation, no drivers, in your browser. Same text editor as vs code (Monaco editor also has a Web based version). Basically connectivity kit + ide with just opening a link. And native install with one click possible as pwa :)

Yeah. But not inside VSCode which is THE code editor of tons of developers already. Plus, you have to code things like syntax highlighting yourself, from scratch. While in VSCode you'd define a language description and that's it. I have VSCode open all the time and when I decide to write a little HP Prime program I'd have to switch to the web browser and open a new URL. Well, it was just a suggestion. I don't have the skills to write a VSCode extension. By all means, Oscar, do the thing you want to do! No problem! I was just suggesting. ;-)

adriweb commented 2 years ago

The thing is that when a stand-alone thing is ready and works as expected, then it can be used to make other things, including, why not, a component of an IDE extension, sure :)

debrouxl commented 2 years ago

Exactly. It may be possible, even easy, to shoehorn a library for WebUSB-based communication (preferably JS/TS for WebUSB glue code, and another programming language which can be targeted to WebAssembly) into a VSCode extension, but said standalone library needs to exist first.

Tons of other developers do also prefer code editors not based on heavyweight Web technologies :)

BeatSkip commented 2 years ago

@debrouxl you're right, it's trivial to port my code over to work on a native platform and make a VScode extension for example. i've made an abstraction layer between my current protocol layer and the actual WebUsb stuff. so once this is done, i'd say within two hours the whole protocol library can be converted to run on windows and use native Hid libraries to communicate with the calculator. as the actual protocol library is purely written in C# (that is just so happen to compile to wasm atm, but no reason you can't just run it as normal). the library that couples my prime protocol library and the actual browser webusb stuff is actually 99% compatible with https://github.com/mikeobrien/HidLibrary that runs on windows. in a later version i'll make it 100% compatible so people can just choose how and where they want to run the library.

I'm planning on separating my web based code editor project and the actual prime communication library once i'm at a working alpha state. (they're already separate projects, just in the same git repo).

debrouxl commented 2 years ago

Nowadays, C# has better portability than it had in the mildly distant past, as long as one does not use OS-specific calls, obviously, be them for Windows or other OS.

And yeah, multiple projects in the same Git repo are easier to handle in the beginning than multiple repos and submodules, they can be split to separate repositories later as time permits and the rate of change decreases.

BeatSkip commented 2 years ago

@debrouxl yeah, I used to be full on c++ for platform flexibility and performance. But c# is going places and the ease is so convenient!