Wifi?

[jhubbardsf]

Hi, it doesn't look like this thing has wifi. Would there be any big issues with switching the main chip to an ESP32 which would have built in wifi and bluetooth?

[joeycastillo]

Thanks for the suggestion! I sense that it might be possible to build this around an ESP32. The eBook Wing actually works with an ESP32 Feather so I know that this is within the realm of the possible. And a lot of the Arduino code would be totally cross platform. But some of the more advanced use cases, like TensorFlow for voice commands or MP3 playback, use SAMD51-specific timers and registers and whatnot, so in terms of software, trying to switch to the ESP32 would be a pretty big change.

For the moment I'm imagining the main use case as reading books or listening to audiobooks that are stored on a microSD card, and plugging the book into a computer with wifi so you can manage the content on the card. For on-device WiFi, you can add an AirLift FeatherWing; it plugs into the Open Book like a backpack. I may eventually add this functionality (ESP32 as wifi coprocessor) to the main board if folks seem to find it useful; it's all just a balance of functionality and cost.

[jhubbardsf]

I've never programmed for the SAMD51, is its framework RTOS based? The ESP32 uses ESP-IDF (well it can use whatever, but that's the official framework for it) which is based on FreeRTOS and can do almost anything. The ESP32 chip is only about 2 bucks so cost wise it's not a lot more. And it could completely replace the SAMD51 I believe. I'm not sure the time that would take to refactor. I've ported code from Arduino to FreeRTOS/ESP-IDF code a couple times so if you ever decide you wanna add wifi capabilities feel free to ping me. I'd be happy to help in my spare time. Cool project!

[ian-tedesco]

I don't want to create a whole separate issue for this since it probably will never be done, or only in a long, long, long time, so I'll ask here. If wi-fi is implemented with that chip, would it be possible to build a very basic UI that connects to Lichess? It's just I always imagine playing on an e-reader would feel great.

[jhubbardsf]

I just checked out Lichess because I wasn't familiar with it. I don't see why a basic UI wouldn't be able to be built for the ESP32 that communicates with the Lichess API. So I'd say very possible and fun!

[ian-tedesco]

It's fully open source and privacy friendly, it is a great project. I can imagine the pixel piece set looking awesome on the Open Book, lol. I like this because I feel that I'm not fully concentrated playing on my PC since there are so many things that can distract me, playing on an e-reader would be perfect, I hope this could be done some day.

[jhubbardsf]

With the quarantine going on I've been a bit bored lately. Maybe I'll throw some time behind the ESP32 port of this device. Unless someone gets to it before me I'll draw up the schematics and PCB design for the ESP32. I'll keep this thread updated.

[joeycastillo]

I think this is a great idea! I'd also consider checking out the new ESP32-S2 module if you're looking at porting it over; it's a single core, but the low power stuff seems like a big win for an ebook use case, and it has even more GPIO than the previous ESP32. I sense we're going to see an explosion of gadgets based around this chip soon :)

[ian-tedesco]

With the quarantine going on I've been a bit bored lately. Maybe I'll throw some time behind the ESP32 port of this device. Unless someone gets to it before me I'll draw up the schematics and PCB design for the ESP32. I'll keep this thread updated.

That's really cool, I hope you do that and can implement it, I would love to see something like that and being able to play chess on an e-reader and other cool stuff.

[val-storm]

Some friends and I are planning to make a wifi port of both the board and the software if necessary. Our interest is based in the fact that we are currently making a very, very large data set of plain text books formatted as json behind a REST api or something similar. Basically, the data set will be used for fun ML projects, but also need a lot of editing due to all the automation in the pipeline. So, we are hoping to have these able to search and query the api like a massive library or wiki. "Volunteers" welcome to "find errors" in our huge "plain text data set".

Any progress on the wifi enabled board? Is there a specific Feather that is wifi and plug and play with current board? This device strikes me that it could take an approach similar to the ornament and crime project https://ornament-and-cri.me/user-manual-v1_3/ ...it's an instrument/tool, but has a few open source apps that can be selected... I think it might be interesting to go from say, playing a game of lichess with a friend, to reading streamed content from a 200,000 book plain text archive... to...??

Let me know if this sounds cool and if we could plug in anywhere. Great project!

[joeycastillo]

Awesome!! For my part I have not made any moves on a wifi version of the board; I've been instead focusing on software support for the existing hardware, as well as some experiments with new screens. In the back of my mind this does feel like a perfect ESP32-S2 project, but what's stopped me there is that board support for that module is just getting started.

Still, both the Open Book and the E-Book Wing are compatible with the Adafruit AirLift FeatherWing, which uses the original ESP32 as a wifi co-processor via the Feather's SPI bus (plus pins 11, 12 and 13). That would get you started with wifi on this device, right now.

It might be a tight squeeze but if you removed the Feather headers and moved some stuff near the lock button, you might be able to fit an ESP32 onto the Open Book board (and maybe even give it its own dedicated SPI bus with the freed-up pins. This is the approach Adafruit takes with their PyPortal device, which uses the well-supported SAMD51 as the main processor, and has the ESP32 available for wifi stuff. This has the added benefit of Adafruit's really excellent software support for this use case.

Let me know if there's anything I can do to help! FWIW I think both options are sound, I just haven't had the bandwidth to explore them :)

[val-storm]

Okay. Great. I guess for simplicity we'll get started with the wifi shield. and hopefully we don't paint ourselves into a corner.

we have kicad experience and could start building a kicad version for future use. is the goal to forgo dev boards and have the mcu on the open book pcb?

We're really excited about the overlap of these two projects. I think the ability to query an all text internet library--i mean data set--would be a cool easter egg in the firmware.

oh, and, what state is the voice command stuff? what model were you planning on using? we might be able to help there if need be, because given the limited controls we were thinking voice search would be helpful.

Best,

On Tue, Aug 18, 2020, 1:17 PM joeycastillo notifications@github.com wrote:

Awesome!! For my part I have not made any moves on a wifi version of the board; I've been instead focusing on software support for the existing hardware, as well as some experiments with new screens. In the back of my mind this does feel like a perfect ESP32-S2 https://www.espressif.com/en/products/socs/esp32-s2 project, but what's stopped me there is that board support for that module is just getting started.

Still, both the Open Book and the E-Book Wing are compatible with the Adafruit AirLift FeatherWing https://www.adafruit.com/product/4264, which uses the original ESP32 as a wifi co-processor via the Feather's SPI bus (plus pins 11, 12 and 13). That would get you started with wifi on this device, right now.

It might be a tight squeeze but if you removed the Feather headers and moved some stuff near the lock button, you might be able to fit an ESP32 onto the Open Book board (and maybe even give it its own dedicated SPI bus with the freed-up pins. This is the approach Adafruit takes with their PyPortal device https://learn.adafruit.com/adafruit-pyportal?view=all, which uses the well-supported SAMD51 as the main processor, and has the ESP32 available for wifi stuff. This has the added benefit of Adafruit's really excellent software support https://learn.adafruit.com/adafruit-airlift-featherwing-esp32-wifi-co-processor-featherwing?view=all for this use case.

Let me know if there's anything I can do to help! FWIW I think both options are sound, I just haven't had the bandwidth to explore them :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/joeycastillo/The-Open-Book/issues/15#issuecomment-675690884, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJZR6ZA5U5DAWSLHJ5HSFFTSBLOT7ANCNFSM4J6CKUSQ .

[joeycastillo]

As it stands now the Open Book does have the MCU on the board (SAMD51J19A); the Feather header is there for compatibility with wings so that folks can add functions and test out other use cases. But you could for the most part just remove the wing connectors and replace that area with whatever bits of whichever wing you wanted to use. Although with the ESP32 you'd probably want to have the antenna closer to the edge and add a cutout in the ground pour.

It does sound like an exciting concept! Stoked to see what y'all come up with. As for voice commands, I will say that where last I left off, it's a bit less versatile than what might be required for voice search. The Open_Book_Heart_Of_Darkness_TensorFlow demo uses TensorFlow Lite and a model trained with just three words — "Left", "Right" and "Go" — to navigate a menu of items for turning pages. I haven't touched that example since late last year but this guide has some notes on training new models. Still I'm not sure that this MCU has the oomph for a full voice recognition model.

[val-storm]

Hey again,

So I've cleared up some project head space and am getting back to Open Book (and the secret "Last Book" web archive) and I have some questions.

You were out of the Open Book PCB so I decided to use it as an opportunity to try out this "JLPCB" business that seems to be everywhere right now. The boards look okay and were insanely cheap (which has to be bad in some way...). So I have 5 boards and I was gonna eventually build them all or pass them off as kits to friends... but here is the first question:

There are a few parts which digikey doesn't have in stock and I was wondering about them...

Battery Connector Q1 Q2 D2, D3, D4 FB1, FB2

Would an Octo-Part search to find the exact part numbers be wise? I did some digging and most of them I was able to find comparable parts. For instance the Battery Connector at a different angle. The one that gave me some trouble was Q2 I think... I didn't know what was the limiting agent there as far as specs go. What am I looking for with that one?

Very excited to get one of these built and start moving on the networked library aspect! Also thinking about making a larger one... but this size is really great... its like a phone.

Best,

On Tue, Aug 18, 2020 at 7:30 PM joeycastillo notifications@github.com wrote:

As it stands now the Open Book does have the MCU on the board (SAMD51J19A); the Feather header is there for compatibility with wings so that folks can add functions and test out other use cases. But you could for the most part just remove the wing connectors and replace that area with whatever bits of whichever wing you wanted to use. Although with the ESP32 you'd probably want to have the antenna closer to the edge and add a cutout in the ground pour.

It does sound like an exciting concept! Stoked to see what y'all come up with. As for voice commands, I will say that where last I left off, it's a bit less versatile than what might be required for voice search. The Open_Book_Heart_Of_Darkness_TensorFlow https://github.com/joeycastillo/The-Open-Book/tree/master/Examples/Open_Book_Heart_Of_Darkness_TensorFlow demo uses TensorFlow Lite and a model trained with just three words — "Left", "Right" and "Go" — to navigate a menu of items for turning pages. I haven't touched that example since late last year but this guide https://learn.adafruit.com/how-to-train-new-tensorflow-lite-micro-speech-models?view=all has some notes on training new models. Still I'm not sure that this MCU has the oomph for a full voice recognition model.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/joeycastillo/The-Open-Book/issues/15#issuecomment-675815194, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJZR6ZBZ226ZKBHHYIBQESTSBM2ODANCNFSM4J6CKUSQ .