More fallback patterns, documentation?

[5ch4um1]

Hi Tom. Finally managed to build one, based on a wemos d1 mini. Tried to compile with the eps8266 boards manager thing, to no avail, but worked fine with another boards manager that i had used with another project ( esp8266-deauther). Still trying to figure out how to talk to them though, any chance there is some documentation on the post requests i need to send in order to make it do things? Also tried the app on android pie, haven't looked into it too much so far, the fact that it creates a bunch of folders with patterns is a little bit annoying imho. The only thing i could make it do so far was obviously change address (whatever that means, change the address of the eeprom?) which somehow put them into a non working state. i read something about udp streaming? Is there an open port then and i could just shovel data to the poi with netcat? To be honest, i haven't looked into that too much yet and if you are busy i will probably have some fun finding out by reading your code, wireshark, nmap, curl and/or netcat, so no worries, just thought i'd ask if there is a manual that explains the networking aspects a bit more in depth? (i read the old manual, btw.)
And i wanted to ask if you would mind if i post about this project on social networks like facebook or reddit, i think it would be really nice if we could get a few more people to actually build those things and take some fotos, contribute patterns (Fire2012 looks really nice btw.), enhancments, etc. I've see a staff project on facebook with accelerometer e.g. BPM and microphone permission means it could listen to the phones mic and make the poi react to music i suppose? That's also really nice! Here some fotos of my first outdoor test, just one poi and only the fallback patterns, only ws2812 but looks pretty nice already:

[tomjuggler]

I'm afraid that I haven't gotten the app to work on Android Pie (9) yet. I am sure I mention it on the circusscientist.com website, Android Nougat (7) is the last supported version as of now. I do have a device which is running Android Pie so at some point I will look at that again for sure, but Google has changed something there to do with networking (probably for security reasons).

To answer a couple of your other questions: The images and folders are there for you to put your own images inside, to stream to the poi. Unfortunately the app crashes if the folders are empty, so I create and fill them for you on first run. UDP streaming works by connecting your phone to the 'main' poi (they connect to each other as well) Access point. You should see the network called "SmartPoi" or something like that? Changing the IP addresses I thiink I put that there for multi-poi setups, involving a router, maybe don't touch that. If you can get the app to work (with an old Android device for now) and connect to the poi hot spot, select 36px or computer generated and the images should stream from your phone to the poi in real time. then select "upload image" to upload the same images to the poi for offline backup.

I am a bit busy with other projects right now, some documentation does sound like a good idea, though. Good luck for now.

[tomjuggler]

I forgot to mention, there is sound reactivity in the app ('Beats') menu option. I have gotten a new job doing a lot of server development, so I plan to use that experience to create an online sharing aspect for the app - so you could import images from the community from within the app.

Feel free to share, just make sure to point people at the website, as I usually update that first.

[5ch4um1]

i have to admit that my knowledge about http is pretty limited, only thing that comes to my mind about post requests is sth. like: that's not safe, avoid them...? And well, i was actually interested in contributing to a project that is in active development, or find some people who would be willing to colaborate on an "up to date", easy to use and easy to build and most important, easy to understand pixel poi project. And since Android is always a moving target, have you thought about a simple web-app? And maybe selecting the images you want to upload from the gallery? And the answer regarding the udp streaming was actually not the type of "information depth" i was looking for, of course i saw the ap, connected to it with both laptop and phone, but as i said, haven't looked into the sketch nor did i do a portscan. I just thought it would be nice to understand how it works, and since it seems to be just http requests, documentation of the communication would be a lot more important that an app, imho?
I'd definitely start with a browser based solution and start adding features from there?

[tomjuggler]

Just wanted to say it's great to hear that you are so interested in how everything works in this project.

"Just build a web app" is not, unfortunately, an option I am afraid. This poi project relies on streaming of UDP packets, which cannot be sent from a browser based app. See this Stack Overflow answer for confirmation of that: https://stackoverflow.com/questions/48739093/how-to-send-udp-packet-using-javascript-to-local-network-broadcast - I did consider it before. The reason UDP streaming is so cool is that it enables real-time computer generated images to be deployed to the poi "screen", not just static images which are repeated over and over.

The full source code for the Android app is available here: https://github.com/tomjuggler/SmartPoi-Android-App

I have done a simplified working version of the UDP streaming setup - with code (in Processing), firmware, and even an emulator to show you how it works: https://www.circusscientist.com/pov-wifi-streaming-emulation/

I even implemented it in plain C: https://github.com/tomjuggler/SmartPoiGenerate (this one really flies on a modern PC!)

If you want to take a look, here is the bit in the Android app where I compress the images to .bin files and use http to upload them as backups to the poi: https://github.com/tomjuggler/SmartPoi-Android-App/blob/c4fbbdf22f17fb539f591b21668014407376f2d6/app/src/main/java/za/tomjuggler/processingdemo/sketch/manyScreensAndroidFlick72px.java#L1137 Maybe you could work out how to do this part in JavaScript, it should be possible..

Also, the IP of poi number 1 is 192.168.1.1 and number two is 192.168.1.78. The UDP port to send to is port 2390

As I said, when I have the time, I will be updating the app to work with Android Pie. It's probably something simple that is tripping it up, after all the app does run.. but no I won't be re-writing the whole thing as a web app any time soon. Some documentation of the new app features will also be forthcoming at some point, I promise!

[5ch4um1]

How about websockets then? Found a simple fastled webserver sketch online, this fits my needs for now a lot better, as it actually works and allows to change patterns and brightness, added some of your code to display an image: https://github.com/5ch4um1/FastledAccessPOInt This would make it very easy to add patterns that are based on code, next would be to have a client poi with a fixed ip and modify the main poi so that it talks to the client poi too, i guess that should not be horribly difficult. And then, yeah, live streaming would be nice, but basic functionality is also a pretty nice thing to have? like, something that actually works out of the box and more universally, not only on android? i'm not an apple fanboy, but browsers would be nice?

[tomjuggler]

OK that makes sense. In the smart poi firmware, uploading and changing "banks" of images to display are implemented via http requests, so it's just the UDP streaming that won't be possible using the browser. The Processing code to prepare and upload images could be ported to the web, and the same code can be run inside a browser or android app.

You might find another project I made interesting here: https://www.circusscientist.com/wifi-controlled-christmas-light-window-decoration/

Which is similar to the one you linked, using a web app hosted on the ESP8266 to change patterns.

OK I will look into this, I've been porting a lot of my apps to P5 lately (the JavaScript version of Processing). Thanks for the ideas!

[tomjuggler]

Here is how to change the pattern that gets displayed - I included 20 .bin image files in the firmware, so if you did LittleFS upload you should have those available. Connect to the poi, go to http://192.168.1.1/pattern?patternChooserChange=1 (in a browser) for on-poi generated patterns, or =2 for images 0-4, =3 for images 5-10, =4 for images 10-20, or =4 for all images (up to 52 images currently supported). Once again I need to document all of the functionality. I was too busy building it out at the time.

This functionality should be easy to incorporate a simple web app, hosted on the poi itself, or locally on pc or phone. Then we just need a local file chooser and converter (.jpg to .bin) and an uploader.

[tomjuggler]

I forgot, http://192.168.1.1/pattern?patternChooserChange=6 should set the poi to loop through the other options 1-5, changing every time you re-start the poi.

[tomjuggler]

Good news, I found the problem with Android 9 - should be able to put out a fix for this in the next week or so!

[tomjuggler]

Ok the Fixed version which runs on Android Pie fine for me is up - I had to create a new branch, called mainGithub. If you just want to get on and play with it, the apk is here: https://circusscientist.com/SmartPoiPieFix.apk Please let me know if it works Pretty stoked that it was so easy to be honest. Thanks Schaumi for the motivation.

Now for that documentation you asked for...