Feature request: Image loading cycle app

[johnhtodd]

I am interested in how multiple images (GIF or PNG would be ideal) could be loaded in by the kublet in timed sequence.

My preference would be to have an app that, when launched, would download a pre-specified JSON file from a static location (configured by the app? hard-coded seems... ugly.) That JSON file would then contain an array with data to describe:

• URL of graphic
• duration of graphic on the screen in milliseconds

There would also be a JSON object (singular) in the file that would describe how long that JSON file was valid before it would be re-loaded.

The app should not stop displaying or cycling graphics while it is checking status for new JSON file download. If there is not enough memory for new file downloads, then the graphics that were played the longest ago should be deleted and the new frames downloaded. There should be no interruption of display unless download time of all new frames exceeds total display time of all existing frames (this means interleaving the "download" loop and the "display" loop into a single loop - tricky!) and as soon as all new frames are downloaded they should start displaying immediately without any remaining old frames being displayed.

Failure of download of JSON should result with some sort of visual indicator (a red "FXXX" in one corner where XXX is the number of seconds since first failure) displayed on all graphics. Existing graphics should continue to be displayed in the event of a JSON download failure, or if it fails on the first iteration, just "FXXX" should be displayed in the corner. If JSON fails to download on first iteration, attempts should be made to re-download every 10 seconds.

Frame downloads should be "smart" and check modified-time of the remote URL if it is identical to a frame that is currently in memory. If the frame modified-time and size is identical, then it should not be downloaded. This will be a significant savings in download time for many rotating graphics such as network monitoring, camera display, or other image arrays where all frames will remain the same except for one that will change in an interval.

If an individual graphic frame fails to download on first attempt, then every time the graphic position is in line for display cycle, it should be re-attempted for download. If it fails, a red "FAILED DOWNLOAD " should be displayed for the number of seconds indicated by the JSON file where URL is the URL for the image.

[johnhtodd]

I suppose this is a very optimistic request, since I don't know how much space a GIF/PNG might take, and I don't know how much memory is allocated in the ESP32 that you are running. What is the usual size of a frame? For that matter, what is the resolution and bit depth of the screen?

[johnhtodd]

Another "wish list" but which is probably too sophisticated for a starter app: instead of an app-configured URL for the JSON file, it could exist in a DNS record. The app could wake up and do a request for "_kublet-rendezvous.local. TXT" which would contain the URL. A keyword of "[$MAC]" would be replaced by the MAC address of the Kublet. (Note that the use of ".local." TLD does not imply MDNS, which is much more complicated and probably un-necessary - see https://en.wikipedia.org/wiki/.local)

Example:

1) Device boots. It gets a DHCP address from the local DHCP server, which includes the DNS resolver of 192.168.1.1

2) Kublet sends DNS query of type TXT to 192.168.1.1 for record "_kublet-rendezvous.local."

3) 192.168.1.1 is configured to reply back with contents of TXT record of: "http://192.168.1.100/kublet.php?deviceid=[$MAC]"

4) Kublet then connects to 192.168.1.100 with request "GET /kublet.php?deviceid=AA:BB:CC:11:22:33" where the device ID is the MAC address of that particular Kublet, so the remote server would know what JSON configuration file to reply with.

This would mean that dozens of kublets could be activated on a network with no special configuration options from the app - just the app would be downloaded and then it would be done, forever. All other configuration would come from the local DNS and web servers.

[johnhtodd]

Further comments: I devised this idea for use by Grafana. I know that the preferred method would be to feed Grafana data to the Kublet via a JSON file, but that probably would be very difficult with most business systems, since having authentication by the kublet would be a challenge. Using graphics (via the image exporter plugin of Grafana) seems to be a way that the data can be extracted and transmitted to a "DMZ" system without having the kublet talk directly to Grafana or whatever the backend database is.

Also, using graphics transmitted at PNG means that the full visualization toolkit of Grafana can be used instead of a limited set of primitives inside of the Kublet - every possible graph, plot, map etc can be rendered using a transmitted graphic, where I suspect a Grafana integration that pulls data will be very limited to just a few line charts and changing the format of any graphs will require C++ code instead of this simpler method which just requires modifying a central Grafana output image(s).

However, this would work for displaying weather maps, temperature, stock prices, or even slow-motion animations. Almost any visual information that can be developed by some more sophisticated head-end process could be displayed either as different frames of data in sequence, or as an "animated" frame set that shows change over time or an action/motion of some sort.

[BStrauss3]

Pretty sure the answer you will get is "go ahead and write it, we'll be happy to host it here".

[johnhtodd]

Pretty sure the answer you will get is "go ahead and write it, we'll be happy to host it here".

The devs asked in the Kickstarter forum for ideas to be posted for next apps, and said they would take them in order of interest. At a minimum, I asked some questions in my comments, and anyone (including myself) who is interested in coding such an app would need those answered.

[audreylim]

@johnhtodd thanks. Let's start with what is required in order to display the gifs/images:

• Each frame/gif has to measure 240x240 pixels.
• The image has to be in JPG format and RGB color coded.
• To display images/gif, we'd need to convert the binary to C-like hexadecimals arrays: eg.

const uint8_t img[] PROGMEM = {
    0x47,0x49,0x46,0x38,0x39,0x61,0xa0,0x00,0x78,0x00,0xf7,0x00,0x00,0x09,0x0c,0x08,
    0x26,0x25,0x26,0x34,0x34,0x34,0x1c,0x25,0x18,0x09,0x79,0x00,0x1a,0x56,0x06,0x5e,
    ...
}

Image conversion

There are 2 ways we can do this:

1. Have the user convert the image/gift to hex arrays with an online tool, then upload the hex array onto a configuration input field on mobile, or
2. User uploads image on our mobile app and we do the conversion

Option 1 is definitely easier on development, but is not very user friendly. What do you think? I'm currently inclined to 2, but it does take more development time.

The kublet should have enough space to store at least 5 images (and even more but I'll have to confirm this). Let's assume it can store 5 images.

Storing images on kublet

There are a few ways we can store images on kublet:

1. Have the user drop the image/hex array in the configuration input fields on mobile, then the mobile app sends them over BLE. This approach is fixed, ie. the user will have to upload new images should they wish to switch images. However, assuming the images we can store on kublet goes up to 15-20, this shouldn't be a problem for most use cases.
2. If we take your JSON file approach, we'll need to decide if the user wants to provide their own URL for the json file, or store the json file on our server. The first approach requires a little technical skill on the user's part, but for more technically advanced users this gives them a lot of control. Either way, if the user doesn't want to host the images themselves, we would still have to host them on our servers, and they will still need to upload the images on our app.

It does sound like the JSON files approach will be very flexible for advanced users and people who want to show image screenshots though, especially with the Grafana images. Maybe we can have separate apps to do that - one for the average user and another for advanced.

As for the DNS approach, yea, it's probably too advanced right now. I think we should get a simple image/gif display up first.

Let me know your thoughts or if I misunderstood what you were trying to propose.

[audreylim]

By the way you can join our Discord server if you'd like to discuss with other developers: https://discord.gg/hpxFXchr. But let's keep the discussion on the image loading cycle app here.

[johnhtodd]

So it appears that the first thing to do would be to write a small program (python? C? something else?) that converts .jpg images into the bitmap format that you describe, correct? In that way, the JSON "configuration file" that I describe could point to URLs that are converted bitmaps instead of JPG/PNG/GIF files. Because I envision this applet being used as part of a larger suite of graphics-generating tools (it's difficult to assume what people will use for the original graphics) then having a tool that converts "normal" images to Kublet-digestible images seems like a first good step, so that a "minimum viable product" model could be done with getting an image from a remote server, via HTTP, and displayed on the kublet. Is this a reasonable first step?

[johnhtodd]

Example image. Of course, I can change this graph immensely by just changing parameters in Grafana (remove labels, change colors, add text, add overlay bar graphs, etc. etc.) without coding anything. I just update the graph, and then call a URL that generates a static version of the graph. This could be done by the Kublet directly if it had some way to parse PNG images, or (better) I would have a tiny simple script that pulled all the interesting graphs every few minutes, put them into a directory, and then ran a PNG-to-Kublet graphic format converter on the files to turn them into easily-digested image formats for the Kublet to download.