noi-techpark / it.bz.opendatahub.epaper.webapp

0 stars 5 forks source link

As a meeting manager I want to upload a custom image to be shown on a display. #38

Closed sseppi closed 2 years ago

sseppi commented 2 years ago

We decided to use a S3 bucket to upload images.

As an example you can see this project which uploads and downloads objects from s3 --> https://github.com/noi-techpark/davinci-innovation-scoreboard-api

We need 4 variables to configure the write access to the bucket:

S3_REGION = "eu-west-1"
S3_BUCKET_NAME = "it.bz.opendatahub.epaper.images-test"
S3_ACCESS_KEY = "..."
S3_SECRET_KEY = "..."

Update: I am not yet sure if we want to use the API to download images or a direct public link to the s3 bucket?

Idea 1: The images should be publicly available to be downloaded easily from each display without authentication. If in the future we want to be more secure, we can then limit the access with access keys, but for now this is not necessary.

Idea 2: The download is from a GET method of the API itself, the file mime type is application/octet-stream to be generic or image/bmp or even image/x-windows-bmp

I tend to Idea 2 at the moment... please comment below

UPDATE 2022-01-12 We implement Idea2!

Piiit commented 2 years ago

@AlePanTS @SaimonasFOS @sseppi @giovannigiannotta See main comment for further details. What do you think about the 2 ideas, which one should we take?

SaimonasFOS commented 2 years ago

@Piiit For me the Idea 2 seems to be better, I will have to test it with our Spring API app. Right now it seems that it shouldn't be hard to implement.

Piiit commented 2 years ago

@SaimonasFOS Perfect, so lets implement Idea2... The link above already has an Java implementation of that approach. How do I send you the credentials? Do you have a PGP public key or do you prefer another method?

SaimonasFOS commented 2 years ago

@Piiit Sorry but I thought that with the Idea 2 we don't use the bucket at all? As I imagined it, pictures would be saved on the eInk database and returned to displays when requested via one of the formats that you described. Maybe I misunderstood the idea?

Piiit commented 2 years ago

@SaimonasFOS Sorry for the confusion, but storing big binary files inside the database does not work very well in terms of costs and maintenance. Please just store the checksum and link to the binary, and then we download and upload files to a s3 bucket. Storing images on a local disk is also not working well, because our docker hosts are volatile and should not store any data directly.

Piiit commented 2 years ago

As an intermediate solution, but only temporary for the first MVP, we could nevertheless store images just on the local machine and then implement the s3 connection in MVP2... if you feel that time is running out

SaimonasFOS commented 2 years ago

@Piiit Okay, now as I understand with the Idea 2 the API server acts as a proxy so the displays don't have to connect to the bucket directly. I think it's a good approach. I can try to implement the bucket but I'm not sure about the authentication method. I don't have a PGP public key, what else could we use?

Piiit commented 2 years ago

I wrote you a mail

AlePanTS commented 2 years ago

@Piiit also for me the second solution seems to be better (I don't like security issues to be fixed later on... too risky!) as far as I understand, the response of the GET call will contain the URL of the image to download, in addition to other information like checksum, etc..

SaimonasFOS commented 2 years ago

@AlePanTS @Piiit Peter's idea was that the API returns not an URL but the picture itself.

I think this could be the flow between display and the backend: The display periodically makes a POST request to URL like https://api_link/display/status and in the body you would specify the current status of the display (battery level, error message, etc.) then the server would return the checksum of the image that needs to be on the display. Then you can compare the checksum that you received with the checksum of an image that the display is showing at the moment. If they are the same, you don't call anything. If they are different, you call a GET request like https://api_link/display/getPicture/{displayId} which returns an actual image. You could test that right now by calling GET https://epaper.opendatahub.testingmachine.eu/img/logo.74bb0640.png . This example request would return you just an image with image/png format. Of course, on the actual system pictures will be in BMP format, not PNG, but the idea is the same.

AlePanTS commented 2 years ago

Thanks @SaimonasFOS, I agree with @Piiit on work flow about checking if a new image is available, but the GET request having https://epaper.opendatahub.testingmachine.eu/img/logo.74bb0640.png as URL is not the same as the GET request with https://api_link/display/getPicture/{displayId} as URL. They are two different requests with different outcomes. For me both are fine, just let me know how the image is sent in the response of the second request (if it's not an URL to use to download image). In alternative, we could decide that the URL of the image will be always the same for each display (let's say https://epaper.opendatahub.testingmachine.eu/img/imageEPD01.bmp), and the display sends the GET request to the URL each time the checksum is not the same as the one of last image shown. With this approach, the image on the server will be always replaced with the new one, for the sake of storage (as a result, there will be only one image per display). I guess storage limitation will not be a problem at this stage, just a good habit ;) This could be an option, but I am open to any solutions.