Ability to specify a data-url response (or just a JPG) instead of HTML wrapping a data-url

[jywarren]

We should make a parameter called response= which can be set to response=html or response=dataurl maybe?

[jywarren]

https://github.com/publiclab/image-sequencer-app/blob/dca8158e18a5d83476cb779da5a8d2dbe418cd2b/src/api/v2/index.js#L71-L79

[jywarren]

Actually we should just redirect to a JSON status file that's actually stored in Google Cloud, which we are periodically updating as the export progresses.

[jywarren]

See how here, we create a JSON file with the status, and repeatedly save it to Google Cloud storage as the process continues. And we report the location of that file back to the user immediately, so they can fetch it as often as they like without running any process on the exporter itself:

https://github.com/publiclab/mapknitter-exporter-sinatra/blob/main/app.rb#L119-L187

[tech4GT]

Okay! This makes sense! So we just make and store a json file in the google cloud and update it periodically! Hmm, I wonder if this file could be stored locally? Since this file can be deleted after the export completes I'm not entirely sure if keeping it in the cloud would be a good idea, don't you think? cc @jywarren

[jywarren]

We actually want a permanent record of the status -- that's how the Ruby exporter works. Thank you!!!

On Sat, Dec 21, 2019 at 1:23 AM Varun Gupta notifications@github.com wrote:

Okay! This makes sense! So we just make and store a json file in the google cloud and update it periodically! Hmm, I wonder if this file could be stored locally? Since this file can be deleted after the export completes I'm not entirely sure if keeping it in the cloud would be a good idea, don't you think?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer-app/issues/36?email_source=notifications&email_token=AAAF6J6U65XMCJYFU2XPPA3QZWY6PA5CNFSM4JXCVBGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHOWFAQ#issuecomment-568156802, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAF6J37ZKAJOGZNIWHPQP3QZWY6PANCNFSM4JXCVBGA .

[tech4GT]

We actually want a permanent record of the status -- that's how the Ruby exporter works. Thank you!!! … On Sat, Dec 21, 2019 at 1:23 AM Varun Gupta @.***> wrote: Okay! This makes sense! So we just make and store a json file in the google cloud and update it periodically! Hmm, I wonder if this file could be stored locally? Since this file can be deleted after the export completes I'm not entirely sure if keeping it in the cloud would be a good idea, don't you think? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36?email_source=notifications&email_token=AAAF6J6U65XMCJYFU2XPPA3QZWY6PA5CNFSM4JXCVBGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHOWFAQ#issuecomment-568156802>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAF6J37ZKAJOGZNIWHPQP3QZWY6PANCNFSM4JXCVBGA .

Oh! Okay! I'll get started on this then! 😃

[jywarren]

Noting also possibility of uploading the JPG to Google Cloud Storage as well; i think it would involve changing these lines:

https://github.com/publiclab/image-sequencer-app/blob/744b521f08342569cea1baeec8670c3b778a3745/src/api/v2/util/process-multisequencer.js#L66-L79

I guess it would be, instead, something like (ref):

       var data = out.replace(/^data:image\/\w+;base64,/, ""); // strip off the data: url prefix to get just the base64-encoded bytes
       var buf = new Buffer(data, 'base64');
       var filename = `export${pid}.jpg`
       var imgPath = path.join(__dirname, `../../../../temp/${filename}`);
       fs.writeFileSync(imgPath, buf);
       mapknitterBucket.upload(imgPath, {
            gzip: true
        }).then(() => {
            mapknitterBucket
                .file(filename)
                .makePublic();
            fs.unlinkSync(path.join(__dirname, imgPath));
            mapknitterBucket.file(filename)
                .getMetadata()
                .then((data) => changeStatus("uploaded", data[0].mediaLink));

[jywarren]

OK i made a PR here: #36

But we will also need to change the status.json output file to now point at the correct JPG file.

[jywarren]

Ah no, that will happen here: https://github.com/publiclab/image-sequencer-app/blob/a3060fb3348130192d1fb3a1efe333ebde03034f/src/api/v2/util/process-multisequencer.js#L80

So this might just work...?