search

asticode / go-astilectron

Build cross platform GUI apps with GO and HTML/JS/CSS (powered by Electron)
MIT License
4.9k stars 344 forks source link

Implement ses.flushStorageData as writing to localStorage doesn't always persist #229

Closed sereeena closed 4 years ago

sereeena commented 4 years ago

I find that when writing to localStorage, data will not persist if the go-astilectron app closes or crashes within about 10 seconds, before the localStorage has had a chance to be written - easily reproducible by loading a page with a script which writes to localstorage then closing the window after 2 seconds. With devtools open you can see the item is written to localstorage, but when the app is restarted it isn't there.

I know this is due to electron localstorage not being flushed to disk, and there are numerous issues about it (like https://github.com/electron/electron/issues/1731) however even when we explicitly call app.Quit() on the astilectron app, which calls app.quit() for electron, the problem still occurs if we quit 2 seconds after writing to localstorage.

I would really like writing to localStorage to be reliable - 10 secs is a bit too long to have to wait to make sure it has been properly written.

Is it possible for astilectron.Session to implement flushStorageData so we can explicitly make electron localStorage be written? I'm not sure how to go about this.

asticode commented 4 years ago

This is a good idea. However I won't have time to work on it but I'm welcoming PRs.

If you feel like contributing, check out the ClearCache feature, since it will be very similar:

sereeena commented 4 years ago

I was attempting to give this a go, but having trouble provisioning my astilectron bundle - I cloned and modified it, then calling the bundler with -a to pass my modified astilectron fails:

astilectron-bundler -a ../../vendor/astilectron 2020/04/17 02:21:52 Bundling for environment linux/amd64 2020/04/17 02:21:52 Binding data 2020/04/17 02:21:52 Removing /tmp/astibundler/bind 2020/04/17 02:21:52 Creating /tmp/astibundler/bind 2020/04/17 02:21:52 Creating /tmp/astibundler/bind/vendor_astilectron_bundler 2020/04/17 02:21:52 Creating /tmp/astibundler/cache 2020/04/17 02:21:52 Zipping /home/ubuntu/Projects/go/src/mini/vendor/astilectron into /tmp/astibundler/cache/astilectron-0.34.0.zip 2020/04/17 02:21:52 bundling failed: bundling for environment linux/amd64 failed: binding data failed: provisioning the vendor failed: provisioning astilectron vendor failed: zipping /home/ubuntu/Projects/go/src/mini/vendor/astilectron into /tmp/astibundler/cache/astilectron-0.34.0.zip failed: astikit: walking failed: lstat /home/ubuntu/Projects/go/src/mini/vendor/astilectron/astilectron-0.34.0: no such file or directory

When I look at the zipped file in /tmp/astibundler/cache it is empty only 22 bytes, can you tell me what is wrong with my setup? Thanks

asticode commented 4 years ago

@sereeena cheers for giving this a go!

1) This is a bug that was happening in an older version of the bundler. And your version of the bundler is trying to work with 0.34.0 version of astilectron whereas now it's at 0.36.0. Therefore I would suggest that you use the latest version for both go-astilectron and go-astilectron-bundler (and don't forget to run go install github.com/asticode/go-astilectron-bundler/astilectron-bundler once done!).

2) I took a look at your commit and line 159 you shouldn't add the .webContents.session part. Indeed, elements[json.targetID] actually points to a session already.

sereeena commented 4 years ago

Do you mean elements[sessionID] points to a session already?

Anyway, I got the bundler working with my go-astilectron branch but I can't get it to provision my astilectron code properly to check my changes.

If in my bundler.json I have "version_astilectron": "0.36.0", it zips it correctly in the cache folder as astilectron-0.36.0.zip, and puts it in the bind/vendor_astilectron_bundler folder as astilectron.zip. Then when the application is run, astilectron-0.36.0.zip is extracted in vendor folder, but it is the original astilectron code, not my code from those working directory folders. I tried to set versionastilectron to empty string but that causes my code to be zipped to cache as "0.37.0" and bundles the original astilectron-0.36.0 code. Where should I go from here? Also deleted and rebuilt bind_amd64.go files to be safe and that didn't help.

But I am not sure if it makes sense to implement FlushStorage on the go-astilectron Session after all, because if you are writing to localStorage on the js side, you are going to want to expose a way to get the browserwindow or webcontents to call flushstorage yourself, rather than sending a message to the go backend to send an event to astilectron to call flushstorage? Can you advise how to I would get one of these objects from js code?

asticode commented 4 years ago

elements[sessionID] does point to a session already.

Can you share your bundle command as well as logs so that I can take a look at what's happening ? If you open your app once, and open it a second time in the same folder, it will detect that vendor files have already been provisioned and it won't try to do anything vendor-wise. That may be why you don't see your custom code. What you would need to do in that situation is cleaning up the directory you're opening the app in.

I think implementing FlushStorage in go-astilectron's Session makes sense but I do agree, if your goal is to call it in JS, all you should have to do is get the session with var session = require('electron').remote.session.

sereeena commented 4 years ago

There is no vendor folder extracted until I run the application, so it is not already provisioned. It is then that the original source is created next to my executable. I am using a working_directory_path "tmp" which is relative to my current directory, so during bundling my astilectron code is zipped into the tmp/cache folder, and I don't know why it isn't using it.

Note I am running on linux, here is the output of my bundle command, do you need anything else?

~/Projects/go/src/github.com/sereeena/go-astilectron-bundler/astilectron-bundler/astilectron-bundler -o ../../artifacts -a ../../vendor/astilectron 2020/04/20 07:07:00 Bundling for environment linux/amd64 2020/04/20 07:07:00 Binding data 2020/04/20 07:07:00 Removing /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind 2020/04/20 07:07:00 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind 2020/04/20 07:07:00 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler 2020/04/20 07:07:00 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache 2020/04/20 07:07:00 Zipping /home/ubuntu/Projects/go/src/mini/vendor/astilectron into /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip 2020/04/20 07:07:00 /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip already exists, skipping download of https://github.com/asticode/astilectron/archive/v0.36.0.zip 2020/04/20 07:07:00 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler/astilectron.zip 2020/04/20 07:07:00 /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/electron-linux-amd64-4.0.1.zip already exists, skipping download of https://github.com/electron/electron/releases/download/v4.0.1/electron-v4.0.1-linux-x64.zip 2020/04/20 07:07:00 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/electron-linux-amd64-4.0.1.zip to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler/electron.zip 2020/04/20 07:07:00 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/resources 2020/04/20 07:07:00 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/resources to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/resources 2020/04/20 07:07:00 Generating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/bind_linux_amd64.go 2020/04/20 07:07:03 Removing /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64 2020/04/20 07:07:04 Creating /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64 2020/04/20 07:07:04 Building for os linux and arch amd64 astilectron: 0.36.0 electron: 4.0.1 2020/04/20 07:07:04 Executing go build -ldflags -X "main.AppName=oncotrack-client" -X "main.BuiltAt=2020-04-20 07:07:04.156201571 +0000 UTC m=+3.372167548" -X "main.VersionAstilectron=0.36.0" -X "main.VersionElectron=4.0.1" -o /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/binary . 2020/04/20 07:07:17 Moving /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/binary to /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/oncotrack-client 2020/04/20 07:07:17 Bundling for environment windows/amd64 2020/04/20 07:07:17 Binding data 2020/04/20 07:07:17 Removing /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind 2020/04/20 07:07:17 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind 2020/04/20 07:07:17 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler 2020/04/20 07:07:17 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache 2020/04/20 07:07:17 Zipping /home/ubuntu/Projects/go/src/mini/vendor/astilectron into /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip 2020/04/20 07:07:17 /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip already exists, skipping download of https://github.com/asticode/astilectron/archive/v0.36.0.zip 2020/04/20 07:07:17 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler/astilectron.zip 2020/04/20 07:07:17 /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/electron-windows-amd64-4.0.1.zip already exists, skipping download of https://github.com/electron/electron/releases/download/v4.0.1/electron-v4.0.1-win32-x64.zip 2020/04/20 07:07:17 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/electron-windows-amd64-4.0.1.zip to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler/electron.zip 2020/04/20 07:07:17 Creating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/resources 2020/04/20 07:07:17 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/resources to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/resources 2020/04/20 07:07:17 Generating /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/bind_windows_amd64.go 2020/04/20 07:07:20 Running rsrc for icon /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/resources/icon.ico into /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/windows.syso 2020/04/20 07:07:20 Removing /home/ubuntu/Projects/go/src/mini/artifacts/windows-amd64 2020/04/20 07:07:20 Creating /home/ubuntu/Projects/go/src/mini/artifacts/windows-amd64 2020/04/20 07:07:20 Building for os windows and arch amd64 astilectron: 0.36.0 electron: 4.0.1 2020/04/20 07:07:20 Executing go build -ldflags -H "windowsgui" -X "main.AppName=oncotrack-client" -X "main.BuiltAt=2020-04-20 07:07:20.081694847 +0000 UTC m=+19.297660642" -X "main.VersionAstilectron=0.36.0" -X "main.VersionElectron=4.0.1" -o /home/ubuntu/Projects/go/src/mini/artifacts/windows-amd64/binary . 2020/04/20 07:07:26 Moving /home/ubuntu/Projects/go/src/mini/artifacts/windows-amd64/binary to /home/ubuntu/Projects/go/src/mini/artifacts/windows-amd64/oncotrack-client.exe

asticode commented 4 years ago

Can you share logs when you open the app?

sereeena commented 4 years ago

I'm sorry I don't have anything logged. I have just tried to set up astilog but after getting it I can't compile it, I get this error

build gitlab.com/nonacus/timeline-desktop/timeline-desktop/cmd/oncotrack-client: cannot load github.com/asticode/go-astilog: module github.com/asticode/go-astilog@latest found (v2.0.1+incompatible), but does not contain package github.com/asticode/go-astilog

Are these the sort of logs you are looking for?

sereeena commented 4 years ago

OK, I looked at the demo to see how logging is implemented in the bootstrapper, it's very useful! Here is the interesting part - shows it downloads astilectron instead of using my custom one.. (I just run my binary with -d or with no flags)

By the way, thank you var session = require('electron').remote.session works well

2020/04/21 01:23:44 astikit: starting worker... 2020/04/21 01:23:44 Starting... 2020/04/21 01:23:44 Provisioning... 2020/04/21 01:23:44 Provisioning Astilectron... 2020/04/21 01:23:44 Removing directory /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron 2020/04/21 01:23:44 Downloading https://github.com/asticode/astilectron/archive/v0.36.0.zip into /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron-v0.36.0.zip 2020/04/21 01:23:44 astikit: sending GET request to https://github.com/asticode/astilectron/archive/v0.36.0.zip (1/1) 2020/04/21 01:23:45 Creating directory /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron 2020/04/21 01:23:45 Unzipping /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron-v0.36.0.zip/astilectron-0.36.0 into /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron 2020/04/21 01:23:45 Provisioning Electron... 2020/04/21 01:23:45 Removing directory /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64 2020/04/21 01:23:45 Downloading https://github.com/electron/electron/releases/download/v4.0.1/electron-v4.0.1-linux-x64.zip into /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64-v4.0.1.zip 2020/04/21 01:23:45 astikit: sending GET request to https://github.com/electron/electron/releases/download/v4.0.1/electron-v4.0.1-linux-x64.zip (1/1) 2020/04/21 01:24:36 Creating directory /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64 2020/04/21 01:24:36 Unzipping /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64-v4.0.1.zip into /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64 2020/04/21 01:24:38 System doesn't require finshing provisioning electron, moving on... 2020/04/21 01:24:38 Listening... 2020/04/21 01:24:38 Executing... 2020/04/21 01:24:38 Starting cmd /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/electron-linux-amd64/electron /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/vendor/astilectron/main.js 127.0.0.1:38813 true

asticode commented 4 years ago

Those logs show what happens when you don't use the bundler to bundle your app, therefore you must be running another binary than the one you've actually bundle. Could you clean up what is in /home/ubuntu/Projects/go/src/mini/artifacts, bundle again and run /home/ubuntu/Projects/go/src/mini/artifacts/linux-amd64/oncotrack-client?

sereeena commented 4 years ago

I most definitely am running the binary which I have bundled and get the same logs as above. Are you sure it does not have something to do with the fact that tmp/cache/astilectron-0.36.0.zip is copied to tmp/bind/vendor_astilectron_bundler/astilectron.zip because my custom code doesn't have a version? Perhaps because it can't find astilectron-0.36.0.zip in the vendor_astilectron_bundler directory it then downloads it from github? Is it supposed to be copying it as astilectron.zip?

2020/04/21 09:32:08 Zipping /home/ubuntu/Projects/go/src/mini/vendor/astilectron into /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip 2020/04/21 09:32:08 /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip already exists, skipping download of https://github.com/asticode/astilectron/archive/v0.36.0.zip 2020/04/21 09:32:08 Copying /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/cache/astilectron-0.36.0.zip to /home/ubuntu/Projects/go/src/mini/cmd/oncotrack-client/tmp/bind/vendor_astilectron_bundler/astilectron.zip

sereeena commented 4 years ago

I found why my provisioning wasn't working, the following was missing from my main.go, (so it wasn't using the bundled code). I think I had initially followed the code in your medium article, which only added the Asset/AssetDir lines towards the end of the article, so I'd missed them out.

It all seems to work and I've created the pull requests for you to review


bootstrap.Run(bootstrap.Options{
        Asset:    Asset,
        AssetDir: AssetDir,```
asticode commented 4 years ago

Sounds good, thanks for the PRs !