binary assets missing/corrupted

ray007 commented 4 years ago

Environment Provide version numbers for the following components (information can be retrieved by running tns info in your project folder or by inspecting the package.json of the project):

CLI: 6.7.4
Cross-platform modules: 6.5.5
Android Runtime: 6.5.1
iOS Runtime: -

Plugin(s):

"dependencies": {                            
"@nativescript/theme": "~2.3.3",           
"@nota/nativescript-webview-ext": "^6.5.3",
"nativescript-bluetooth": "^3.0.0-beta.10",
"nativescript-inappbrowser": "^2.3.0",     
"nativescript-vue": "~2.6.4",              
"node-forge": "^0.9.1",                    
"tns-core-modules": "^6.5.5",              
"url-parse": "^1.4.7",                     
"url-search-params-polyfill": "^8.1.0"     
},

Describe the bug I'm showing a webapp in a webview (with @nota/nativescript-webview-ext) from a subfolder of my assets folder. This works fine for most things, but the binary assets seem to be missing/corrupted:

a splash gif
an ico file (not really used)
material ui icon font (ttf)

Looking at the debug-output in VSCode, it seems that webpack tries to bundle the embedded webapp instead of just copying it, and I'm not sure why or how to prevent that from happening.

I could ignore the missing splash.gif for now, but the icons are used in my parts of the webui, and missing those is bad.

NathanaelA commented 4 years ago

You need to add to your webpack config in the copy phase the directory you want copied;

In this location you can add your additional directory that you want copied (as-is) just like the fonts/** is copied over...

ray007 commented 4 years ago

The files do get taken, otherwise the js-app wouldn't load at all. The section in my webpack.config.js looks like this:

            new CopyWebpackPlugin([
                { from: { glob: "fonts/**" } },
                { from: { glob: "**/*.+(jpg|png)" } },
                { from: { glob: "assets/**/*" } },
            ], { ignore: [`${relative(appPath, appResourcesFullPath)}/**`] }),

and my webview src folder is a subfolder of assets.

And as I wrote, everything works to start, but loading of the binary assets fails. Displaying an svg file which is part of the resources works fine.

So the question is: how do I make webpack not destroy my js-app?

NathanaelA commented 4 years ago

When I last did it (about two weeks ago); my rules was:

  new CopyWebpackPlugin([
                { from: { glob: "fonts/**" } },
                { from: { glob: "**/*.jpg" } },
                { from: { glob: "**/*.png" } },
                { from: { glob: "assets/**"}},

And this worked fine. You might try removing the "/*" from your copy rule, it might be causing issues...

Then if you check your actually created .apk or .ipk you can see if everything was copied 100% correctly; but in my case this worked fine...

ray007 commented 4 years ago

The copying is not the issue, the files are there. The problem is the corruption of the binary files.

To test I moved the webapp dir to besides the fonts and assets direcotories (directly in app), copied the font to the fonts-directory and edited the css for the material icons to point there (url(../fonts/MaterialIcons-Regular.ttf)). And now the icons show up as they're supposed to.

First tests of the app look good, but I also don't know if webpack might have ruined something else, since the app was compiled with google closure compiler in advanced mode.

So the issue is, that I want the directory copied, but for webpack to not use any of its magic on it. Is there any way to do that?

ray007 commented 4 years ago

If I should ask the webpack guys I can do that, but this issue is not resolved and should not be closed.

NathanaelA commented 4 years ago

@ray007 - If you have a copy rule set to "assets/" and your assets folder is in the same directory as your application like so:

The the webpack copy rule will copy it as-is, no corruption. HOWEVER, if you have any js, xml, css, html in that folder it will ALSO be added to the webpack files; but the copied version will still exist perfectly fine.

If you are using a webview to use the files in the assets folder; you should be fine. If you are attempting to use NativeScript's require to load those files; that won't work as webpack hijacks require and will always use its bundled version of the resource; rather than the physical copy.

The above image actually has both a assets and a webassets I have rules for both folders; and the NS app will load the proper directory depending on which button I click in the NS side. One of those folders is a complete web app made with totally different tooling that I dropped in and run from inside NativeScript. Please note; that you DO have to have certain features enabled in the webview and certain application properties set to make the webview work properly as security will block things (like File:// urls) by default on both Android and iOS depending on which things you are trying to do.

ray007 commented 4 years ago

I already did move my webassets folder to beside assets like in your image and added the copy-rule for it, but that didn't help. The only (very indirect) reference to my webapp is the src attribute of my webview (shown with nativescript-webview-ext). And I'm using nativescript with the vue-js wrapper if that makes any difference.

I can see on the build-console that my webassets get bundled:

Bundling application for entryPath .\app...
[BABEL] Note: The code generator has deoptimised the styling of C:\rtc-sandbox\tnsApp\app\webassets\main.js as it exceeds the max of 500KB.

And a little further down:

                                    Asset       Size  Chunks             Chunk Names
                        assets/splash.gif   17.9 KiB          [emitted]
                                bundle.js   1.46 MiB       0  [emitted]  bundle
          fonts/MaterialIcons-Regular.ttf    125 KiB          [emitted]
                  fonts/fa-brands-400.ttf    131 KiB          [emitted]
                 fonts/fa-regular-400.ttf   39.1 KiB          [emitted]
                   fonts/fa-solid-900.ttf    204 KiB          [emitted]
                    webassets/css/app.ico   2.43 KiB          [emitted]
                    webassets/css/app.svg   6.09 KiB          [emitted]
                webassets/css/app_114.png   12.5 KiB          [emitted]
              webassets/img/app_views.svg     80 KiB          [emitted]
                 webassets/img/splash.gif   31.7 KiB          [emitted]
                     webassets/index.html   2.26 KiB          [emitted]
                  webassets/lib/basket.js    9.9 KiB          [emitted]
                  webassets/lib/loader.js   1.94 KiB          [emitted]
webassets/lib/m/MaterialIcons-Regular.ttf    147 KiB          [emitted]
       webassets/lib/m/material-icons.css  987 bytes          [emitted]
          webassets/lib/m/materialize.css    139 KiB          [emitted]
                        webassets/main.js    636 KiB          [emitted]
                     webassets/applibs.js    393 KiB          [emitted]
                  webassets/appstyles.css    161 KiB          [emitted]
                      webassets/msgs.json   47.8 KiB          [emitted]
                     webassets/sw-load.js   10.4 KiB          [emitted]
                             package.json  112 bytes          [emitted]
                               runtime.js   1.86 KiB       1  [emitted]  runtime
                      tns-java-classes.js    0 bytes          [emitted]
                                vendor.js   1.97 MiB       2  [emitted]  vendor
Entrypoint bundle = runtime.js vendor.js bundle.js
 [24] ./lib/appCfg.js 3.72 KiB {0} [built]
 [34] ./lib/appBT.js 4.82 KiB {0} [built]
 [50] ./lib/appUrl.js 2.05 KiB {0} [built]
 [89] ./lib/tnsSrvConn.js 9.15 KiB {0} [built]
[129] ./app.js 1.84 KiB {0} [built]
[169] ./app.scss 120 KiB {0} [optional] [built]
[170] ./lib/certs.js 2.8 KiB {0} [built]
[268] ./components/Home.vue + 3 modules 7.1 KiB {0} [built]
      |    4 modules
[426] . sync nonrecursive ^\.\/app\.(css|scss|less|sass)$ 175 bytes {0} [built]
[427] . sync (?<!\bApp_Resources\b.*)(?<!\.\/\btests\b\/.*?)\.(xml|css|js|kt|(?<!\.d\.)ts|(?<!\b_[\w-]*\.)scss)$ 534 bytes {0} [built]
[514] ./webassets/lib/basket.js 13.9 KiB {0} [optional] [built]
[515] ./webassets/lib/loader.js 2.08 KiB {0} [optional] [built]
[516] ./webassets/lib/m/material-icons.css 1.77 KiB {0} [optional] [built]
[517] ./webassets/lib/m/materialize.css 140 KiB {0} [optional] [built]
[518] ./webassets/main.js 638 KiB {0} [optional] [built]
    + 523 hidden modules
Webpack compilation complete.

Btw: which application properties are you talking about? Other than the binary asset problem I didn't notice any troubles troubles with local file loading. Ignoring a few IndexedDB issues for now...

NathanaelA commented 4 years ago

The Nota/webview-ext actually takes care of most of those issues I was referencing... However, I still had several issues myself using it and had to write custom code to make it work on iOS and Android. One issue is that iOS was not able to load the new ESM JS when it also loaded normal JS in the same html page... So if you are building your html from another tool; you might have to manually tweak the index file...

In addition, I had to create another server plugin to actually server the html and files like a web server I think to make it work on Android (I think it was android) properly. Been a couple months, but their are some weird issues on both platforms and file:// based urls.

This is an area I do plan on trying to make a blog post on and some plugins to simplify; but I never had any issues with Webpack other than it wanting to include everything in the assets folder in the bundle/vendor.js in addition to copying it (making the app a bit bigger).

ray007 commented 4 years ago

A helpful comment at https://github.com/webpack/webpack/issues/11104#issuecomment-651801989 didn't tell me why the binary issues got corrupted, but helped me find a way to avoid the bundling.

The bundling command seems to come from ...\tnsApp\platforms\android\app\src\main\assets\app\bundle.js (and I suspect something similar for iOS). And it seems to bundle everything in the app directory not below App_Resources or test.

So I was able to avoid the problem by moving my webapp into the App_Resources folder. I then stripped the App_Resources prefix by adding a transformPath parameter to the CopyWebpackPlugin, but that would not be needed if I changed the load-path for the webapp to include App_Resources.

@NathanaelA thanks for the help, and where will I find the blog post you're talking about?

NathanaelA commented 4 years ago

I haven't written the blog post yet; but all my blog posts typically end up on my blog at https://fluentreports.com/blog

ray007 commented 4 years ago

Is putting stuff to not be bundled into App_Resources the way this is supposed to work, or did I miss something?

NativeScript / nativescript-cli

binary assets missing/corrupted #5328