Trying to implement LibLouis w/ React, Storybook & WebPack, console says tables could not be compiled.

[faddah]

Hello,

We are trying to implement your LibLouis build & JavaScript NPM Package wrapper for it into our project which is a React Native app for iOS & Android to teach families of the vision impaired Braille.

We are having problems implementing it in our front-end process. Allow me to explain —

For the front end, our process is to create components in an implementation of Storybook with React Native Web, which, as we add & finish components, becomes an updated NPM package which we import into our React Native app for use.

We are having problems implementing your liblouis/js-build & liblouis/liblouis-js with the Storybook/React Native Web part. We think it's some sort of conflict or configuration problem with the liblouis/js-build tables loading with either WebPack or Storybook or a combination thereof.

Here's our WebPack configuration, set-up to include your liblouis/js-build files in loading, following your instructions —

const path = require('path')
const webpack = require('webpack')

module.exports = (storybookBaseConfig, configType) => {
  const DEV = configType === 'DEVELOPMENT'

  storybookBaseConfig.externals = {
    'liblouis-build': 'commonjs liblouis-build',
  }

  storybookBaseConfig.node = {
    fs: 'empty',
    module: 'empty',
  }

  storybookBaseConfig.module.rules.push({
    test: /(\.cti|\.ctb|\.utb|\.dis|\.uti|\.tbl|\.dic)$/,
    loader: 'file-loader?name=tables/[name].[ext]',
  })

  storybookBaseConfig.module.rules.push({
    test: /\.css$/,
    use: ['style-loader', 'css-loader'],
  })

  storybookBaseConfig.module.rules.push({
    test: /\.(png|woff|woff2|eot|ttf|svg)$/,
    loader: 'url-loader?limit=100000',
  })

  storybookBaseConfig.module.rules.push({
    test: /\.(gif|jpe?g|png|svg)$/,
    use: {
      loader: 'url-loader',
      options: { name: '[name].[ext]' },
    },
  })

  storybookBaseConfig.plugins.push(
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(
        process.env.NODE_ENV || 'development'
      ),
    })
  )

  storybookBaseConfig.resolve = {
    modules: ['node_modules'],
    extensions: ['.web.js', '.js', '.json', '.web.jsx', '.jsx'],
    alias: {
      'react-native': 'react-native-web',
      'react-native-svg': 'react-native-svg-web',
    },
  }

  return storybookBaseConfig
}

We loaded your libs into our package.json and node_modules for the Storybook/React Native Web project the way you instructed us to in issues we communicated with you here and here, like so —

$ npm install liblouis/js-build --save
$ npm install liblouis/liblouis-js --save

And here's that section of our package.json with those dependenices now working —

...
  "dependencies": {
    ...,
    "liblouis": "github:liblouis/liblouis-js",
    "liblouis-build": "github:liblouis/js-build",
    ...
  }
...

And here is the relevant code in the React Native Web component we are trying to build there, this is a test case just to see if we can at least to get a translation to return in the JavaScript console, the es-lint comments are to keep es-lint from preventing us using the file-loader! expression you say to use —

import React from 'react'

...
const liblouis = require('liblouis/easy-api')
// eslint-disable-next-line import/no-webpack-loader-syntax
const capi_url = require('file-loader!liblouis-build')
// eslint-disable-next-line import/no-webpack-loader-syntax
const easyapi_url = require('file-loader!liblouis/easy-api')

console.log({ liblouis })
console.log({ capi_url })

require.context('liblouis-build/tables', false)
const table_url = 'tables/'

const asyncLiblouis = new liblouis.EasyApiAsync({
  capi: capi_url,
  easyapi: easyapi_url,
})

asyncLiblouis.enableOnDemandTableLoading(table_url)
asyncLiblouis.translateString(
  'tables/unicode.dis,tables/en-ueb-g2.ctb',
  'Hi, Mom! You owe me: $3.50.',
  e => {
    console.log(e)
  }
)

...
const ComponentName = () =>

  return (
      <View style={styles.container} ref={onMount}>
        {asyncLiblouis.version(version => {
          console.log(`Liblouis Version using Easy API:  ${version}`)
        })}
      ...
      </View>
  ...
  )
}

To let you know, we have tried this with the location of the tables string being either 'unicode.dis,en-ueb-g2.ctb' or 'tables/unicode.dis,tables/en-ueb-g2.ctb'; in the case of the former, it gives a 404 not found" message in the JavaScript console.

And in the case of the latter, when we try and run this code in Storybook in our browser to look at the component and the JavaScript console to see the result, we get the following in the JavaScript console and the translation never happens —

{liblouis: {…}}
{capi_url: "/296124e2f136a52937d2e3458d7fe4d5.js"}
client.js:87 [HMR] connected
[40000] Cannot resolve table 'tables/unicode.dis'
[40000] 1 errors found.
[40000] tables/unicode.dis,tables/en-ueb-g2.ctb could not be compiled
null
Liblouis Version using Easy API:  3.5.0

So the Liblouis Version console log part that is in the return(...) part of React.JS works fine, but the other test code above it to the JavaScript console seems to see the tables, but cannot compile them for use in LibLouis and do the translation. I know it can see the files, because if I direct my browser URL to http://localhost:9001/tables/en-ueb-g2.ctb, where Storybook is running locally in my browser, it downloads the en-ueb-g2.ctb just fine and I can open it in a code editor like Atom or Visual Studio Code, and it is, indeed, that particular translation table from your liblouis/js-build.

So is there any way you can help us here in implementing your LibLouis to work here with Storybook, React Native Web & WebPack in our project correctly so it does the translations? Please let us know. Thank you in advance for a prompt review & response to this.

best,

— faddah portland, oregon, u.s.a.

[reiner-dolp]

I have never used Storybook or React Native Web, so I cannot comment on your setup.

This is a shot in the dark, but could you please confirm that your server is correctly answering to HTTP HEAD requests?

Instead of testing the availability of a resource by typing it into your browser bar, please take a look at the network tab in dev tools. What does it say?

[faddah]

Hello @reiner-dolp,

Thank you for responding.

Ok, so here is a screen shot of the Chrome Dev Tools JavaScript console, similar to what I had printed above —

And here's a screen shot of the Network tab after that same page has loaded —

And I did a screen recording in the Chrome Dev Tools Network Tab of the page reloading, here are the frames from that. Here's at 90 ms —

Here's at 487 ms —

Here's at 1.23 secs. —

Here's at 1.99 secs. —

And finally, here is at 2.24 secs. —

The one that seems to take the longest to download from the web is the __webpack_hmr, but that should not be a surprise. Any ideas on your end, please?

Thank you again for your prompt response.

best,

— faddah

[reiner-dolp]

Given the setup code you use, there should be at least one outgoing XHR request to http://localhost:9001/tables/unicode.dis, which is not present in your screenshots... which is really fishy.

There are at least two ways to go from here:

1. You can step through the gui/main thread and the web worker code with a debugger to check why the request is not made at all. (Why enableOnDemandTableLoading does not work.)
2. You can compile a fat binary that has already all required table files bundled. The README has instructions for the process. Older versions of liblouis/js-build contain a build with all table files backed into the binary (31MB), which you could use to test the setup. Afterwards compile your own binary only containing unified englisch braille (or whatever table files you need).

You could also use setLogLevel to increase the number of messages from the C source code. But the C code is most likely not the culprit.

[faddah]

Hello again, @reiner-dolp ,

Thank you again for replying so promptly, it is very much appreciated.

I agree with you it's fishy that there are no XHR requests for the LibLouis tables, either unicode.dis or en-ueb-g2.ctb.

However, in a strange way, that "kind of" makes sense, because, remember, the console logs show that Cannot resolve table 'tables/unicode.dis' and tables/unicode.dis,tables/en-ueb-g2.ctb could not be compiled. So maybe the get loaded later and that is why the initial loading of the page gives those errors? I'm just spitballing here.

Also, my boss just reminded me above that I didn't give you any of the HEAD request information. Sorry for forgetting to do that. Here it is now —

Each of the 10 requests that get loaded with the Storybook page in my Chrome (for Mac OS) browser all have GET Requests that return positive either as 200 OK (from either the disk cache or memory cache) or 304 Not Modified, none of the requests are HEAD. I have done screen shots of each and they are at the bottom of this response.

So I tried the following —

The main WebPack request, __webpack_hmr, has a URL of http://localhost:9001/__webpack_hmr. If I send that through the command line as you say, with: curl -X HEAD http://localhost:9001/__webpack_hmr, i get back:

Warning: Setting custom HTTP method to HEAD with -X/--request may not work the 
Warning: way you want. Consider using -I/--head instead.

and then, after 30 - 60 seconds, it comes back with:

curl: (52) Empty reply from server

...and exits.

If I do, as it suggests, curl -I http://localhost:9001/__webpack_hmr --head, it waits for 30 - 60 seconds, and again returns curl: (52) Empty reply from server again.

Again, to remind you, if I put http://localhost:9001/tables/unicode.dis or http://localhost:9001/tables/en-ueb-g2.ctb on the browser URL, it downloads that exact table file from LibLouis into my downloads folder. I know it’s that correct file because when i open the downloaded file in VS Code, it is, indeed, that LibLouis Table file. So it is seeing the files in there.

if i do curl -I http://localhost:9001/tables/unicode.dis --head, however, I get the following —

HTTP/1.1 404 Not Found
X-Powered-By: Express
Content-Security-Policy: default-src 'self'
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=utf-8
Content-Length: 158
Date: Fri, 18 May 2018 21:43:31 GMT
Connection: keep-alive

and for curl -I http://localhost:9001/tables/en-ueb-g2.ctb --head, i get —

HTTP/1.1 404 Not Found
X-Powered-By: Express
Content-Security-Policy: default-src 'self'
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=utf-8
Content-Length: 160
Date: Fri, 18 May 2018 21:48:14 GMT
Connection: keep-alive

And, again, I don't wish to belabor this point, but even thought the HEAD returns a 404 Not Found on each of those table requests, if i put their URLs into the Chrome browser itself, it automatically downloads that correct LibLouis table file entact.

So any thoughts on the above? Please let me know and I will pursue what you said about reading your README.md on the section of compiling the fat binary and see if I can do that and make it work for us also.

Also, my boss thought it strange also that neither of those tables, unicode.dis & en-ueb-g2.ctb were shown loading in the network tab of Chrome Dev Tools, and he was wondering if the worker maybe didn’t show the requests in the same thread, and that's why we don't see the requests? Would that be worth pursuing here?

Thank you again for your prompt & helpful replies — any help you could give us in making this work is most appreciated. Below are those screen shots. And by the way, the 296124e2f136a52937d2e3458d7fe4d5.js & 1e2709bdd659364377c2f9f5bc2dab80.js turn out to be part of your liblouis-js files, if that helps.

best,

— faddah

[faddah]

@reiner-dolp - any further help here in making this work? i've posted screen shots of all the pieces being loaded in the network tab of Chrome Dev Tools in my last post above. 👆 👆 👆

please let us know when you have a chance.

best,

— faddah

[faddah]

hello, @reiner-dolp - just checking back in to see if there is any further help you could offer based on the information i have given above? 👆 👆 👆

please let us know when you have a chance.

best,

— faddah

[reiner-dolp]

I was curious about HEAD requests as (if I remember correctly) emscripten will make a HEAD request first, before downloading any data using a GET request to get the allocation size. So your server should be able to handle both requests correctly.

As there is no outgoing request for http://localhost:9001/tables/unicode.dis at all, the information provided by you is useless for further debugging of this issue.

[faddah]

@reiner-dolp,

thank you for responding.

ok, so what information can i provide for you that would assist us in debugging this? would really like to get this working for our client, please?

best,

— faddah

[reiner-dolp]

Closing this since I cannot reproduce the issue. Given all the frameworks used, this is probably an issue external to liblouis.