Problem in SimpleJson with Fable 3 output using webpack

[tomcl]

Description

This is possibly not a fable 3 issue, but probably some interaction between fable 3, fable.core, webpack. And maybe just my setup. Or maybe a SimpleJson issue. So I'm posting here only in case anyone has ideas because I'm out of them and it is breaking my build with fable 3.

The symptoms are that an F# call to SimpleJson.tryParseAs<ComponentType> in an electron renderer process behaves differently between the electron-webpack dev envt (correct) and the electron-webpack compile followed by electron-builder built envt.

This same code works fine both ways under fable 2, and the port to fable 3 did not change the webpack setup except as needed, e.g. deleting fable_loader and fable_compiler, changing entry to a js file. That is of course a big change.

I'm thinking that this maybe some library pick-up issue, but I am not that confident to debug this. One possibly relevant issue is that in the webpack processing, in both dev and build cases, I get a whole load of warning messages about inconsistent casing of functions in the Fable library. I've put a few of these (they are all similar, covering many library functions) at the end of this issue as an appendix. I'd guess somone can rule these out for me as implicated?

Repro code

Please provide the F# code to reproduce the problem or a link to the REPL. Ideally, it should be possible to easily turn this code into a unit test.

In https://github.com/tomcl/issie/tree/repro-build-bug

after initial build, or subsequent npm run dev the dev envt will run. the function checkJson below will run from init and print out in dev tools console the correct answer (Ok ....)

after npm run dist followed by running the built binaries e.g. dist\win_unpacked\issie.exe the app starts again. In dev tools window the checkjson printout is now an Error ...

I looked at the bundled JS source code for checkJson and the (HM loaded) source code for checkJson under dev tools in the two cases (dev and built). It is as follows.

• F# source code

let checkJson () =
    let json = """{"H":43,"Id":"60dd5d7b-6466-e522-7295-19e9ee39def8","InputPorts":[],
                "Label":"ZERO(3:0)","OutputPorts":[{"HostId":"60dd5d7b-6466-e522-7295-19e9ee39def8",
                "Id":"8120bfaf-432f-8aa9-feba-26fa433be4a6","PortNumber":0,"PortType":"Output"}],
                "Type":{"Input":4},"W":66,"X":370,"Y":208} """
    Json.tryParseAs<CommonTypes.Component> json
    |> printfn "%A"

• dev code: renderer.fs.js behaves right

import { SimpleJson_tryParse } from "./.fable/Fable.SimpleJson.3.17.0/SimpleJson.fs.js";

...

export function checkJson() { let matchValue, inputJson, typeInfo; const json = "{\"H\":43,\"Id\":\"60dd5d7b-6466-e522-7295-19e9ee39def8\",\"InputPorts\":[],\r\n \"Label\":\"ZERO(3:0)\",\"OutputPorts\":[{\"HostId\":\"60dd5d7b-6466-e522-7295-19e9ee39def8\",\r\n \"Id\":\"8120bfaf-432f-8aa9-feba-26fa433be4a6\",\"PortNumber\":0,\"PortType\":\"Output\"}],\r\n \"Type\":{\"Input\":4},\"W\":66,\"X\":370,\"Y\":208} "; let arg10; try { arg10 = (new FSharpResult$2(0, (matchValue = SimpleJson_tryParse(json), (matchValue != null) ? (inputJson = matchValue, (typeInfo = createTypeInfo(Component$reflection()), Convert_fromJson(inputJson, typeInfo))) : (() => { throw (new Error("Couldn\u0027t parse the input JSON string because it seems to be invalid")); })()))); } catch (ex) { arg10 = (new FSharpResult$2(1, ex.message)); } const clo1 = toConsole(printf("%A")); clo1(arg10); }

* Built code (from bundle, pretty-printed) - behaves wrong

function checkJson() {
    let r, n, a, i;
    try {
        i = new fable_library_3_0_0_nagareyama_rc_007_Choice_FSharpResult$2(0,(r = SimpleJson_tryParse('{"H":43,"Id":"60dd5d7b-6466-e522-7295-19e9ee39def8","InputPorts":[],\r\n                "Label":"ZERO(3:0)","OutputPorts":[{"HostId":"60dd5d7b-6466-e522-7295-19e9ee39def8",\r\n                "Id":"8120bfaf-432f-8aa9-feba-26fa433be4a6","PortNumber":0,"PortType":"Output"}],\r\n                "Type":{"Input":4},"W":66,"X":370,"Y":208} '),
        null != r ? (n = r,
        a = createTypeInfo(Component$reflection()),
        Convert_fromJson(n, a)) : (()=>{
            throw new Error("Couldn't parse the input JSON string because it seems to be invalid")
        }
        )()))
    } catch (r) {
        i = new fable_library_3_0_0_nagareyama_rc_007_Choice_FSharpResult$2(1,r.message)
    }
    String_toConsole(String_printf("%A"))(i)
}

### Expected and actual results

I expect this function always to succeed with an Ok result (as it does in the dev envt).

### Related information

* Fable version: `3.0.0-nagareyama-rc-007`
* Operating system: win10

### Appendix

Sample warning messages from webpack processing fable 3 output (identical for dev and build processing)

WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Types.js There are multiple modules with names that only differ in casing. This can lead to unexpected behavior when compiling on a filesystem with other case-semantic. Use equal casing. Compare these module identifiers:

• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js Used by 368 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\UI\ModelType.fs.js

• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js Used by 239 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Common\Helpers.fs.js @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Types.js @ ./src/Renderer/UI/ModelType.fs.js @ ./src/Renderer/Renderer.fs.js @ multi ./src/Renderer/Renderer.fs.js ./src/Renderer/scss/main.scss ./static/index.html

  WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Util.js
  There are multiple modules with names that only differ in casing. This can lead to unexpected behavior when compiling on a filesystem with other case-semantic. Use equal casing. Compare these module identifiers:

• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Util.js Used by 413 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\Renderer.fs.js

• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Util.js Used by 165 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Common\WidthInferer.fs.js @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Util.js @ ./src/Renderer/Renderer.fs.js @ multi ./src/Renderer/Renderer.fs.js ./src/Renderer/scss/main.scss ./static/index.html

  WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/lib/big.js There are multiple modules with names that only differ in casing. This can lead to unexpected behavior when compiling on a filesystem with other case-semantic. Use equal casing. Compare these module identifiers:

• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer.fable\fable-library.3.0.0-nagareyama-rc-007\lib\big.js Used by 11 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Decimal.js
• C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer.fable\fable-library.3.0.0-nagareyama-rc-007\lib\big.js Used by 11 module(s), i. e. C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer.fable\fable-library.3.0.0-nagareyama-rc-007\Decimal.js @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/lib/big.js
  @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Decimal.js
  @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/String.js
  @ ./src/Renderer/Renderer.fs.js @ multi ./src/Renderer/Renderer.fs.js ./src/Renderer/scss/main.scss ./static/index.html

[alfonsogarciacaro]

I need to look into this in more detail. Sorry, what version of SimpleJson were you using at the end. Maybe @Zaid-Ajaj can look into this too?

About the webpack warnings, seems that in some cases the path includes src/Renderer and in others src/renderer. When I tested your repo I noticed the Renderer folder is in upper case, but your commands here use lower case, maybe it's related? Or is there some other place in your config that uses lower case instead of upper case.

[Zaid-Ajaj]

I can't say for sure why this isn't working :/ @tomcl When you say:

I expect this function always to succeed with an Ok result (as it does in the dev envt).

Then the function somehow errors out when built for production and instead gives you Error back. What is the message you get back exactly when this happens?

@alfonsogarciacaro Could this be a polyfill problem because I see the config files aren't using babel loader to process Fable's output before generating the eventual JS bundle

[tomcl]

Thanks for this. Good point about case: I will consistently use lower case for files. It worries me a bit that a case error which would maybe break under linux can so easily not be an obvious error in windows. I was thinking that file references under npm would always be case insensitive.

@Zaid-Ajaj the Error string I get is:

Error (Cannot convert {"H":65,"Id":"a46275de-c854-18f1-a9bd-929544c3336b","InputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"8f654572-db8a-f3f4-5a01-71f470db0cc7","PortNumber":0,"PortType":"Input"}],"Label":"G1","OutputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"b911a531-46f2-5e15-8e6e-39a38c49af69","PortNumber":0,"PortType":"Output"}],"Type":"Not","W":42,"X":379,"Y":60} to CommonTypes.Component)

When I did a lot of A/B comparison, simplifying the type and its JSON input, I was sometimes getting a more informative error about not matching Input:4 (which should match to a D.U. | Input of int.). But I was hand-crafting the JSON and that is not reliable: perhaps I temporarily had the wrong JSON.

More interestingly: I have an A/B comparison in which this operation using the same type + subtypes, renamed, written in renderer.fs, works normally. I cut and pasted the type definitions adding 2 to each type name. Whereas the original type written in CommonTypes module (which itself is in a dependent project) generates this error. In all cases there is no problem with the dev envt app, everything works.

I'm thinking perhaps this could be some issue about type reflection?

I will do some more investigation moving the renamed type to different modules.

Re polyfill. Polyfill is again something I have never had to think about and therefore don't understand. In both these cases (dev and built) the code runs on the same Electron 9 code - and this is a recent copy of Chrome.

Re versions. See below. Fable.SimpleJson is v3.17 from paket.lock. I thought maybe that might be a solution because of what I found out in my next post, but it does not quite seem to be relevant. However I do not yet exactly understand how paket and npm versions interact with webpack. What changes between different dotnet projects, if anything? My paket setup has the two projects use here in the same group.

[tomcl]

OK. I put the sometimes does not parse with SimpleJson.tryParseAs type into a submodule and inserted it, inside submodule named differently to remove possibility of error, at different points in my source code structure.

It does not work (for bundled build only) when the type definition used by SimpleJson.tryParseAs comes from a project dependency. It is fine when in the same file, or an earlier file/module in the same project. That gives me a workaround of coalescing all my projects but that would mean I could not do dotnet testing of the pure F# parts.

I checked that fable3 was correctly recompiling the dependent module code, from timestamps it is.

Given the protection offered by the submodule I cannot see any possibility of type shadowing.

See below for exact details.

Re versions, see Appendix 3 (all latest I think). Given that the different projects have (possibly) different paket references I'm wondering whether there could be a dependency issue there. However the problem I have is not different behaviour when the tryParseAs function is in different modules, but different behaviour when the type it is parsing for is in different modules. In addition both renderer and common projects are in the same paket group, and have almost the same references files. But I have to say I have not much clue how paket setup works so maybe have something weird there. I have run femto on both projects and all is good, but I'm not sure femto can check very much here yet.

Code used for testing (for type definition see Appendix) CommonTypes is in an F#/dotnet dependent project common that gets compiled with Fable3 together with renderer.

In this code Renderer is the entry renderer process source module contained in file Renderer.fs. The project structure is renderer.fsproj --> common.fsproj

see also Appendix 2 because no-one has yet given me any idea about the weird identical but differently cased library functions that webpack complains about finding. I'd like to understand that.

    Json.tryParseAs<TestJson.Component> json3 // error
    |> printfn "TestJson: \n%A\n"
    Json.tryParseAs<ModelType.TestJson2.Component> json3 // error
    |> printfn "TestJson2: \n%A\n"
    Json.tryParseAs<CommonTypes.TestJson3.Component> json3
    |> printfn "testJson3: \n%A\n"

Output from SimpleJson.tryParseAs

TestJson: 
Ok ({ Id = a46275de-c854-18f1-a9bd-929544c3336b
  Type = Not
  Label = G1
  InputPorts = [{ Id = 8f654572-db8a-f3f4-5a01-71f470db0cc7
  PortNumber = 0
  PortType = Input
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  OutputPorts = [{ Id = b911a531-46f2-5e15-8e6e-39a38c49af69
  PortNumber = 0
  PortType = Output
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  X = 379
  Y = 60
  H = 65
  W = 42 })

TestJson2: 
Ok ({ Id = a46275de-c854-18f1-a9bd-929544c3336b
  Type = Not
  Label = G1
  InputPorts = [{ Id = 8f654572-db8a-f3f4-5a01-71f470db0cc7
  PortNumber = 0
  PortType = Input
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  OutputPorts = [{ Id = b911a531-46f2-5e15-8e6e-39a38c49af69
  PortNumber = 0
  PortType = Output
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  X = 379
  Y = 60
  H = 65
  W = 42 })

testJson3: 
Error (Cannot convert {"H":65,"Id":"a46275de-c854-18f1-a9bd-929544c3336b","InputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"8f654572-db8a-f3f4-5a01-71f470db0cc7","PortNumber":0,"PortType":"Input"}],"Label":"G1","OutputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"b911a531-46f2-5e15-8e6e-39a38c49af69","PortNumber":0,"PortType":"Output"}],"Type":"Not","W":42,"X":379,"Y":60} to CommonTypes.TestJson3.Component)

Appendix - type definition used in tests

module TestJson =
    type PortType = Input | Output

    type Port = {
        Id : string
        // For example, an And would have input ports 0 and 1, and output port 0.
        // If the port is used in a Connection record as Source or Target, the Number is None. 
        PortNumber : int option
        PortType : PortType
        HostId : string
    }

    /// Name identified the LoadedComponent used.
    /// The labels define legends on symbol.
    /// Label strings are unique per CustomComponent.
    /// Multiple CustomComponent instances are differentiated by Component data.
    type CustomComponentType = {
        Name: string
        // Tuples with (label * connection width).
        InputLabels: (string * int) list
        OutputLabels: (string * int) list 
    }

    type Memory = {
        // How many bits the address should have.
        // The memory will have 2^AddressWidth memory locations.
        AddressWidth : int 
        // How wide each memory word should be, in bits.
        WordWidth : int
        // Data is a list of <2^AddressWidth> elements, where each element is a
        // 64 bit integer. This makes words longer than 64 bits not supported.
        // This can be changed by using strings instead of int64, but that is way
        // less memory efficient.
        Data : Map<int64,int64>
    }

    // Types instantiating objects in the Digital extension.
    type ComponentType =
        | Input of BusWidth: int | Output of BusWidth: int | IOLabel 
        | BusSelection of OutputWidth: int * OutputLSBit: int
        | Constant of Width: int * ConstValue: int
        | Not | And | Or | Xor | Nand | Nor | Xnor |Decode4
        | Mux2 | Demux2
        | NbitsAdder of BusWidth: int
        | Custom of CustomComponentType // schematic sheet used as component
        | MergeWires | SplitWire of BusWidth: int // int is bus width
        // DFFE is a DFF with an enable signal.
        // No initial state for DFF or Register? Default 0.
        | DFF | DFFE | Register of BusWidth: int | RegisterE of BusWidth: int 
        | AsyncROM of Memory | ROM of Memory | RAM of Memory // memory is contents

    /// JSComponent mapped to F# record.
    /// Id uniquely identifies the component within a sheet and is used by draw2d library.
    /// Label is optional descriptor displayed on schematic.
    type Component = {
        Id : string
        Type : ComponentType
        Label : string // All components have a label that may be empty.
        InputPorts : Port list
        OutputPorts : Port list
        X : int
        Y : int
        H : int
        W : int
    }

Appendix 2

    WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/lib/long.js
    There are multiple modules with names that only differ in casing.
    This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
    Use equal casing. Compare these module identifiers:
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\lib\long.js
        Used by 48 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Long.js
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\lib\long.js
        Used by 48 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Long.js
     @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/lib/long.js      
     @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Long.js
     @ ./src/Renderer/UI/PopupView.fs.js
     @ ./src/Renderer/Renderer.fs.js
     @ multi ./src/Renderer/Renderer.fs.js ./src/Renderer/scss/main.scss ./static/index.html
    Child HtmlWebpackCompiler:
         2 assets
        Entrypoint HtmlWebpackPlugin_0 = __child-HtmlWebpackPlugin_0
        Entrypoint HtmlWebpackPlugin_1 = __child-HtmlWebpackPlugin_1
        [0] ./node_modules/html-loader/dist/cjs.js?minimize=false!./dist/.renderer-index-template.html 322 bytes {0} [built]
        [1] ./node_modules/html-webpack-plugin/lib/loader.js!./static/index.html 610 bytes {1} [built]
    Child mini-css-extract-plugin node_modules/css-loader/dist/cjs.js??ref--6-1!node_modules/tippy.js/themes/material.css:
        Entrypoint mini-css-extract-plugin = *
           2 modules
    Child mini-css-extract-plugin node_modules/css-loader/dist/cjs.js??ref--8-1!node_modules/sass-loader/dist/cjs.js!src/Renderer/scss/main.scss:
        Entrypoint mini-css-extract-plugin = *
        [1] ./node_modules/css-loader/dist/cjs.js??ref--8-1!./node_modules/sass-loader/dist/cjs.js!./src/Renderer/scss/main.scss 236 KiB {0} [built]
            + 11 hidden modules

Appendix 3 Versions

nagareyama: rc007

other dotnet stuff: all

  "dependencies": {
    "bulma": "^0.9.0",
    "core-js": "^3.6.5",
    "cross-zip": "^3.1.0",
    "cross-zip-cli": "^1.0.0",
    "draw2d": "^1.0.38",
    "electron-devtools-installer": "^3.1.0",
    "electron-is-dev": "^1.2.0",
    "font-awesome": "^4.7.0",
    "jquery": "^3.5.1",
    "jquery-ui": "^1.12.1",
    "react": "^16.7.0",
    "react-dom": "^16.7.0",
    "react-tooltip": "^4.2.8",
    "rechart": "^0.0.1",
    "source-map-support": "0.5.19",
    "tippy.js": "^6.2.6"
  },
  "devDependencies": {
    "bufferutil": "^4.0.1",
    "devtron": "^1.4.0",
    "electron": "^9.1.0",
    "electron-builder": "^22.7",
    "electron-webpack": "^2.8.2",
    "file-loader": "^3.0.1",
    "html-webpack-plugin": "^4.3.0",
    "loglevel": "^1.6.8",
    "node-sass": "^4.12.0",
    "remotedev": "^0.2.9",
    "resolve-url-loader": "^3.1.1",
    "sass": "^1.26.10",
    "sass-loader": "^7.3.1",
    "url-loader": "^4.1.0",
    "utf-8-validate": "^5.0.2",
    "webpack": "^4.43.0",
    "webpack-cli": "^3.3.12"

framework: auto-detect
source https://api.nuget.org/v3/index.json
nuget Fable.Browser.Css

group Electron
    source https://api.nuget.org/v3/index.json

    nuget Fable.Core 
    nuget Fable.Electron 
    nuget Fable.Elmish 
    nuget Fable.Elmish.Debugger
    nuget Fable.Elmish.HMR 
    nuget Fable.Elmish.React 
    nuget Fable.Import.Node
    clitool dotnet-fable
    nuget Fable.Browser.Dom

    nuget Fable.SimpleJson
    nuget Fable.Promise 
    nuget Fable.React  
    nuget FSharp.Core  redirects:force
    nuget Fulma

group Testing
    source https://api.nuget.org/v3/index.json

    nuget Expecto 
    nuget Expecto.BenchmarkDotNet
    nuget Expecto.FsCheck 
    nuget Expecto.TestResults 
    nuget Foq 
    nuget FSharp.Core redirects:force
    nuget FSharp.Data 
    nuget Logary 
    nuget Microsoft.NETCore.Platforms

group FakeBuild
    source https://api.nuget.org/v3/index.json

    storage: none

    nuget Fake.Core.Target 
    nuget Fake.Core.ReleaseNotes 
    nuget Fake.DotNet.AssemblyInfoFile 
    nuget Fake.DotNet.Cli 
    nuget Fake.DotNet.MSBuild 
    nuget Fake.DotNet.Paket 
    nuget Fake.IO.FileSystem 
    nuget Fake.Tools.Git 
    nuget Fake.JavaScript.Npm 
    nuget FSharp.Json 
    nuget dotnet-fake
    nuget dotnet-fable

Femto run on renderer project

[11:24:35 INF] Analyzing project C:\github\issie\src\renderer\renderer.fsproj
[11:24:35 INF] Running dotnet restore against the project
[11:24:45 INF] Using npm for package management
[11:24:48 INF] Found package.json in C:\github\issie
[11:24:53 INF] Fable.Electron requires npm package electron
[11:24:53 INF]   | -- Required range >= 9.0 < 10 found in project file
[11:24:53 INF]   | -- Used range ^9.1.0 in package.json
[11:24:53 INF]   | -- ✔ Installed version 9.3.4 satisfies required range >= 9.0 < 10[11:24:53 INF] Fulma requires npm package bulma
[11:24:53 INF]   | -- Required range >= 0.9.0 found in project file
[11:24:53 INF]   | -- Used range ^0.9.0 in package.json
[11:24:53 INF]   | -- ✔ Installed version 0.9.1 satisfies required range >= 0.9.0 

Femto run on common project

[11:26:50 INF] Analyzing project C:\github\issie\src\common\common.fsproj
[11:26:50 INF] Running dotnet restore against the project
[11:26:55 INF] Using npm for package management
[11:26:58 INF] Found package.json in C:\github\issie
[11:26:58 INF] Fable.Electron requires npm package electron
[11:26:58 INF]   | -- Required range >= 9.0 < 10 found in project file
[11:26:58 INF]   | -- Used range ^9.1.0 in package.json
[11:26:58 INF]   | -- ✔ Installed version 9.3.4 satisfies required range >= 9.0 < 10[11:26:58 INF] Fulma requires npm package bulma
[11:26:58 INF]   | -- Required range >= 0.9.0 found in project file
[11:26:58 INF]   | -- Used range ^0.9.0 in package.json
[11:26:58 INF]   | -- ✔ Installed version 0.9.1 satisfies required range >= 0.9.0   

paket.references file for renderer:

Fable.Browser.Css
group Electron
   dotnet-fable
   Fable.Core
   Fable.Browser.Dom
   Fable.Electron
   Fable.Elmish
   Fable.Elmish.React
   Fable.Elmish.Debugger
   Fable.Elmish.HMR
   Fable.Node
   Fable.Promise
   Fable.React
   Fulma
   Fable.SimpleJson

paket.references file for common

group Electron
    dotnet-fable
    Fable.Core
    Fable.Browser.Dom
    Fable.Electron
    Fable.Elmish
    Fable.Elmish.React
    Fable.Elmish.Debugger
    Fable.Elmish.HMR
    Fable.Node
    Fable.Promise
    Fable.React
    Fulma    
    Fable.SimpleJson

[Zaid-Ajaj]

@tomcl The error you are getting back from SimpleJson is most generic one where we don't know much about the the type itself. This makes me confident that the error has to do with accessing reflection metadata across projects

A small repro would be great!

[tomcl]

@Zaid-Ajaj

Small repro

NB - this is small in terms of decoupling the failing json parse from the rest of the project. I guess you might want me to try reducing the size of the parse too! It has been a big adventure for me to cut down the build so much. I'll next have a go at reducing the number of nested type definitions.

https://github.com/tomcl/issie/tree/repro-build-bug

The (slimmed down) example is the renderer process which consists of src\renderer\renderer.fs and (separate project) src\commontypes\commontypes.fs.

To run under electron there is also a (not slimmed down) main process main.fs.

See issie.sln.

The files now contain the bug repro and nothing else.

Repro (win)

(1) clone repo branch as above (2) build (3) exit dev envit (which shows no error if you do Ctrl\Shift-I to see dev tools console printout) (4) npm run dist -- build the bundled envt, asar is switched off so you can see files gen. (5) dist\win-unpacked\issie.exe -- run the bundled code (6) Ctrl/Shift-I - open dev tools and look at console printout.

Should see:

Starting renderer...
TestJson: 
Ok ({ Id = a46275de-c854-18f1-a9bd-929544c3336b
  Type = Not
  Label = G1
  InputPorts = [{ Id = 8f654572-db8a-f3f4-5a01-71f470db0cc7
  PortNumber = 0
  PortType = Input
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  OutputPorts = [{ Id = b911a531-46f2-5e15-8e6e-39a38c49af69
  PortNumber = 0
  PortType = Output
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  X = 379
  Y = 60
  H = 65
  W = 42 })

testJson3: 
Ok ({ Id = a46275de-c854-18f1-a9bd-929544c3336b
  Type = Not
  Label = G1
  InputPorts = [{ Id = 8f654572-db8a-f3f4-5a01-71f470db0cc7
  PortNumber = 0
  PortType = Input
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  OutputPorts = [{ Id = b911a531-46f2-5e15-8e6e-39a38c49af69
  PortNumber = 0
  PortType = Output
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  X = 379
  Y = 60
  H = 65
  W = 42 })

done

Actually see:

Starting renderer...
TestJson: 
Ok ({ Id = a46275de-c854-18f1-a9bd-929544c3336b
  Type = Not
  Label = G1
  InputPorts = [{ Id = 8f654572-db8a-f3f4-5a01-71f470db0cc7
  PortNumber = 0
  PortType = Input
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  OutputPorts = [{ Id = b911a531-46f2-5e15-8e6e-39a38c49af69
  PortNumber = 0
  PortType = Output
  HostId = a46275de-c854-18f1-a9bd-929544c3336b }]
  X = 379
  Y = 60
  H = 65
  W = 42 })

testJson3: 
Error (Cannot convert {"H":65,"Id":"a46275de-c854-18f1-a9bd-929544c3336b","InputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"8f654572-db8a-f3f4-5a01-71f470db0cc7","PortNumber":0,"PortType":"Input"}],"Label":"G1","OutputPorts":[{"HostId":"a46275de-c854-18f1-a9bd-929544c3336b","Id":"b911a531-46f2-5e15-8e6e-39a38c49af69","PortNumber":0,"PortType":"Output"}],"Type":"Not","W":42,"X":379,"Y":60} to CommonTypes.TestJson3.Component)

done

[tomcl]

@Zaid-Ajaj

Ok - I've cut down the example. It seems that reflection fails on all bundled types from a different F# project.

Steps - as above.

Actual:

Starting renderer...

Parsing Renderer.TestJson: 
Ok ({ H = 10 })

Parsing CommonTypes.TestJson: 
Error (Cannot convert {"H":10} to CommonTypes.TestJson.Component)

done

Expected: the error output should be the same as the Ok output.

[tomcl]

Could it be something to do with the not understood by me warning errors from webpack:

    WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Types.js 
    There are multiple modules with names that only differ in casing.
    This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
    Use equal casing. Compare these module identifiers:
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js
        Used by 236 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\Renderer.fs.js
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js
        Used by 6 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Common\CommonTypes.fs.js
     @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Types.js
     @ ./src/Renderer/Renderer.fs.js
     @ multi ./src/Renderer/Renderer.fs.js ./static/index.html

    WARNING in ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Util.js  
    There are multiple modules with names that only differ in casing.
    This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
    Use equal casing. Compare these module identifiers:
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Util.js
        Used by 169 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js
    * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Util.js
        Used by 22 module(s), i. e.
        C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0-nagareyama-rc-007\Types.js
     @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Util.js
     @ ./src/Renderer/.fable/fable-library.3.0.0-nagareyama-rc-007/Types.js
     @ ./src/Renderer/Renderer.fs.js
     @ multi ./src/Renderer/Renderer.fs.js ./static/index.html

And lots of others like for nagareyama libraries (nothing that I define).

[tomcl]

Here is the pretty-printed bundled (not working) code for checkJson obtained by running (npm run dist) the small bug repro. In this code Json parsing using target type Renderer.testJson works, CommonTypes.testJson does not work. They should be identical.

It seems to me that either fable_library_3_0_0_nagareyama_rc_007_Reflection_TypeInfo has a bug when code is minified, or the parameters it is given here are wrong? This same code works fine when run in dev envt.

    function Component$reflection() {
        return function Reflection_record_type(e, t, n, r) {
            return new fable_library_3_0_0_nagareyama_rc_007_Reflection_TypeInfo(e,t,n,void 0,r)
        }("CommonTypes.TestJson.Component", [], CommonTypes_fs_Component, ()=>[["H", Ae]])
    }
    class Renderer_fs_TestJson_Component extends Record {
        constructor(e) {
            super(),
            this.H = 0 | e
        }
    }
    function TestJson_Component$reflection() {
        return record_type("Renderer.TestJson.Component", [], Renderer_fs_TestJson_Component, ()=>[["H", r]])
    }
    function checkJson() {
        let e, t, n, r, o, i;
        let a;
        try {
            a = new Choice_FSharpResult$2(0,(e = SimpleJson_tryParse('{"H":10}'),
            null != e ? (t = e,
            n = createTypeInfo(TestJson_Component$reflection()),
            Convert_fromJson(t, n)) : (()=>{
                throw new Error("Couldn't parse the input JSON string because it seems to be invalid")
            }
            )()))
        } catch (e) {
            a = new Choice_FSharpResult$2(1,e.message)
        }
        let s;
        toConsole(printf("\n\nParsing Renderer.TestJson: \n%A\n"))(a);
        try {
            s = new Choice_FSharpResult$2(0,(r = SimpleJson_tryParse('{"H":10}'),
            null != r ? (o = r,
            i = createTypeInfo(Component$reflection()),
            Convert_fromJson(o, i)) : (()=>{
                throw new Error("Couldn't parse the input JSON string because it seems to be invalid")
            }
            )()))
        } catch (e) {
            s = new Choice_FSharpResult$2(1,e.message)
        }
        toConsole(printf("\n\nParsing CommonTypes.TestJson: \n%A\n"))(s)
    }

[tomcl]

And, the same without minification, which does not change the problem:

class CommonTypes_fs_Component extends Types_Record {
  constructor(H) {
    super();
    this.H = H | 0;
  }

}
function Component$reflection() {
  return Reflection_record_type("CommonTypes.TestJson.Component", [], CommonTypes_fs_Component, () => [["H", Reflection_int32_type]]);
}
// CONCATENATED MODULE: ./src/Renderer/Renderer.fs.js

class Renderer_fs_TestJson_Component extends Record {
  constructor(H) {
    super();
    this.H = H | 0;
  }

}
function TestJson_Component$reflection() {
  return record_type("Renderer.TestJson.Component", [], Renderer_fs_TestJson_Component, () => [["H", int32_type]]);
}
function checkJson() {
  let matchValue, inputJson, typeInfo, matchValue_1, inputJson_1, typeInfo_1;
  const json = "{\"H\":10}";
  let arg10;

  try {
    arg10 = new Choice_FSharpResult$2(0, (matchValue = SimpleJson_tryParse(json), matchValue != null ? (inputJson = matchValue, (typeInfo = createTypeInfo(TestJson_Component$reflection()), Convert_fromJson(inputJson, typeInfo))) : (() => {
      throw new Error("Couldn\u0027t parse the input JSON string because it seems to be invalid");
    })()));
  } catch (ex) {
    arg10 = new Choice_FSharpResult$2(1, ex.message);
  }

  const clo1 = toConsole(printf("\n\nParsing Renderer.TestJson: \n%A\n"));
  clo1(arg10);
  let arg10_1;

  try {
    arg10_1 = new Choice_FSharpResult$2(0, (matchValue_1 = SimpleJson_tryParse(json), matchValue_1 != null ? (inputJson_1 = matchValue_1, (typeInfo_1 = createTypeInfo(Component$reflection()), Convert_fromJson(inputJson_1, typeInfo_1))) : (() => {
      throw new Error("Couldn\u0027t parse the input JSON string because it seems to be invalid");
    })()));
  } catch (ex_1) {
    arg10_1 = new Choice_FSharpResult$2(1, ex_1.message);
  }

  const clo1_1 = toConsole(printf("\n\nParsing CommonTypes.TestJson: \n%A\n"));
  clo1_1(arg10_1);
}

[tomcl]

For my own project, I have an easy workaround I've now done which is putting the Json parse function in the same F# project as the types they parse. That will mostly be possible.

But this should maybe be noted as a feature of SimpleJson / Fable3 for others that the parsed type must be in the same F# project as the parsing function? I can't see anything unusual about my webpack setup so would expect this to break other people's code as well.

@Zaid-Ajaj

[alfonsogarciacaro]

So with the fix for the case of the path being passed as CLI argument, can we close this @tomcl?

[tomcl]

Thanks for all your work making a nicely sorted out compiler.

I’m pretty sure we can. Anyway I will be testing with Issie lots more before Xmas, and with a LOT of students next Term doing development, so if we find anything else we will tell you.

Just for my interest, when that bug did not build on your system, was that system windows? Or linux? Because I fixed a possible “renderer window does not open” bug on linux. It makes not much difference because very few people want it on linux, and it will work for them with WINE, but nice to know. And if it is not installing on windows I need to keep trying to find out why.

Best wishes, Tom

From: Alfonso Garcia-Caro notifications@github.com Sent: 04 December 2020 15:36 To: fable-compiler/Fable Fable@noreply.github.com Cc: Clarke, Thomas J W t.clarke@imperial.ac.uk; Mention mention@noreply.github.com Subject: Re: [fable-compiler/Fable] Problem in SimpleJson with Fable 3 output using webpack (#2285)

This email from notifications@github.commailto:notifications@github.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders listhttps://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address.

So with the fix for the case of the path being passed as CLI argument, can we close this @tomclhttps://github.com/tomcl?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/fable-compiler/Fable/issues/2285#issuecomment-738850768, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABZSB4KKM4DT6YXMC7H2XMTSTD6XXANCNFSM4T5FQV5Q.

[aaronmu]

I think we're experiencing the same issue.

Three projects; Client, Server, and Shared. We use Fable.Remoting to communicate between Client and Server. The types that go over the wire are defined in Shared.

Everything works in dev when we use fable watch, in production, we get the following error at runtime.

Cannot build proxy. Exepected type X to be a valid protocol definition which is a record of functions

cc @Zaid-Ajaj

[Zaid-Ajaj]

@aaronmu That is the same issue that @tomcl has: the reflection metadata are not being read correctly in a production build. Did you have try latest stable Fable 3?

[tomcl]

Are you using 3.0.0? If not upgrade to it and see if the issue goes away.

For my case there were two apparently different bugs which seemed to be related to incorrect type reflection.

Both went away when an issue to do with variable case in paths was sorted out (in 3.0.0 - rc-011 still has it). I sorted this out prior to 3.0.0 by aligning build paths (in case) with actual disk paths. On windows this was not otherwise necessary so there was no other warning of the incorrect casing. 3.0.0 incorporates lots of build path case insensitivity which should fix this anyway. Let me know if 3.0.0 does not fix your problems and I'll put back the erroneous casing and check 3.0.0 fixes that, which it should.

The mechanism for this is possibly related to multiple copies of library files existing in .fable (differing in casing of paths). But I don't myself know the details, that is just a guess.

It is possible that there is some other reflection issue at work here not solved in 3.0.0 however I have no evidence from my case that is true.

[tomcl]

OK - so not exactly good news.

I went back to my old bug repro. I was expecting to be able to switch it from non-working to working deterministically by altering paths and/or compiler. Unfortunately what happened is that it did not work for some time, switching compiler and path casing. Then it did work for some time, again switching paths but not casing. For the second working ones it was all with the later compiler. Howeve rthe later compiler also did not work, and did not work after an npm clean-install which deltes the place the multiple libraries are put.

So there is obviously something I am not understanding here. Maybe some level of caching that I'm not managing to clean?

The working builds (and these were dev builds - the difference previously had to do with variant casing in scripts) did not show an error mesage about muliple library paths. The non-working builds did show:

 WARNING in ./src/Renderer/.fable/fable-library.3.0.0/Reflection.js
  There are multiple modules with names that only differ in casing.
  This can lead to unexpected behavior when compiling on a filesystem with other case-semantic.
  Use equal casing. Compare these module identifiers:
  * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\Renderer\.fable\fable-library.3.0.0\Reflection.js
      Used by 3 module(s), i. e.
      C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\common\commontypes.fs.js
  * C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\.fable\fable-library.3.0.0\Reflection.js
      Used by 201 module(s), i. e.
      C:\github\issie\node_modules\babel-loader\lib\index.js??ref--4!C:\github\issie\src\renderer\renderer.fs.js
   @ ./src/Renderer/.fable/fable-library.3.0.0/Reflection.js
   @ ./src/common/commontypes.fs.js
   @ ./src/renderer/renderer.fs.js
   @ multi ./src/renderer/renderer.fs.js ./static/index.html

I am pretty sure that i was compiling with 3.0.0, and had done npm clean-install, and still got this message and it did not work. But maybe I did not compile everything, or something else.

Now, the same repo works whatever I do, and no warning as above. And it is running 3.0.0.

My suggestion is download your repo from scratch to eliminate any possibility of caching artifacts. Make sure you have dotnet-tools config set for fable 3.0.0. Buid everything. See if that gets rid of the problem.

Sorry not to be more definitive. there may be some issue here not yet understood.

@alfonsogarciacaro

[alfonsogarciacaro]

Hmm, I just created a fresh clone of issie and running dotnet tool restore && npm install && npm run dev worked fine for me. I could test the app without issue. I'm on macOS btw.

[tomcl]

Well I don't have any problems, @aaronmu maybe does! I can't find any replicable problem myself so I think all is good.

[aaronmu]

@aaronmu That is the same issue that @tomcl has: the reflection metadata are not being read correctly in a production build. Did you have try latest stable Fable 3?

@Zaid-Ajaj, I'll try to create a reproduce!

[alfonsogarciacaro]

Closing for now as the issue seems to be fixed, please feel free to reopen if you still have problems.

[tomcl]

I'll let you know if it comes back!