data-forge / data-forge-ts

The JavaScript data transformation and analysis toolkit inspired by Pandas and LINQ.
http://www.data-forge-js.com/
MIT License
1.34k stars 77 forks source link

Unable to build #14

Closed theNewFlesh closed 5 years ago

theNewFlesh commented 6 years ago

I am unable to build my typescript web app with dataforge.

I get the following errors to do with the fs module:

ERROR in ./node_modules/data-forge/build/index.js
Module not found: Error: Can't resolve 'fs' in '/home/vagrant/kesm/apps/threescan-data-viewer/node_modules/data-forge/build'
 @ ./node_modules/data-forge/build/index.js 157:17-30 253:17-30 264:17-30
 @ ./node_modules/ts-loader!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./src/ComplexRoiSelector.vue
 @ ./src/ComplexRoiSelector.vue
 @ ./src/index.ts

ERROR in ./node_modules/data-forge/build/lib/dataframe.js
Module not found: Error: Can't resolve 'fs' in '/home/vagrant/kesm/apps/threescan-data-viewer/node_modules/data-forge/build/lib'
 @ ./node_modules/data-forge/build/lib/dataframe.js 4377:21-34 4401:17-30 4433:21-34 4457:17-30
 @ ./node_modules/data-forge/build/index.js
 @ ./node_modules/ts-loader!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./src/ComplexRoiSelector.vue
 @ ./src/ComplexRoiSelector.vue
 @ ./src/index.ts

My code literally just tries to import dataforge and instantiate a dataframe with no data.

ashleydavis commented 6 years ago

Thanks for raising an issue.

Are you using Angularjs?

Can you please submit a working code sample? You could give me a link to a github repo containing the code or send me a zip file.

theNewFlesh commented 6 years ago

No, I am using Vue.

theNewFlesh commented 6 years ago

The repo is private, so I cannot share it. I am not sure how to set up a working coding sample in say codesandbox, because I did not create the build system.

For all intents and purposes, I am using Typescript, Vue , Webpack and Dataforge.

theNewFlesh commented 6 years ago

Here is an example .vue file:

<template>
    <div>
    </div>
</template>

<script lang="ts">
    import * as dataforge from "data-forge";
    import Vue from "vue";
    import { Component, Prop } from "vue-property-decorator";

    @Component
    export default class ComplexRoiSelector extends Vue {
        public df = dataforge;
    }
</script>

<style lang="scss">
</style>
theNewFlesh commented 6 years ago

Then after build in the console, I would resolve the Vue component to a variable vm, and call vm.dataforge.DataFrame(foo) for example.

theNewFlesh commented 6 years ago

If I comment out the all the require('fs') lines in dataframe.js, then only the first error shows up.

theNewFlesh commented 6 years ago

So, that tells me that nesting the import inside the function definition does not protect the library from requiring fs. Additonally, fs is somehow required elsewhere (ie index.js).

ashleydavis commented 6 years ago

I know people have had a similar problem with the new version of Angularjs (but not an older version oddly enough).

I was considering removing 'fs' from the core library and make it a plugin. Although this makes Data-Forge less usable in Node.js.

Can you think of any other way to solve the problem?

Can you please give a working code sample via GitHub or email me a zip file? I don't have time to learn Vue and put together an example so I'd really appreciate your help on this.

theNewFlesh commented 6 years ago

https://codesandbox.io/s/l9mq2xxv6m

theNewFlesh commented 6 years ago

Not sure if that is exactly correct, as codesandbox will likely have a slightly different build system.

theNewFlesh commented 6 years ago

Look in HelloWorld.vue

ashleydavis commented 6 years ago

Thanks for the example code.

I think I'm definitely going to have to remove the 'fs' code from Data-Forge. My plan is to remove it a different npm library.

So in the future instead of the readFile/writeFile functions being built in, you have to include the separate library like this:

const dataForge = require('data-forge');
require('data-forge-fs');

That will will get rid of Data-Forge's dependency on the 'fs' module and will allow it to work under Vue and Angularjs.

ashleydavis commented 6 years ago

Actually I think I may have a simpler solution!

While looking at the code and considering what I was going to do about the problem I spotted a global include of 'fs'. It's possible that this is what is causing your problem!

Please update to using Data-Forge 1.2.6 and let me know if it fixes your problem!

theNewFlesh commented 6 years ago

Just tried with 1.2.6 Still no dice.

Module not found: Error: Can't resolve 'fs' in '/home/vagrant/kesm/apps/threescan-data-viewer/node_modules/data-forge/build'
Module not found: Error: Can't resolve 'fs' in '/home/vagrant/kesm/apps/threescan-data-viewer/node_modules/data-forge/build'
ashleydavis commented 6 years ago

Ok thanks for trying it out.

I'll continue to extract 'fs' into a separate module.

ashleydavis commented 6 years ago

I've started the process of removing 'fs' from Data-Forge.

This will take some time to complete, but already I have published a 'beta' version of Data-Forge with 'fs' removed.

Can you please try using the beta package of Data-Forge with your project instead of the regular package:

http://npmjs.com/package/data-forge-beta

Please let me know how that goes. If that works for you then I'll know it's worth completing the change.

theNewFlesh commented 6 years ago

Worked like a charm. Thank you.

ashleydavis commented 6 years ago

That's great. Thanks for your help!

Please be sure to star this repo if you haven't already.

I'll come back to you again once this code change has been rolled into the production release.

theNewFlesh commented 6 years ago

Awesome sauce. Thank you for your time.

markledwich2 commented 6 years ago

For me, this was a problem with Webpack. https://github.com/webpack-contrib/css-loader/issues/447

Adding node: { fs: 'empty' } to webpack config file resolve the problem for me.

ashleydavis commented 5 years ago

Data-Forge 1.3.0 is officially released. The file system functions have been extracted from the core API to the new file system module: https://github.com/data-forge/data-forge-fs

This should help Data-Forge be more usable in the browser and specifically with the latest AngularJS.

If you were using "data-forge-beta" please change over to the regular "data-forge".