using dataforge in angular

[capaneus]

hi Ashley, I'm building an angular 6 application recently and was looking for some data processing lib. So happy to find data-forge as I used pandas and it's a great lib on python. Would you mind put some instruction on how to import data-forge into angular to be used there?

[ashleydavis]

Hi, I haven't used AngularJS since version 1, so please be aware that any instructions I have for you might be out of date.

For a start, Data-Forge is TypeScript so you might just be able to install it via npm then import it into your AngularJS app as you would any other TypeScript module.

I've used Data-Forge when I've made apps using AngularJS v1+ and I do have some instructions that will work for that.

Browser installation: https://github.com/data-forge/data-forge-ts/blob/master/docs/guide.md#browser-installation-and-setup

AngularJS v1+ usage: https://github.com/data-forge/data-forge-ts/blob/master/docs/guide.md#working-with-http-in-angularjs

Hope this helps. Please let me know how it goes. Maybe you could help me update the instructions noce you figure it out?

[yiu31802]

This worked perfectly to me, this lib can be "the" de-facto standard of dataframe in Angular! Thank you @ashleydavis so much, I will try to spread this library within my team:-)

import { DataFrame } from 'data-forge'

@Component({
selector: 'app-top',
templateUrl: './main.html',
styleUrls: ['./main.css']
})
export class TopPageComponent implements OnInit {

public constructor(
){}

public ngOnInit(): void {
    let df = new DataFrame({
        columns: {
            'regiment': ['Nighthawks', 'Nighthawks', 'Nighthawks', 'Nighthawks', 'Dragoons', 'Dragoons', 'Dragoons', 'Dragoons', 'Scouts', 'Scouts', 'Scouts', 'Scouts'],
            'company': ['1st', '1st', '2nd', '2nd', '1st', '1st', '2nd', '2nd','1st', '1st', '2nd', '2nd'],
            'TestScore': [4, 24, 31, 2, 3, 4, 24, 31, 2, 3, 2, 3]
        }
    });
    const pivotted = df.pivot(["regiment", "company"], "TestScore", testScores => testScores.average());
    console.log(pivotted.toArray())

[ashleydavis]

Thanks mate! I'll add that to the docs.

Don't forget to star the repo.

You might also be interested to know that I'm writing a book on working with data in JavaScript. Please help spread the word!

[capaneus]

nah, I'm not super lucky :-( @yiu31802, do you mind share a bit more how you setup it in the project? I tried different options, it gives me:

ERROR in ./node_modules/data-forge/build/index.js
Module not found: Error: Can't resolve 'fs' in 'E:\MyCode\flaskagular\project_planning\app\static\node_modules\data-forge\build'
ERROR in ./node_modules/data-forge/build/lib/dataframe.js
Module not found: Error: Can't resolve 'fs' in 'E:\MyCode\flaskagular\project_planning\app\static\node_modules\data-forge\build\lib'

and did some search seems it's normal for angular to not support fs... https://github.com/angular/angular-cli/issues/8272

how did you solve this problem?

[yiu31802]

Hi, @capaneus Weird, I have not met that error at all. My environment is Angular 5.0.5. But it makes also sense that Angular should not support fs in the browser... I will check my environment later and update you.

[ashleydavis]

Yeah you can't load files from the file system when running in the browser (whether you are using AngularJS, React, etc doesn't matter).

You can only use the readFile/writeFile functions under Node.js.

Don't use those functions in your code and it should work under AngularJS for you.

[capaneus]

hi Ashley, thanks and back to js again :-) I actually didn't use any of the fs function in the code, what I used to test is the code from yiu31802. It comes at the compiling phase, seems it complained about something in dataframe.js which I haven't triggered at all...

[ashleydavis]

That's odd. Can you please give me more information about how you are compiling it?

[ashleydavis]

Please give me more information about how you are trying to use Data-Forge with AngularJS. Otherwise I'll close this issue.

[maykon-oliveira]

I have de same problem! ERROR in ./node_modules/data-forge/build/index.js Module not found: Error: Can't resolve 'fs' in 'C:\Users\Maykon Oliveira\Documents\analise-das-escolas\node_modules\data-forge\build' ERROR in ./node_modules/data-forge/build/lib/dataframe.js Module not found: Error: Can't resolve 'fs' in 'C:\Users\Maykon Oliveira\Documents\analise-das-escolas\node_modules\data-forge\build\lib'

I don't know what I do, I need this library. Has anyone managed to solve it?

[ashleydavis]

Please share the broken code with me on github so I can run it and debug.

[maykon-oliveira]

The project repository I'm at the beginning

[ashleydavis]

@maykon-oliveira I got your code, installed dependencies and then ran it using npm start.

I see a bunch of errors in the console, but I don't think any of them relate to Data-Forge.

Can you give me more information about the problem you are experiencing?

[maykon-oliveira]

I was trying to fix it, I changed dependency to continue with the project. This code is a new commit, the error was in the previous one. I gave up to use data-forge.

[yiu31802]

Hi, @maykon-oliveira and @ashleydavis .

When I tried to add data-forge on a fresh Angular repository, indeed, the same error with fs occurred. I am trying to figure out which of many dependency package may implicitly resolve the issue in my data-forge working project. Note that our project is using DataForge in Angular even today flawlessly...

[ashleydavis]

Thank @yiu31802 please let me know what you come up with.

If anyone else finds a problem please submit working code for me to troubleshoot.

I used to use Angular version 1, but I've switched to React since then and I don't have time at the moment to learn the latest version of Angular. So I'll appreciate anyone who can submit code that demonstrates an issue.

[yiu31802]

Hi, @ashleydavis, @capaneus @maykon-oliveira

I think I understood what happened; it is simply a version issue. With Angular 5.x, it works, but something is getting wrong with Angular 6.x versions.

Here's the sample code, which is built on top of pure Angular 5 and adding data-forge. https://github.com/yiu31802/sample-angular5-with-data-forge

[ashleydavis]

Thanks @yiu31802 , hoping to look into it this week!

[ashleydavis]

@yiu31802 I've got your project, installed dependencies, run npm start and viewed the web page.

It says 'app works!'.

Does this have to be upgraded to Angular 6 to cause the problem? How do I do that? Is it just a matter of doing an npm install of the latest version?

[ashleydavis]

So I've done a migration of that project to Angular 6 using this guide.

I've reproduced the issue with Data-Forge. Thanks for submitting the code.

[ashleydavis]

So it seems you can't use 'fs' any more in Angular 6.

There's an open issue about it (related to Electron): https://github.com/angular/angular-cli/issues/10681 There's some workarounds suggested in that issue, but I haven't tried them myself.

Other libraries are experiencing this problem, not just Data-Forge.

I'm going to sit on this for a bit. Angular might fix the problem. Otherwise I'm may have to deliver a separate version of Data-Forge without 'fs'.

[yiu31802]

Got it, @ashleydavis. Thanks!

I as a user of both Angular and Data-Forge, I stay with the combination of Angular5 and Data-Forge. Data-Forge simply works perfectly on Angular5.

[ashleydavis]

I may have a solution to this problem for AngularJS v6!

While looking at the code and considering what I was going to do about this issue I spotted a global include of 'fs'. It's possible that this is what is causing your problem!

Please update to using Data-Forge 1.2.6 and let me know if it fixes your problem for Angularjs v6.

[yiu31802]

@ashleydavis

Thanks, I just tried data-forge@1.2.6 but I am guessing your lib is trying to yet fs, according to the error message:

ERROR in ./node_modules/data-forge/build/index.js
Module not found: Error: Can't resolve 'fs' in '/home/CORPUSERS/ss/workspace/xx-individuals/dataforge-6/myapp/node_modules/data-forge/build'
ERROR in ./node_modules/data-forge/build/lib/dataframe.js
Module not found: Error: Can't resolve 'fs' in '/home/CORPUSERS/ss/workspace/xx-individuals/dataforge-6/myapp/node_modules/data-forge/build/lib'
ℹ ｢wdm｣: Failed to compile.
^C

Here's the UNIX command list:

 2060  mkdir dataforge-6
 2061  cd dataforge-6/
 2074  yarn add @angular/cli
 2077  ./node_modules/@angular/cli/bin/ng new myapp
 2079  cd myapp/
 2081  yarn install
 2082  ../node_modules/@angular/cli/bin/ng -v
 2083  ../node_modules/@angular/cli/bin/ng serve
 2084  yarn add data-forge
 2085  nano src/app/app.component.ts # Add data-forge test code by referring to the previous commit
 2086  ../node_modules/@angular/cli/bin/ng serve

The previous commit link: https://github.com/yiu31802/sample-angular5-with-data-forge/blob/bff58198d62937f63960c136a5d5c8e7229da8bb/src/app/components/one-test-view/app.component.ts

[ashleydavis]

Thanks for trying it out.

I'm moving forward with plans to fully extract 'fs' to a separate module.

[ashleydavis]

I've started the process of removing 'fs' from Data-Forge.

This will take some time to complete, but already I have published a 'beta' version of Data-Forge with 'fs' removed.

Can you please try using the beta package of Data-Forge with your project instead of the regular package:

http://npmjs.com/package/data-forge-beta

Please let me know how that goes. If that works for you then I'll know it's worth completing the change.

[yiu31802]

@ashleydavis

Thanks, I tried it out. The situation was changed, but still not working correctly.

• The compilation error message disappeared
• A new error appeared and browser app is crashing due to that, because if I remove the DataFrame variable definition, Angular app works correctly.

  TypeError: globalContext[name] is undefined[Learn More] sugar-core.js:791

  I have no clue about this message at all.

[ashleydavis]

Thanks for trying it.

Do you have updated example code for me to look at? I'd like to get this working!

[newbleach]

Thanks for trying it.

Do you have updated example code for me to look at? I'd like to get this working!

I'm using data-forge-beta with angular6 and Node 8.11.1 sugar-core.js:791 Uncaught TypeError: Cannot read property 'prototype' I keep looking for i a good version that's work with angular6 :)

[ashleydavis]

The 'fs' code has been extracted from Data-Forge, so it should be close to working.

Please submit example code so that I can debug, otherwise let me know what I have to change to get it working.

I don't use Angularjs (sorry React only) so I'm not sure how to reproduce the problem.

[yiu31802]

Hello, @ashleydavis . Do you have any update on this? I just re-tried it with 1.2.7 version but I still met the same symptom as I described above (https://github.com/data-forge/data-forge-ts/issues/3#issuecomment-419795632)

[ashleydavis]

Have you tried the beta version with the 'fs' dependency extracted?

http://npmjs.com/package/data-forge-beta

[ashleydavis]

In a week or two I'll be publishing the beta version.

[yiu31802]

As commented later (https://github.com/data-forge/data-forge-ts/issues/3#issuecomment-420516926), I already tried and failed with 1.3.0 there.

[ashleydavis]

Can you please submit code that demonstrates the problem. Also please submit any error messages you are receiving that might be relevant.

[yiu31802]

You first build a plain Angular app without installing any in your system. It will take approx. 10 minutes to reproduce the error from scratch:

mkdir dataforge-6
cd dataforge-6/
yarn add @angular/cli
./node_modules/@angular/cli/bin/ng new myapp # Hit Enter during the dialogue

Then one more local installation of packages and serve the page

cd myapp/
yarn install
../node_modules/@angular/cli/bin/ng -v
../node_modules/@angular/cli/bin/ng serve

Make sure Angular is up at localhost:4200 Then adding dataforge, and rerun the server. In the browser, you can find error message in the console log.

yarn add data-forge-beta
nano src/app/app.component.ts # See my code below
../node_modules/@angular/cli/bin/ng serve # Page is up but no content to show because of error

Use this code in src/app/app.component.ts

import { DataFrame } from 'data-forge-beta'

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'myapp';

  public ngOnInit(): void {
    let df = new DataFrame({
        columns: {
            'regiment': ['Nighthawks', 'Nighthawks', 'Nighthawks', 'Nighthawks', 'Dragoons', 'Dragoons', 'Dragoons', 'Dragoons', 'Scouts', 'Scouts', 'Scouts', 'Scouts'],
            'company': ['1st', '1st', '2nd', '2nd', '1st', '1st', '2nd', '2nd','1st', '1st', '2nd', '2nd'],
            'TestScore': [4, 24, 31, 2, 3, 4, 24, 31, 2, 3, 2, 3]
        }
    });
    const pivotted = df.pivot(["regiment", "company"], "TestScore", testScores => testScores.average());
    console.log(pivotted.toArray())
  }
}

[ashleydavis]

Please put it in Gist or repo and give me instructions on how to boot it up.

I don't work with AngularJS and I don't have much time to test the problem for you.

[yiu31802]

I understand that you are a busy person, but putting it in Git is not a reasonable choice as you are not using Angular. I am sure this way is much faster than (1) Installation of Angular in your system (2) Git cloning of the test code.

The instruction above is a way to bypass the system installation of Angular, and putting every files in Git is meaningless cause it mostly deals with package management.

[ashleydavis]

I'd just like to be able to npm install and npm start on a GitHub repo and be able to reproduce in 1-2 mins.

[yiu31802]

I do not have clue now other than that unless your system has ng (Angular's) command, but let me think. But I will share the error I am seeing in data-forge.

./node_modules/sugar/string/index.js
index.js:4
__webpack_require__
bootstrap:78
./node_modules/sugar/index.js
index.js:3
__webpack_require__bootstrap:78
./node_modules/data-forge-beta/build/index.js
index.js:13:12

[ashleydavis]

I'm happy to look at this problem. You just need to provide code in a GitHub repo.

I should be able to clone the repo, then type npm install to install required dependencies and then npm start to start the application and have it immediately reproduce the problem - I'm happy to follow other steps as long the code is ready to go and the command line for install and run are straight forward.

Sorry I should have had a guide for people that submit issues to say the requirements and make it clear what I need from you.

[yiu31802]

Please have a try with this:

git clone git@github.com:yiu31802/sample-angular-data-forge.git -b issue-3
npm install
npm start

Sorry that I was too lazy to put things in Git repository.

[ashleydavis]

The git clone doesn't work for me.

I'm getting a 'permission denied' error.

Do you have a link to the repo so that I can check it from the browser?

[yiu31802]

Weird, it is a public repository at GitHub as you see, can you recheck the repository page?

https://github.com/yiu31802/sample-angular-data-forge

[ashleydavis]

Got it thanks.

[ashleydavis]

This seems to be a problem with using the Sugar library that Data-Forge is depending on.

Trying to track it down.

[ashleydavis]

I believe this is the same problem:

https://github.com/andrewplummer/Sugar/issues/632

[ashleydavis]

Here is a workaround for you:

Add this line of code to your polyfills.ts:

window['global'] = window; Source: https://github.com/andrewplummer/Sugar/issues/632#issuecomment-398725011

[ashleydavis]

I'll leave this issue open until the problem has been resolved with Sugar, then I'll update Sugar in Data-Forge and do a new release.

[ashleydavis]

@yiu31802 thanks so much for your help to reproduce this problem.