alberthaff / ngx-papaparse

Papa Parse wrapper for Angular
https://alberthaff.dk/projects/ngx-papaparse/docs/v8
MIT License
90 stars 19 forks source link

Start the header at a specific row number. #71

Closed gitalvininfo closed 3 years ago

gitalvininfo commented 3 years ago

Describe the bug

A clear and concise description of what the bug or problem is. The documentation tells that if you set the option "header" to true, the first row will be interpreted as field names, however in my case, the header of CSV file starts at row number 3.

To Reproduce

Please provide the necessary steps to reproduce the unwanted behaviour. This is the screen shot for my CSV file (My personal trade history) Screenshot_2020-10-13_14-48-34

Expected behavior

A clear and concise description of what you expected to happen. So, generally I want to start the header at line 3 or maybe just ignore the first two lines. As I read the documentation there is an option called "comments" which is of type string, which skips the line whenever it encounters that comment (e.g "#" or "//") but these does not solve my issue.

Software and platform

Additional context

Thank you for providing this library which enable us to parse csv files easily.

alberthaff commented 3 years ago

Hi. I'm sorry - unfortunately the underlying library does not seem to support this. However, you could simply skip / unset the first couple of rows (depending on how you work with the parsed data).

I actually had a similar issue with some bank statements, since the banks sometimes add a few lines with metadata, before the transactions. I solved it by skipping the first X number of items in the outputted array, when working with the data.

gitalvininfo commented 3 years ago

Hi @alberthaff.

Thanks for your response. One last thing. Can you spare a minute to point me out where should I skip or slice the first two rows. This is my code for parsing csv.

 this.papa.parse(csv, {
        skipEmptyLines: true,
        header: true,
        complete: (results )=> {
          results.data.splice(0, 1);
          for (let i = 0; i < results.data.length; i++) {
            let tradehistory = {
              Date: results.data[i].Date,
              Side: results.data[i].Side,
              ConfNumber: results.data[i].ConfNumber,
              Stock: results.data[i].Stock,
              Shares: results.data[i].Shares,
              PhpShare: results.data[i].PhpShare,
              Fees: results.data[i].Fees,
              Total: results.data[i].Total,
            };
            this.loading = false;
            this.test.push(tradehistory);
          }
          console.log("Parsed: k", results.data);
        }
      });

As you can see, I am splicing the index 1 in results array which I think will proceed to the 3rd row. Then I am creating a new array which will contain my desired result. However, this does not solve my issue and I am also aware that setting the header to true will create an objects of row, setting to false will create an array for each row. I am actually confused where did you put the code that skips the row.

This is the result of my code.

Capture

The object keys are still the first row.

Any help would be appreaciated, Thank you so much for helping.

alberthaff commented 3 years ago

If you want to use header: true, you must remove the first couple of lines before running it through papaparse - here are some suggestions.

Otherwise, you'll have to use header: false and map the columns yourself. You can do it like this:

    this.papa.parse(csv, {
        skipEmptyLines: true,
        header: true,
        complete: (results )=> {
          for (let i = 2; i < results.data.length; i++) { // Notice that i changed i to 2, so that we skip the line 0 and 1.
            let tradehistory = {
              Date: results.data[i][0],
              Side: results.data[i][1],
              (...)
            };
            this.loading = false;
            this.test.push(tradehistory);
          }
          console.log("Parsed: k", results.data);
        }
    });
gitalvininfo commented 3 years ago

Thanks for the patience. Exactly what I was thinking, I cannot do it when header is true. Thanks again. 👍

alberthaff commented 3 years ago

You're welcome