search

AccelerationNet / cl-csv

A common lisp library providing easy csv reading and writing
Other
116 stars 22 forks source link

Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv #28

Closed pjcollins21 closed 6 years ago

pjcollins21 commented 6 years ago

Having looked at the readme and the documentation, reddit, stackoverflow, and the internet-at-large, I still can't wrap my head around how to read a single specified row from CSV. The following gets me row 1:

(cl-csv:read-csv-row #P[filename])

I can't figure out how to pass the desired row number into this function given the documentation. I'm sure it's a logic error on my part. A quick bit of sample code added to the readme would be fantastic. Thanks!

bobbysmith007 commented 6 years ago

I don't think there is a function call to do that directly.

I would probably write something like:

(iter (for row in-csv #P"path") 
      (for i from 0) 
      (when (= i N)
        (return row)))

Or if you have a distaste for iterate:

(cl-csv::with-csv-input-stream (str #P"path") 
     (let ( row )
       (dotimes (i N row) 
          (setf row (cl-csv:read-csv-row str))))

This doesn't seem like a particularly common use case (cursoring through a csv to a specific row), why are you doing that? I dont mind adding a function to do that if there is a compelling reason, but its never been needed up till now, so I was curious.

bobbysmith007 commented 6 years ago

Didnt mean to close it yet, sorry

pjcollins21 commented 6 years ago

Hi Russ,

Thanks so much for your help. Will start testing your suggestions asap.

This is for a common lisp neural net using the MNIST dataset, and I'm using a CSV of pixel data values instead of the images. So for each run of the NN, I want to pull the next row of data as a list and run it through the network.

I hope this makes sense. I'm a lisp novice, and fairly clueless. If you need any follow up information, let me know, but I suspect one of your solutions will do the trick.

Thanks again!

Kind regards,

Patrick Collins

4th Year Computing in Games Development Student - Dundalk Institute of Technology

From: Russ Tyndall notifications@github.com Sent: Monday, April 16, 2018 9:25:23 PM To: AccelerationNet/cl-csv Cc: Patrick Collins; Author Subject: Re: [AccelerationNet/cl-csv] Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv (#28)

I don't think there is a function call to do that directly.

I would probably write something like:

(iter (for row in-csv #P"path") (for i from 0) (when (= i N) (return row)))

Or if you have a distaste for iterate:

(cl-csv::with-csv-input-stream (str #P"path") (let ( row ) (dotimes (i N row) (setf row (cl-csv:read-csv-row str))))

This doesn't seem like a particularly common use case (cursoring through a csv to a specific row), why are you doing that? I dont mind adding a function to do that if there is a compelling reason, but its never been needed up till now, so I was curious.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/AccelerationNet/cl-csv/issues/28#issuecomment-381737485, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKcKkE-w9RNeXpiCSrNPU1i1ADxUQlpCks5tpP4zgaJpZM4TXKYd.

pjcollins21 commented 6 years ago

The second option should work, but I get a resolvable issue with "You are in the top-level Read-Eval-Print loop."

Again, I'm sure this is a noob mistake, but is there a flag or parameter I need to use to avoid this? Screenshot attached.

Thanks a million.

Regards,

Patrick Collins[cid:cc07a07a-79f1-4562-bf7b-c4abb971fdc6]

From: Patrick Collins Sent: Monday, April 16, 2018 9:32:32 PM To: AccelerationNet/cl-csv Subject: Re: [AccelerationNet/cl-csv] Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv (#28)

Hi Russ,

Thanks so much for your help. Will start testing your suggestions asap.

This is for a common lisp neural net using the MNIST dataset, and I'm using a CSV of pixel data values instead of the images. So for each run of the NN, I want to pull the next row of data as a list and run it through the network.

I hope this makes sense. I'm a lisp novice, and fairly clueless. If you need any follow up information, let me know, but I suspect one of your solutions will do the trick.

Thanks again!

Kind regards,

Patrick Collins

4th Year Computing in Games Development Student - Dundalk Institute of Technology

From: Russ Tyndall notifications@github.com Sent: Monday, April 16, 2018 9:25:23 PM To: AccelerationNet/cl-csv Cc: Patrick Collins; Author Subject: Re: [AccelerationNet/cl-csv] Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv (#28)

I don't think there is a function call to do that directly.

I would probably write something like:

(iter (for row in-csv #P"path") (for i from 0) (when (= i N) (return row)))

Or if you have a distaste for iterate:

(cl-csv::with-csv-input-stream (str #P"path") (let ( row ) (dotimes (i N row) (setf row (cl-csv:read-csv-row str))))

This doesn't seem like a particularly common use case (cursoring through a csv to a specific row), why are you doing that? I dont mind adding a function to do that if there is a compelling reason, but its never been needed up till now, so I was curious.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/AccelerationNet/cl-csv/issues/28#issuecomment-381737485, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKcKkE-w9RNeXpiCSrNPU1i1ADxUQlpCks5tpP4zgaJpZM4TXKYd.

bobbysmith007 commented 6 years ago

If you just need to run a function across each row of a CSV there are a couple ways to do that directly:

(cl-csv:read-csv #P"Path" :row-fn #'do-neural-network-stuff)

Or:

(iter (for row in-csv #P"path") 
      (do-neural-net-stuff row))

Both of those should handle only keeping one row in memory at a time (allowing you to cursor across arbitrarily large files).

I don't know the specifics of what you are doing, but I feel like you are going to be using a bunch more time and space by doing this as CSV strings representing pixel data vs just manipulating the binary directly (common lisp has good facilities for dealing with binary and bit vectors). Good luck either way though.

bobbysmith007 commented 6 years ago

The second option should work, but I get a resolvable issue with "You are in the top-level Read-Eval-Print loop." Again, I'm sure this is a noob mistake, but is there a flag or parameter I need to use to avoid this? Screenshot attached.

Which implementation are you using? That sounds like there is an issue with the conditions (it sounds like a restart is being invoked that is not bound)

pjcollins21 commented 6 years ago

I'm using GNU clisp on windows 10. Testing on a tiny subsection of the actual training data which is 785 cells by 60000 rows, using 15 row csv, this code results in the messaging pictured, but it does print the correct row to console on entering :h. If there's a way to silence this messaging, that would be really helpful. If it's not in your scope, I totally get it. You've given me a huge boost with this project as it is. I've been beating my head against this for 2 days, so I really appreciate the help.

From: Russ Tyndall notifications@github.com Sent: Monday, April 16, 2018 10:20:50 PM To: AccelerationNet/cl-csv Cc: Patrick Collins; Author Subject: Re: [AccelerationNet/cl-csv] Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv (#28)

The second option should work, but I get a resolvable issue with "You are in the top-level Read-Eval-Print loop." Again, I'm sure this is a noob mistake, but is there a flag or parameter I need to use to avoid this? Screenshot attached.

Which implementation are you using? That sounds like there is an issue with the conditions (it sounds like a restart is being invoked that is not bound)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/AccelerationNet/cl-csv/issues/28#issuecomment-381753669, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKcKkGJV46UISqhdICfm48btx0ZkSocNks5tpQsygaJpZM4TXKYd.

bobbysmith007 commented 6 years ago

Sorry, I dont have windows or clisp, so I am unlikely to be able to help much there (I didnt get a screen shot if you tried to send one). If you do attach one on the ticket at github, or copy / paste the stacktrace / console, I might be able to spot something.

pjcollins21 commented 6 years ago

Here's what I'm seeing in CLISP when I run this code: (cl-csv::with-csv-input-stream (str #P"test3.csv") (let ( row ) (dotimes (i 5 row) (setf row (cl-csv:read-csv-row str)))) )

clisp-issue

bobbysmith007 commented 6 years ago

I installed CLISP, and that message only shows up for me if I type :h into the shell. It doesnt seem like a side effect of cl-csv, but rather of your usage of CLISP (ie: you seem to have included :h into the forms)

bobbysmith007 commented 6 years ago

I'm leaving for the day, but will check back tomorrow. Good luck 👍

pjcollins21 commented 6 years ago

Thanks again so much for your help!

bobbysmith007 commented 6 years ago

Did you get it figured out?

pjcollins21 commented 6 years ago

Yes, I used a version of this:

(cl-csv::with-csv-input-stream (str #P"path") (let ( row ) (dotimes (i N row) (setf row (cl-csv:read-csv-row str))))

to finish and submit my project. This issue is resolved. Thanks so much for your help!

Kind regards,

Patrick Collins

From: Russ Tyndall notifications@github.com Sent: Tuesday, April 24, 2018 4:41:02 PM To: AccelerationNet/cl-csv Cc: Patrick Collins; Author Subject: Re: [AccelerationNet/cl-csv] Feature Request: ReadMe Sample Code Addition: How to read in a single specified row from csv (#28)

Did you get it figured out?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/AccelerationNet/cl-csv/issues/28#issuecomment-383979774, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKcKkGNL84aYE5BUxqtN2ZY53v3QJ33lks5tr0eOgaJpZM4TXKYd.