search

AccelerationNet / cl-csv

A common lisp library providing easy csv reading and writing
Other
116 stars 22 forks source link

Replace read-dispatch table-entry defclass with defstruct. #33

Closed sirherrbatka closed 9 months ago

sirherrbatka commented 5 years ago

The following changes greatly improve the performance of cl-csv reader.

bobbysmith007 commented 9 months ago

This seems like a quality fix, but I do not have Common Lisp accessible anymore and this branch has conflicts. If someone wants to update this PR, to latest and run the test suite, I would be happy to merge. This project needs a new maintainer as well

sirherrbatka commented 9 months ago

I simply rolled out my own parser, with much better performance.

richardwesthaver commented 9 months ago

just tried running the test suite but ran into a few errors. is this type declaration correct? https://github.com/AccelerationNet/cl-csv/pull/33/files#diff-6a8e96795371c2729b3bf343384d7518de2b63ef93ec0079b71962376292970cR247 - test 'parser-1' fails because of it, but easy fix. after that a bunch of tests passed but got the next error in test 'different-newlines':

Starting: CL-CSV-TEST::early-end-of-stream
CL-CSV-TEST::early-end-of-stream - passed (0.00s) : 3 assertions passed

Starting: CL-CSV-TEST::read-into-buffer-until-test
CL-CSV-TEST::read-into-buffer-until-test - passed (0.02s) : 14 assertions passed

Starting: CL-CSV-TEST::buffer-spanning-new-lines
CL-CSV-TEST::buffer-spanning-new-lines - passed (0.00s) : 3 assertions passed

Starting: CL-CSV-TEST::buffer-spanning-new-lines2
CL-CSV-TEST::buffer-spanning-new-lines2 - passed (0.00s) : 4 assertions passed

Starting: CL-CSV-TEST::different-newlines
[...] ERROR:
The value
  "|"
is not of type
  simple-vector
when setting slot cl-csv::delimiter of structure cl-csv::read-dispatch-table-entry
   [Condition of type type-error]

Backtrace:
  0: (cl-csv::make-read-dispatch-table-entry :delimiter "|" :didx -1 :dlen 1 :dlen-1 0 :dispatch #<function cl-csv::reading-separator>)
  1: (cl-csv::make-table-entry "|" #<function cl-csv::reading-separator>)
  2: (cl-csv::make-default-csv-reader)
  3: (cl-csv::read-csv-with-reader #<sb-impl::string-input-stream {100AFA2A83}> :csv-reader nil :row-fn nil :map-fn nil :data-map-fn nil :skip-first-p nil)
  4: (cl-csv:read-csv #<sb-impl::string-input-stream {100AFA2A83}> :newline "**" :separator "|")
  5: (cl-csv-test::different-newlines-test-thunk)
  6: (lisp-unit2::record-result-context #<function cl-csv-test::different-newlines-test-thunk>)
  7: (lisp-unit2::%run-test #<lisp-unit2:unit-test CL-CSV-TEST::different-newlines {1009CAD253}> :test-contexts (#<function lisp-unit2::record-result-context> nil))
  8: (lisp-unit2::%run-test-name cl-csv-test::different-newlines :test-contexts (#<function lisp-unit2::record-result-context> nil))
  9: (cl-csv-test::different-newlines :test-contexts (#<function lisp-unit2::record-result-context> nil))

there's an ASDF error that occurs when trying (asdf:test-system :cl-csv) (not related to this patch just a heads up for future maintainer) I just ran the tests manually instead on SBCL.

sirherrbatka commented 9 months ago

I can't remember, i wrote this code back in 2019.