search

scamden / jquery-csv

Automatically exported from code.google.com/p/jquery-csv
MIT License
0 stars 0 forks source link

using gviz http://jsfiddle.net/asgallant/edT7T/7/ error #15

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1.use above fiddle
2.import this csv file 
"TimeStamp (ms)","MID 128|PID 91|Accelerator pedal position (%)","MID 128|PID 
92|Engine load(%)","MID 128|PID 100|Engine oil pressure (bar)","MID 128|PID 
105|Intake manifold temperature (N0C)","MID 128|PID 110|Engine coolant 
temperature (N0C)","MID 128|PID 190|Engine speed (r/min)"
115,10.4,0,0,0,0,0
175,,40,,,,
309,,,4.55,,,
529,,,,17,,
630,,,,,46,
682,,,,,,1011.25
751,10.4,,,,,
803,,40,,,,
856,,,4.55,,,
921,,,,17,,
1013,,,,,46,
1521,,,,,,1016.5
1593,10.4,,,,,
1641,,39,,,,
1704,,,4.58,,,
1761,,,,17,,
1832,,,,,46,
1897,,,,,,1023
1961,10.4,,,,,
2025,,37,,,,
2088,,,4.58,,,
2145,,,,17,,
2217,,,,,46,
2283,,,,,,1022.25
2344,10.4,,,,,
2408,,36,,,,
2464,,,4.58,,,
2576,,,,17,,
2633,,,,,46,
2698,,,,,,1001
2767,10.4,,,,,

error thrown from google 'Uncaught Error: Type mismatch. Value  does not match 
type number in column index 6'

Works with another csv file of similar construction. 

Any ideas?

Thanks for this great script.

Original issue reported on code.google.com by tribbleh...@gmail.com on 24 Oct 2012 at 7:31

GoogleCodeExporter commented 9 years ago 
For some reason the link to the fiddle didn't show up.

One question, what browser are you using when it fails. 

Either way, I think this may follow the same lines as bug 14. A new entry 
parser is in the works so it may take a little time before I can get it fully 
completed and tested. I'll post an update once the new code is available.

Original comment by evanpla...@gmail.com on 24 Oct 2012 at 8:48

GoogleCodeExporter commented 9 years ago 
OK, so but 14 is fixed and a completely new parser has been rolled out. 
Unfortunately, it still doesn't properly parse your dataset.

It's the null values that aren't being handled properly.

This subset of data will work:

"TimeStamp (ms)","MID 128|PID 91|Accelerator pedal position (%)","MID 128|PID 
92|Engine load(%)","MID 128|PID 100|Engine oil pressure (bar)","MID 128|PID 
105|Intake manifold temperature (N0C)","MID 128|PID 110|Engine coolant 
temperature (N0C)","MID 128|PID 190|Engine speed (r/min)"
115,10.4,0,0,0,0,0

With my spiffy new parser, it shouldn't be too hard to fix.

Original comment by evanpla...@gmail.com on 25 Oct 2012 at 3:34

GoogleCodeExporter commented 9 years ago 
Got it...

I have to commend you for providing a dataset that covers nearly every type of 
null value use case scenario.

According to the specification, trailing commas aren't legal but that's a dumb 
rule considering that unquoted null values are legal. My guess would be that 
the people who wrote the RFC didn't take the time to consider how null values 
would be used under realistic circumstances. Especially, since Excel (and most 
other spreadsheet apps that can output CSV) don't quote all values by default.

Anyway, enough blabbering...

Feel free to try it out in the 'Basic Usage Demo':
http://jquery-csv.googlecode.com/git/examples/basic-usage.html

The code won't be officially available until 0.65 drops; if you want it now, 
feel free to grab the source directly from the repository browser.
http://code.google.com/p/jquery-csv/source/browse/#git%2Fsrc

One last thing, the lineSplitter method that properly handles new-lines in 
values is now the default; so you won't need to set experimental:true anymore.

Original comment by evanpla...@gmail.com on 25 Oct 2012 at 4:26

GoogleCodeExporter commented 9 years ago

Original comment by evanpla...@gmail.com on 27 Oct 2012 at 5:40

GoogleCodeExporter commented 9 years ago 
Hi again. I might have another case scenario for you. lol
Replied the other day by email but i think it didn't get to you.

I have attached a 'raw file' that i am trying to parse (it was the original i 
pasted the data out of) which does not seem to work. It drops off the last 
value from each row if empty. 
This is the sort of file I am trying to turn into a gviz on a regular basis.

Original comment by tribbleh...@gmail.com on 27 Oct 2012 at 4:41

Attachments:

GoogleCodeExporter commented 9 years ago 
Let me finish rolling the custom delimiter/separator code in the new parser.

Technically, with the new parser it shouldn't encounter any issues with null 
last values. I actually test for that specific use case now.

You can see it under tests, RFC Compliance, Amendment 3:
http://jquery-csv.googlecode.com/git/test/test.html

Original comment by evanpla...@gmail.com on 28 Oct 2012 at 1:10

GoogleCodeExporter commented 9 years ago 
That's strange, I can reproduce the issue even though the parser should cover 
that issue already.

I'll keep at it until I can get it to work.

Original comment by evanpla...@gmail.com on 1 Nov 2012 at 10:41

GoogleCodeExporter commented 9 years ago 
OK, so after a whole lot of head scratching I think I finally found it.

It's actually comprised of 4 independent bugs.

The first two were related to null-last-values. The tests were passing because 
the null last value fix isn't quite right. I was running the null-last-value 
test using toArrays instead of toArray; the combination of two bugs combined 
was causing the test to pass even though it wasn't coded correctly.

If I had tried it on multi-line data, the issue would have shown itself by 
having every entry with a trailing null be short one value and the last one 
having an extra null value added on.

My test samples are pretty extensive but apparently not enough.

The third bug, and the reason why the data was only being parsed to two cells 
wide was being caused by the castToScalar function. For parser hooks, if you 
return false it will skip that cell; this is useful for evaluation behavior 
like extracting specific rows/cells. I forgot that Javascript mimics C 
including some of the dumber practices like the fact that false == 0 and true 
== 1. Not to difficult to fix once I realized the problem.

The last bug had to do with the way I was evaluating null values in 
castToScalar. Technically, null can equal either '', 0, or false. For some 
reason, isNaN interprets them as being a number. To fix it, I had to check the 
return value from parseInt() for (ironically) NaN values.

It turns out I was right when I said that your dataset would break the null 
parsing in every conceivable way possible.

Anyway, I'm still working out fixes for the first two bugs. While the FSM is a 
lot easier to fine tune, it's very difficult to maintain a good mental model.

As soon as I have everything working I'll post a message here.

Original comment by evanpla...@gmail.com on 2 Nov 2012 at 3:45

GoogleCodeExporter commented 9 years ago 
It's fixed in the repo source.

I'd push another pre-release but to my complete surprise Google's back-end is 
experiencing an issue preventing me from pushing to the public repo. 
Apparently, it's a known issue and I just have to wait for them to apply the 
fix.

No custom separator/delimiter support yet so the 'Sir Henry SDB.slk) data set 
still won't work. That's next on my todo list. As soon as it's done, the 
release number will jump to 7.0 (finally).

In the mean time, feel free to grab the latest via git or through the online 
repo browser.

One question, do you have any reservations against me using the data set you 
provided in this bug as a sample? That's one fancy looking data set, I wish I 
knew of a place to get more like it.

I managed to tweak your vis code into a fully functional demo. There's a minor 
bug that is screwing up the UI (and I can't push the fix yet). Once it's fixed 
I suggest you check it out.

Original comment by evanpla...@gmail.com on 3 Nov 2012 at 2:39

GoogleCodeExporter commented 9 years ago 
I am glad I could help with your development.
I will check the data I sent you just to make sure there are no issues with
data protection. Serial numbers etc.
The file is from a Volvo penta marine engine log test. My plan is to have
the chart web app sit on my web site where engineers can drop in files to
create viz representations for easier analysis. Will send send you the link
once I upload the page.

Original comment by tribbleh...@gmail.com on 3 Nov 2012 at 6:17

GoogleCodeExporter commented 9 years ago 
Gviz chart now set up.

http://www.rrandi.co.uk/testing.php

will be modifying in due time but for now it works.

Original comment by tribbleh...@gmail.com on 3 Nov 2012 at 8:52

GoogleCodeExporter commented 9 years ago 
Nice. Works perfectly.

I ended up dropping the data table and file info in the demo but it is 
essentially the same. I'm still waiting for Google to fix their issue so I 
can't push to the public repo yet but apparently my last push finally went 
through.

Here's the link:
http://code.google.com/p/jquery-csv/source/browse/examples/google-visualization.
html

Original comment by evanpla...@gmail.com on 4 Nov 2012 at 4:07

GoogleCodeExporter commented 9 years ago 
Looks good.
Only problem with the data file conversion of the particular files i am
working with is the null values are all converted to zeros.
is there any way for them to stay at null?

The other files i use work brilliantly. A most excellent plug in.

Brian

Original comment by tribbleh...@gmail.com on 4 Nov 2012 at 1:46

GoogleCodeExporter commented 9 years ago 
I wasn't quite sure about that particular detail. In retrospect, I guess 
setting null makes more sense.

Still no push access here but you can make the change to your own source. 

Change:
if(isNaN(integer)) {
  return 0;

To:
if(isNaN(integer)) {
  return '';

Original comment by evanpla...@gmail.com on 4 Nov 2012 at 11:06

GoogleCodeExporter commented 9 years ago 
I remember now, nulls aren't considered valid data in Google Vis (and most 
likely flot too).

It's kind of a gray area because casting a null '' value to a scalar in 
Javascript should technically return NaN.

Your thoughts?

Original comment by evanpla...@gmail.com on 5 Nov 2012 at 4:54

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Have just looked at the gviz playground. It appears to accept the value 'null' 
and '' in a data table but not as the last value of a row. it only accepts 
'null' as the last value. Tried out on this page 
http://code.google.com/apis/ajax/playground/?type=visualization#annotated_time_l
ine 

don't know if that will help in my case.

Original comment by tribbleh...@gmail.com on 6 Nov 2012 at 6:37

GoogleCodeExporter commented 9 years ago 
ah my error. only accepts 'null' as null value.

Original comment by tribbleh...@gmail.com on 6 Nov 2012 at 6:42

GoogleCodeExporter commented 9 years ago 
More thoughts......
I am going to modify the code used on my web page to replace 0 with 'null'.
Change:
if(isNaN(integer)) {
  return 0;

To:
if(isNaN(integer)) {
  return 'null';

and let you know if it works.

Brian

Original comment by tribbleh...@gmail.com on 7 Nov 2012 at 7:30

GoogleCodeExporter commented 9 years ago 
Well, technically 'null' represents a string not a null value. Try it without 
the quotes to see if that works.

The whole implicit type system used by Javascript can be a pain sometimes when 
it isn't obvious what type should be used.

Original comment by evanpla...@gmail.com on 7 Nov 2012 at 12:17

GoogleCodeExporter commented 9 years ago 
Spot on. Null seems to work.
Have to try on a few of the files before I open the beer! Lol.

Original comment by tribbleh...@gmail.com on 7 Nov 2012 at 12:20

GoogleCodeExporter commented 9 years ago 
Tested now with all files i can find that i hope to use. replacing the following

if(isNaN(integer)) {
  return 0;

with

if(isNaN(integer)) {
  return null;

works! now null entries are correctly recognised by gviz. 
will this be an addition that you will add to the code or am i a 'special case' 
and need to host the script myself? If I need to host it myself then i had 
better swap from Yola hosting as they wont let me upload js files unless i 
upgrade......

Original comment by tribbleh...@gmail.com on 7 Nov 2012 at 5:19

GoogleCodeExporter commented 9 years ago 
Duh... I probably should have taken a look at the documentation first.

"Each cell in the table holds a value. Cells can have a null value, or a value 
of the type specified by its column."

_Source: https://developers.google.com/chart/interactive/docs/reference_

It makes sense that type null represents a non-value. That is actually the 
primary purpose for its existence because 0 is considered a value. Anyway, the 
change has been incorporated into the latest commit of the public repo.

If you don't mind the risk that future changes could possibly break the code 
temporarily, you could always link directly to the repo. That's basically what 
I do in all of the example code.

Here's the link:
http://jquery-csv.googlecode.com/git/src/jquery.csv.js

This project is still somewhat unstable and prone to change (ie beta). When I 
reach the 1.0 keystone I plan to lock-in the spec and provide a hosted 
long-term release. Who knows, maybe one of the main jQuery CDNs will host a 
copy so I won't have to.

Original comment by evanpla...@gmail.com on 7 Nov 2012 at 11:16

GoogleCodeExporter commented 9 years ago 
Many thanks for the update. I understand regarding the beta state. Will
leave it linked for now. Hope you don't mind but I have borrowed your demo
code and modified for my site and given you and asgallant credit. Will be
modifying to accommodate other functions as needs arise. And where ever
possible making others aware of your work and wonderful code.

Original comment by tribbleh...@gmail.com on 8 Nov 2012 at 6:45

GoogleCodeExporter commented 9 years ago 
Lol, no need to give me attribution in the demo code. If you didn't notice the 
code you posted was based largely on what you originally posted. That saved a 
lot of time that would have otherwise been spent trying to figure out the 
Google Viz API.

All I did was clean up the styling to be more consistent with the rest of my 
demos and wrapped it in a jQuery UI to make it look nice.

I appreciate the feedback, not only did you help identify some issues with the 
core parser but also helped to refine the castToScalar functionality in a way 
that will be more robust for re-use in the future.

Your feedback is really appreciated. It's users like you that help ensure that 
projects like this can achieve a high level of quality.

Original comment by evanpla...@gmail.com on 8 Nov 2012 at 2:35

GoogleCodeExporter commented 9 years ago 
Evening. Just catching up with projects I had to shelve and am now picking
back up.

Nice to see that this is still going. Have been using it quite a bit and am
now redesigning it to enable others at work to use it. got to be fool proof
and not complicated. lol.

Will keep you appraised.

Original comment by tribbleh...@gmail.com on 23 Sep 2013 at 8:51