Import from csv variable?

[zacjordaan]

I have been using alasql very successfully thus far and in a wide range of situations. However, I've now come across a case that I'm not sure is possible.

Even though I have been able to easily import from local csv files and perform queries on them, I now need to use zip files (containing csv files) as my source. So, using jszip I am able to unzip a local file into the browser's memory. Now that I have a csv file in a variable, might it be possible to query it from alasql? A normal select simply slices the string into an array of individual characters rather than parsing it as a csv.

If my source files (before zipping) were in json format I think it could be done but at the moment I don't see any way to import from a csv string variable rather than a file. Unless you know of a trick maybe?

Many thanks Rgds Zac

Ps: What an amazing library! A job expertly well done and sincerely appreciated.

[zacjordaan]

Hmmm. It looks as if I am able to simply append a function to alasql.from and replace the fileName parameter with one that can accept my csv text variable, and then just comment out the alasql.utils.loadFile logic; something like this fragment:

alasql.from.CSVV = function(text, opts, cb, idx, query)...

I then do this:

alasql.promise("select * from CSVV(?,{headers:true})", [ my_csv_string_variable ])...

...and, incredibly, it seems to work! However, might it be ok to do it this way? Your advice most appreciated, thank you.

Rgds Zac

[agershun]

Zac, hi. Thank you for the your words about AlaSQL project.

For binary files (like XLS) I think the point of solution is somewhere here:

        } else if(path instanceof Blob) {
            success(path);
        }

AlaSQL test the type of first argument of XLS() function. So. if you pass Blob as very first argument, it should work....

For CSV... The situation is a little bit different, because AlaSQL uses another loading function for text files (loadFile() instead loadBinaryFile()).

I think we can add the flag asstring: true to the standard CSV() function to replace loadFile() logic. E.g.:

alasql('SELECT * FROM CSV(?,{asstring:true})',['10,20,30\n20,30,40']);

What do you think what is the best name for this parameter?

[zacjordaan]

Hi Andrey

Thanks for your super-quick and helpful response, which seems to be standard behaviour for you on this project; I suspect you're a machine!

Yes, the way you propose is obviously the most correct - extending the existing function rather than appending a new one. Of course I only did that so I could separate my addendum from the alasql core code, and also only as a test. Anyway, you now raise the eternal question for all developers: _what to name the thing?_

As somebody once said, "There are only two difficult things in Computer Science: cache invalidation, naming things, and off-by-one errors." ;-)

Personally, I tend to name function variables either according to what they are or according to their function or use, and always as briefly/short as possible. This is only a personal style and if you search for "javascript style guides" you'll easily find some from many of the legendary programmers/vendors out there. The problem is that many of them disagree with one another so in the end you will find yourself back to your personal preference. Which is just fine of course.

My naming approach would be:

alasql('SELECT * FROM CSV(?, {headers: true, fromString: true})', ['A,B,C,\n10,20,30\n20,30,40']);

Or way in the future, from: '[file | string | arrayBuffer]'

Tough one. I think just go with your gut. And in the meantime, how about a short break with some entertaining reading (and actually some interesting points) in a slide-show about naming things: How to name things: the hardest problem in programming (I promise it gets good a few slides in...)

[mathiasrw]

from: '[file | string | arrayBuffer]'

Feels good to me too. Its consistent.

a slide-show about naming things: How to name things: the hardest problem in programming

Nice - thanks...

[mathiasrw]

♥

a slide-show about naming things: How to name things: the hardest problem in programming