Why there is no implimentaion for exclude_headers=true

[atom217]

I am having issue with mismatch of length of string and header. Just wondering why exclude_headers=true is not implemented.

For example : https://www.quandl.com/api/v1/datasets/WIKI/AAPL.json?column=4&collapse=quarterly&transformation=normalize Is implemented

But adding another param exclude_headers=true is not possible with current api ? https://www.quandl.com/api/v1/datasets/WIKI/AAPL.csv?column=4&collapse=quarterly&transformation=normalize&exclude_headers=true

[jimmoores]

It's really because I didn't see a use for it - the normal reason for it is when you're calling the REST API raw, you don't have to parse the header row, but as all the tabular data is parsed by the library I didn't see how that added anything - it's parsed for you and you can just iterate over the data if you're not interested in it. Can you elaborate on your issue a bit more?

[atom217]

Hi , I am getting following error :

11:37:51.678 [pool-1-thread-29] ERROR com.jimmoores.quandl.Row - Attempt to create a Row with a header definition containing 6 columns and a values array containing 8 values Exception in thread "pool-1-thread-29" com.jimmoores.quandl.util.QuandlRuntimeException: headerDefinition and values array are of differing length at com.jimmoores.quandl.Row.<init>(Row.java:27) at com.jimmoores.quandl.Row.of(Row.java:45) at com.jimmoores.quandl.util.DefaultRESTDataProvider.getTabularResponse(DefaultRESTDataProvider.java:101) at com.jimmoores.quandl.QuandlSession.getDataSet(QuandlSession.java:159) at com.atom217.thread.QueryTask.run(QueryTask.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Exception in thread "pool-1-thread-20" java.lang.NullPointerException at java.util.Collections$ReverseComparator.compare(Unknown Source) at java.util.Collections$ReverseComparator.compare(Unknown Source) at java.util.TimSort.countRunAndMakeAscending(Unknown Source) at java.util.TimSort.sort(Unknown Source) at java.util.TimSort.sort(Unknown Source) at java.util.Arrays.sort(Unknown Source) at java.util.Collections.sort(Unknown Source) at com.atom217.thread.QueryTask.run(QueryTask.java:60) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Exception in thread "pool-1-thread-47" java.lang.NullPointerException at java.util.Collections$ReverseComparator.compare(Unknown Source) at java.util.Collections$ReverseComparator.compare(Unknown Source) at java.util.TimSort.countRunAndMakeAscending(Unknown Source) at java.util.TimSort.sort(Unknown Source) at java.util.TimSort.sort(Unknown Source) at java.util.Arrays.sort(Unknown Source) at java.util.Collections.sort(Unknown Source) at com.atom217.thread.QueryTask.run(QueryTask.java:60) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 11:37:52.647 [pool-1-thread-45] ERROR com.jimmoores.quandl.Row - Attempt to create a Row with a header definition containing 6 columns and a values array containing 8 values Exception in thread "pool-1-thread-45" com.jimmoores.quandl.util.QuandlRuntimeException: headerDefinition and values array are of differing length at com.jimmoores.quandl.Row.<init>(Row.java:27) at com.jimmoores.quandl.Row.of(Row.java:45) at com.jimmoores.quandl.util.DefaultRESTDataProvider.getTabularResponse(DefaultRESTDataProvider.java:101) at com.jimmoores.quandl.QuandlSession.getDataSet(QuandlSession.java:159) at com.atom217.thread.QueryTask.run(QueryTask.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 11:37:56.755 [pool-1-thread-77] ERROR com.jimmoores.quandl.Row - Attempt to create a Row with a header definition containing 6 columns and a values array containing 8 values Exception in thread "pool-1-thread-77" com.jimmoores.quandl.util.QuandlRuntimeException: headerDefinition and values array are of differing length at com.jimmoores.quandl.Row.<init>(Row.java:27) at com.jimmoores.quandl.Row.of(Row.java:45) at com.jimmoores.quandl.util.DefaultRESTDataProvider.getTabularResponse(DefaultRESTDataProvider.java:101) at com.jimmoores.quandl.QuandlSession.getDataSet(QuandlSession.java:159) at com.atom217.thread.QueryTask.run(QueryTask.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

I don't have control over the data that web service is giving to me. Instead what I can verify is that although the header values are wrong but underlying row values are correct (in my case)

The main issue with the code base is that , there is too much dependency on the header value .

Can you suggest any workaround ?

[jimmoores]

Ok, I'll see what I can come up with to work around the issue. I will probably add something to the SessionOptions to allow an unchecked mode for creating Row objects.

[jimmoores]

I've pushed a change to the master branch, if you clone and build it locally and then run mvn install, it should install a local copy of the library with version number 1.4.0-SNAPSHOT. If you can try and run against that and let me know if it fixes your problem that would be helpful. If it does, i'll do a full 1.4.0 release.

I tried a whole bunch of things, but decided the behaviour now should be that if any rows contain more data than the header, then they are padded out with nulls, and the header is expanded with names like 'Column 12', 'Column 13', etc. It will then go back over the existing rows and expand them out and replace the headers in them with the expanded header before the whole lot is used to create the TabularResult object.

[jimmoores]

Oh, by the way, you shouldn't need to modify your code with any changes to SessionOptions or anything.

[atom217]

Hi , I must say you know your code better then I do :dart:

I was trying to create an additional class without headers. But your way of thinking is quite out of the box.

I checked out code on my PC and all examples are running well.

BUT playing with . jar file is giving me following error.

Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONException at QClient.main(QClient.java:30) Caused by: java.lang.ClassNotFoundException: org.json.JSONException at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 1 more

In-fact I have created a new project but code is just not working with the jar file. Am I missing something here ? target.zip

Please find attach generated .jar files

[jimmoores]

Hi,

You still need all the dependencies (one of which is a json library) as you do with the normal version, these are normally pulled in by Maven. Are you using Maven? If you are, it should be a simple case of changing your pom.xml to refer to version 1.4.0-SNAPSHOT of quandl4j instead of 1.3.0 (after running 'mvn install' to put those files you attached in 'target.zip' into your local maven repository). If you're not using maven, you need to pull in the dependencies as you did previously. If you're using gradle, make sure you follow the gradle instructions in the docs, as you'll need to add a repository - one of my libraries (actually the json one), is in a repo that isn't maven central (it's maven.opengamma.com). I need to replace that dependency with something else at some point to simplify things...

Hope that helps,

Cheers,

Jim

On 17 July 2016 at 15:36, Pushpendra.paliwal@gmail.com < notifications@github.com> wrote:

Hi , I must say you know your code better then I do 🎯

I was trying to create an additional class without headers. But your way of thinking is quite out of the box.

I checked out code on my PC and all examples are running well.

BUT playing with . jar file is giving me following error.

Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONException at QClient.main(QClient.java:30) Caused by: java.lang.ClassNotFoundException: org.json.JSONException at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 1 more

In-fact I have created a new project but code is just not working with the jar file. Am I missing something here ? target.zip https://github.com/jimmoores/quandl4j/files/367880/target.zip

Please find attach generated .jar files

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jimmoores/quandl4j/issues/18#issuecomment-233184957, or mute the thread https://github.com/notifications/unsubscribe-auth/AADG-7diZnGIyM9dRBniDVGxq7fkcvWUks5qWj3ogaJpZM4JMgjQ .

[atom217]

Hi Jim ,

I am closing this issue on my side.

Thanks for help

[jimmoores]

Hi,

Did it work for you? Or have you persued another avenue? I just need to know whether I should do a release.

Jim

On Monday, 18 July 2016, Pushpendra.paliwal@gmail.com < notifications@github.com> wrote:

Hi Jim ,

I am closing this issue on my side.

Thanks for help

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jimmoores/quandl4j/issues/18#issuecomment-233248341, or mute the thread https://github.com/notifications/unsubscribe-auth/AADG-6bJbRSwydoYO2d1L8LEerkbvixrks5qWySvgaJpZM4JMgjQ .

[atom217]

Yes it did worked for me after adding all dependencies to the .pom file. It worked for me.

[jimmoores]

Ok, thanks, I'll do a release in the next day or two and comment back here so you can change your pom to point to the version in maven central.

[benmccann]

This parameter is now called exclude_column_names in the v3 API