search

molgenis / molgenis-tools-emx-downloader

Stand alone tool to download data in EMX format from molgenis servers
GNU Lesser General Public License v3.0
2 stars 8 forks source link

Cannot reliably download large datasets from servers < 6.x #61

Open fdlk opened 6 years ago

fdlk commented 6 years ago

Downloader doesn't specify sort for large data sets, which may trigger https://github.com/molgenis/molgenis/issues/6836 on servers with version below 6.x

esthervanenckevort commented 6 years ago

Should be fixed now? Can close ticket?

cpavanrun commented 5 years ago

I am running into issues with any entity with a number larger than pageSize against a molgenis instance of v4.1. Simply turning the pageSize sufficiently low can easily reproduce this.

And I am wondering if this issue the cause? Should this be fixed for versions below 6.x?

[ERROR] - Error streaming entity data for sys_FileMeta.
org.apache.http.conn.HttpHostConnectException: Connect to localhost:8080 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:158)
Caused by: java.net.ConnectException: Connection refused (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
marijevdgeest commented 5 years ago

Hi Chris @cpavanrun ,

Can you open a new issue and provide us with more information, so we can reproduce this? If I look at the error, it seems there can't be a connection established with localhost:8080, are you sure the Molgenis is running there?

cpavanrun commented 5 years ago

I will create a new ticket with more information and a reproduction scheme under docker. The localhost:8080 was reachable: it downloaded up to the pageSize limit but then failed. Other entities with instances/rows under the pageSize downloaded without errors.

~Further testing suggest that it might be related to the reverse-proxy setup with httpd/apache that is being used. Once debugged I will post an issue with solution and directly close it again. For prosperity.~

Created an issue on the main molgenis board: https://github.com/molgenis/molgenis/issues/8534