search

paleobiodb / data_service

The PBDB Data Service, API and table/system maintenance scripts
Artistic License 2.0
12 stars 0 forks source link

Large downloads produce 500 error. #37

Closed markuhen closed 6 years ago

markuhen commented 6 years ago

From Steve Wang (swang1@swarthmore.edu:

I'm trying to download phylum- or class-level datasets from @PaleoDB, but I keep getting '500 Server Error' using either the API or Navigator. It seems to work for smaller taxa (Bryozoa), but not for larger ones (Mollusca). Is there a way to fix this problem, or a workaround?

Example: the following gives me a ‘500 Server error’:

https://paleobiodb.org/data1.2/occs/list.csv?datainfo&rowcount&base_name=MOLLUSCA&taxon_reso=lump_genus&show=class,acconly

but if I replace MOLLUSCA with BRYOZOA, it works fine. It seems to get stuck on larger taxa (with many genera).

I have been able to reproduce this problem.

mmcclenn commented 6 years ago

I stopped and restarted the MariaDB server (the database manager), and I believe this has fixed both this problem and #36. I'm not sure what was going on, but I think it was a glitch in the operation of the database manager itself. Something involving temporary files. Please re-try your download and confirm that the problem is now gone.

jpjenk commented 6 years ago

Thank you, @mmcclenn. I've tested with the above query and several other large data volume retrievals. Valid data retuned with HTTP Status 200 OK each time. The query mentioned in this thread takes almost 2 minutes to compile so don't worry, it's working.