configuration generation for large functional groups causes API timeout

[jhpoelen]

Currently, all traits are retrieved from fishbase api synchronously, in a single web call. This means, that if many traits are requested, the call times-out after 30s or so and an error message is created.

Suggested: whenever a big list of groups/taxa is provided, the request to generate the osmose configuration is accepted and some request id is returned. After receiving the request, the server retrieves all the necessary information and creates the osmose_config.zip . Now, the client can retrieve the osmose_config.zip the first time the results of the specific request id is polled.

see #66

[jhpoelen]

I've introduce an optimization that reduces the time to retrieve traits. Previously, all traits for each taxa in a group where retrieved. Now, only traits for a taxon in a group are retrieved if no traits where previously retrieved for other taxa in that group. While the time to retrieve traits are reduced, some traits have sparse data across taxa, causing the algorithm to iterate across large groups of taxa in an attempt to get a first value for a specific trait in that group.

Workaround for this issue: reduce the number of species in functional groups.

@agruss2 would it be worth the effort to get some reporting on sparse traits, so that surgical data improvement can be organized in fishbase?

[jhpoelen]

I've implemented another optimization: instead of retrieving traits one-by-one, the traits are now retrieved in batch using the fishbase cache v2.0.0 (see https://github.com/jhpoelen/fishbase_archiver/releases/tag/v0.2.0).

After applying the changes, I was able to download the osmose configuration without any changes to the configuration wizard (see attached). osmose_config (3).zip

@agruss2 please confirm that you can reproduce.

[jhpoelen]

Closing following fix confirmation through #66 .