search

navis-org / pymaid

Python library to interface with CATMAID servers. Fully interoperable with navis.
https://pymaid.readthedocs.io/en/latest/
GNU General Public License v3.0
23 stars 11 forks source link

connection to server reset by peer during get_neuron call #245

Open rbetzel opened 5 months ago

rbetzel commented 5 months ago

Trying to import a set of neurons but never get further than ~20% before connection issues. Until this afternoon this was never a problem. Any guidance is welcome. See attached screenshots for partial descriptions of errors.

Screen Shot 2024-04-11 at 12 41 02 PM Screen Shot 2024-04-11 at 12 40 55 PM
clbarnes commented 5 months ago

I think I've had similar issues before with large sets of neurons. I'm not sure why, as pymaid makes a separate request for each neuron so the payload for each isn't very big and its concurrency is limited so the server shouldn't get upset. But try using the annotations to get the skids, then chunk them and make several calls to get_neuron, see if that helps.

schlegelp commented 5 months ago

I would also try to reduce the number of concurrent requests and see if that improves things:

>>> rm = pymaid.CatmaidInstance("URL", None, max_threads=5)  # default is max_threads=10
schlegelp commented 5 months ago

FYI: I'm also seeing sporadic 500 Errors on my end and Robbie from VFB says they had a massive outage and are still recovering.

clbarnes commented 5 months ago

Did we DDoS VFB :eyes:

schlegelp commented 5 months ago

Haha, I don't think so. From what Robbie told me it's a mix of hardware and certificate issues.