CFDev suddenly stops working with EOF

[NickyMateev]

CFDev suddenly stops working with EOF

Hey, so as part of an E2E test suite I have a client which communicates with cfdev and frequently creates/deletes organizations, asserts the existance of organizations, service-brokers and service offerings.

A problem that I've stumbled upon is that I can run my test suite only one, max 2 times, because cfdev stops working at some point so my calls eventually fail. More specifically, all requests start failing with errors like this one:

Error performing request: Get https://api.dev.cfdev.sh/v2/service_brokers: EOF

Even when I try to manually log in through cf CLI, I get:

Error performing request: Get https://api.dev.cfdev.sh/v2/info: EOF

Seems like all calls to CC endpoints fail. This is very inconvenient, because the only solution I've found so far is to restart cfdev, which costs me at least 20min.

I'd be happy to get some pointers as to where to look, why cfdev might be crashing, any logs I can check..

Thanks!

[cf-gitbot]

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/165568156

The labels on this github issue will be updated when the story is started.

[aemengo]

@NickyMateev Depending on how rigorous your test suite is, your CF Dev instance might be running out of RAM. If your workstation has some to spare, it might be worth starting CF Dev with the -m flag in order to allocate more memory to the VM.

[NickyMateev]

@aemengo The test suite really isn't that rigorous. I also had that assumption so I tried executing only one of the tests which only creates 3 organizations and registers a couple of service-brokers. Everything seems okay on the first run, but on the second I start getting this error.

I even tried setting the memory parameter to 12GB, alas, it did not help..

[aemengo]

Certainly a bit odd...

Well the VM is still up and kicking if you even get that response at all but the internal deployment is in a bad state. There's really isn't much to go on, plus we've never really seen the phenomenon that you're describing. If you're able to give us some reproduction steps, we'll be happy to look further into it.

[NickyMateev]

Well I had some assumptions and theories what might be causing the issue and tried deriving smaller code samples which attempt to prove my assumptions, but I couldn't really reproduce the issue in an isolated context. So I'm still not sure what might be the issue.

I'm closing this issue, since it's so high-level and not concrete and when I do have some progress I can post an update here.

It would be helpful if I knew where I could look at some logs related to the CF deployment itself so that I can inspect it in different scenarios.