search

CCI-MOC / ORE

MOC OpenStack Research Environment
Apache License 2.0
2 stars 3 forks source link

Neutron.conf settings enabled trace_sqlalchemy #24

Open lkatalin opened 6 years ago

lkatalin commented 6 years ago

It's possible this may have just been my system because Emre did not have this problem, but in my newest ORE, /etc/neutron/neutron.conf had neutron.db tracepoints enabled with the line trace_sqlalchemy = True, which gave me really wonky results on the traces until we figured it out.

I edited it to say trace_sqlalchemy = False and restarted with sudo systemctl restart devstack@* and then it worked. But I didn't edit that file to begin with, so I'm not sure why this happened. I brought up the ORE the same way as before, following the instructions in the README.

HuzefaMandvi commented 6 years ago

I just launched a fresh ORE VM and got the same result as you did. Pinging @EmreAtes on this, maybe he will have a better idea of what caused it.

We could just postfix this in stack.sh by replacing the relevant line in /etc/neutron/neutron.conf, but my gut says this is wrong and hack-ey. Especially since you said that this wasn't an issue before.

Would be helpful if you happened to know the exact time you launched this machine and any previous working machines, so we could cross-reference to the exact commits in ORE.

HuzefaMandvi commented 6 years ago

Looking at local.conf, I notice that we have specific trace_sqlalchemy = False for nova, keystone, cinder, and glance. Perhaps we should also be specifying this for neutron? @hgibson1 perhaps you have some insight on the local.conf stuff?

EmreAtes commented 6 years ago

First, I think we have many (all?) services in local.conf, and we can add neutron there as well, since there is a [profiler] option for it, and I don't think it's more hacky than the rest of devstack. This snippet is the correct one for neutron as far as know:


[[post-config|/etc/neutron/neutron.conf]]
[profiler]
enabled = True
connection_string = redis://localhost:6379
hmac_keys = Devstack1
trace_sqlalchemy = False

Also, in osprofiler ocata, only under devstack, trace_sqlalchemy is set to True no matter what local.conf suggests. (see here). There's our fork of it here, which doesn't do this override. However, the sql traces didn't cause a problem so far so I'm not using this fork.

lkatalin commented 6 years ago

In case it's helpful, I launched the VM with this problem at 10:25am July 10. The most recent previous VM that did not have this problem was created at 11:05am July 6. The time zones are whatever is shown in Horizon, presumably Eastern time.

HuzefaMandvi commented 6 years ago

Emre I agree, now that I think about it looks like this does belong in local.conf. I just wanted to get Hannah's input on whether she decided specifically not to include it in local.conf for any reason.

ghost commented 6 years ago

Documentation on neutron suggested trace_sqlalchemy was set to false by default but apparently this is not the case. I'll add trace_sqlalchemy = False to local.conf if it isn't there already

EmreAtes commented 6 years ago

@hgibson1 , this is actually because osprofiler turns on trace_sqlalchemy if it's running with devstack. We can have a fork which doesn't do this, or not.

ghost commented 6 years ago

@EmreAtes which do you think will be easier to manage?

EmreAtes commented 6 years ago

There's already an osprofiler fork that does this at https://github.com/docc-lab/osprofiler I think we can change the default osprofiler to be this one. Note that we'll implement sampling and other features on this fork as well (maybe this week).

If there are going to be other users of ORE who want a separate version of osprofiler, we can think about it then.