Closed arcosx closed 2 years ago
Hello, this issue seems still deeply related to the two previous issues that you linked, I appreciate it if you could allow some time for further investigation. And thanks for the detailed case trial. @arcosx
Hello, this issue seems still deeply related to the two previous issues that you linked, I appreciate it if you could allow some time for further investigation. And thanks for the detailed case trial. @arcosx
OK, waiting for good news😁. If there is something I need to cooperate with you can reply.
Hi @kezhenxu94 @Superskyyy , after some research, I don't think skywalking-python has any problem, it's uWSGI that needs special treatment, this solution below can solve my problem, including issues https://github.com/apache/skywalking/issues/6324.
from uwsgidecorators import postfork
from skywalking import agent, config
@postfork
def init_tracing():
config.protocol = 'grpc'
config.init(collector_address='xxxxx', service_name='test-1')
config.flask_collect_http_params = True
config.logging_level = 'DEBUG'
agent.start()
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
From uWSGI doc we can know
uWSGI is a preforking (or “fork-abusing”) server, so you might need to execute a fixup task after each fork(). The postfork decorator is just the ticket. You can declare multiple postfork tasks. Each decorated function will be executed in sequence after each fork().
Do you have a better solution? Maybe we can note this in skywalking's documentation, after all uWsgi is a popular library.
@arcosx Hello, great to see you found a solution, would you mind opening a PR to document this? I believe it is valuable since both packages are widely used, and from our observation, the issue occurs rather frequently. Probably this should go to the FAQ folder, with a doc name of your choice. https://github.com/apache/skywalking-python/tree/master/docs/en/setup/faq
@arcosx thanks for the sharing, I'll modify the label to documentation
if you'd like to open a PR to add that into our doc.
@arcosx Hello, great to see you found a solution, would you mind opening a PR to document this? I believe it is valuable since both packages are widely used, and from our observation, the issue occurs rather frequently. Probably this should go to the FAQ folder, with a doc name of your choice. https://github.com/apache/skywalking-python/tree/master/docs/en/setup/faq
I think adding to the FAQ is a good way 😃 and I will submit a PR.
Notice, if FAQ menu doesn't exist, you need to update menu yaml file to make this available on the website.
To be specific, this file https://github.com/apache/skywalking-python/blob/master/docs/menu.yml#L43.
@arcosx Any update?
@arcosx Any update?
Sorry for the late submit, the PR here https://github.com/apache/skywalking-python/pull/188
Hi @Superskyyy I'm trying to run a Django application with gunicorn. It is unable to start the agent. I followed this thread and https://opentelemetry-python.readthedocs.io/en/latest/examples/fork-process-model/README.html
Here is my wsgi.py
import os
from django.core.wsgi import get_wsgi_application
from uwsgidecorators import postfork
from skywalking import agent, config
def post_fork(server, worker):
print("abc")
config.init(collector_address='xxxxx', service_name='test-1')
agent.start()
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")
application = get_wsgi_application()
this is the command to start the application gunicorn myproject.wsgi --workers 2
Any suggestions.
@pacific28 I think your issue originates from that you haven't registered the
post_fork
hook in the gunicorn startup command.
try something like this gunicorn -c path_to_your_hook_config myproject.wsgi --workers 2
Take a look here for an complete example https://github.com/benoitc/gunicorn/blob/master/examples/example_config.py
Actually we have our own docs for using with post-fork servers now, feel free to enhance it as it didn't explicitly mention gunicorn.
https://skywalking.apache.org/docs/skywalking-python/latest/en/setup/faq/how-to-use-with-uwsgi/
@Superskyyy Thanks for your response. Tried with config file and it worked.
I have a similar problem. wsgi.py: `import os
from django.core.wsgi import get_wsgi_application from skywalking import agent, config
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'self-healing.settings_prod')
config.init(collector_address='xxxx:12800', service_name='self-healing', protocol='http') agent.start()
`
start command:
gunicorn self-healing.wsgi:application -b0.0.0.0:8000 -w1 --access-logfile=/tmp/access.log
skywalking web: `Traceback: Traceback (most recent call last): File "/root/miniconda3/envs/self-healing/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 233, in _get_session return self._session_cache AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/root/miniconda3/envs/self-healing/lib/python3.7/site-packages/skywalking/plugins/sw_mysqlclient.py", line 70, in execute return self._self_cur.execute(query, args) File "/root/miniconda3/envs/self-healing/lib/python3.7/site-packages/skywalking/plugins/sw_pymysql.py", line 49, in _sw_execute span.tag(TagDbInstance((this.connection.db or b'').decode('utf-8'))) AttributeError: 'str' object has no attribute 'decode'`
Thanks for your help
Hello this is not the same problem related to forking. It seems like its not a problem with skywalking agent but with Django?
Hello this is not the same problem related to forking. It seems like its not a problem with skywalking agent but with Django?
I can access it normally through python manage.py run without reporting an error, but there will be a problem through gunicorn. Finally, I refer to https://skywalking.apache.org/docs/skywalking-python/next/en/setup/faq/how-to-use-with-uwsgi/, and the same problem occurs through uwsgi.
Is there any way to find out what the problem is?
Hello this is not the same problem related to forking. It seems like its not a problem with skywalking agent but with Django?
I can access it normally through python manage.py run without reporting an error, but there will be a problem through gunicorn. Finally, I refer to skywalking.apache.org/docs/skywalking-python/next/en/setup/faq/how-to-use-with-uwsgi, and the same problem occurs through uwsgi.
Is there any way to find out what the problem is?
Is everything fine when you use uwsgi/gunicorn Django without enabling skywalking agent? Thanks.
There could be bug in the plugins as they cannot be comprehensively tested. In the case of
span.tag(TagDbInstance((this.connection.db or b'').decode('utf-8')))
AttributeError: 'str' object has no attribute 'decode'`
Probably the plugin was outdated and it should be without the .decode('utf-8')
part, can you remove it and see if everything works>? @weiwei-9527
@Superskyyy It is normal without enabling the agent. When I start with gunicorn, the debugging code and execution results are as follows:
span.tag(TagDbType('mysql'))
print(this.connection.db)
print(type(this.connection.db))
#span.tag(TagDbInstance((this.connection.db or b'').decode('utf-8')))
span.tag(TagDbStatement(query))
result: b'self-healing' <class 'bytes'> self-healing <class 'str'> self-healing <class 'str'> self-healing <class 'str'> self-healing <class 'str'> self-healing <class 'str'> self-healing <class 'str'>
I finally referred to the solution of # 7457, specified post_fork in gunicorn, and encountered a similar # 7457 problem.
@Beancurd-vv @pacific28 @WinterWaterWarm FYI 1.0.0 is voting soon, prefork server support is added and can be automatically injected without postfork hooks.
Search before asking
Apache SkyWalking Component
Python Agent (apache/skywalking-python)
What happened
I can't use skywalking under uwsgi and flask in master workers mode and threads mode. I have try solution in https://github.com/apache/skywalking/issues/5263 and https://github.com/apache/skywalking/issues/6324 , but not work for me.
When I use uwsgi in master workers mode and threads mode, the skywalking python sdk does not report the metrics while the program is running, it reports metrics only after I close uwsgi...
I think this problem is caused by mixing gevent and uwsgi together.
What you expected to happen
sdk should be able to report metrics in real time...
How to reproduce
Ways
clear Docker env
code 1
code 2
uwsgi run mode 1
uwsgi run mode 2
Result
Not use uwsgi Work OK
code 1 + uwsgi run mode 1 Not OK
code 1+ uwsgi run mode 2 NOT OK
code 2+ uwsgi run mode 1 NOT OK
hang when request ...
code 2+ uwsgi run mode 2 NOT OK
Anything else
No response
Are you willing to submit PR?
Code of Conduct