Open Md-Aashiqq opened 1 year ago
Could you please post the error logs?
Also, how large is the data? That is, for the following line:
ws.send(data) # Connection drops when data is audio bytes
What is the output of
print(len(data))
Which websocket server are you using, i.e., which file are you running?
from flask import Flask, jsonify
from flask_socketio import join_room, send, SocketIO
import websocket
import random
from string import ascii_uppercase
from flask_cors import CORS
import uuid
import threading
import time
from datetime import datetime
app = Flask(__name__)
CORS(app)
app.config["SECRET_KEY"] = "somesecretkey"
socketio = SocketIO(app, cors_allowed_origins=["http://localhost:5500"])
SERVER_B_WS_URL = 'ws://localhost:6006'
ws = None # Declare ws here to ensure it's globally accessible
def on_close(ws, *args):
print("WebSocket connection closed. Reconnecting...")
time.sleep(5) # Wait for 5 seconds before retrying
ws.connect(SERVER_B_WS_URL) # Reconnect logic
def on_open(ws):
print("WebSocket connection opened.")
def on_message(ws, message):
print(f'Received response from Server B: ', message)
def ws_run():
global ws
ws = websocket.WebSocketApp(SERVER_B_WS_URL,
on_message=on_message,
on_close=on_close,
on_open=on_open)
ws.run_forever()
if ws is None:
ws_thread = threading.Thread(target=ws_run)
ws_thread.start()
@socketio.on("message")
def recieve_from_frontend(data):
print(f'Sending data to Server B {datetime.now()}')
print("Len of the Data",len(data))
if not isinstance(data, bytes):
print("Expected bytes, received another datatype.")
return jsonify({"error": "Invalid data format received from frontend"}), 400
ws.send(data)
@socketio.on("connect")
def connect():
user_uuid = str(uuid.uuid4())
print(f"New connection. Assigning UUID: {user_uuid}")
# Join the user to the room with the UUID
join_room(user_uuid)
send({"status": "Connected to the Server", "uuid": user_uuid})
if __name__ == "__main__":
socketio.run(app,debug=True, use_reloader=False)
WebSocket connection closed. Reconnecting...
Sending data to Server B 2023-08-11 14:45:48.410658
Len of the Data 2972
<class 'bytes'>
What are your error logs?
Which file from sherpa-onnx are you using?
I'm encountering an issue. When I attempt to send data to sherpa-onnx, the connection automatically closes. there was no Error Message.
And Im using Streaming_Server.py in Python-api-emaple folder
Could you use the latest streaming_server.py?
There should be error logs from the server as well as from the client side.
Any output is helpful for debugging your issue.
Yes, I'm using the latest streaming_Server.py.
This is the log file from the sherpa-onnx- server.
2023-08-11 07:13:45,856 INFO [streaming_server.py:631] {'encoder_model': './sherpa-onnx-streaming-zipformer-en-2023-06-21/encoder-epoch-99-avg-1.int8.onnx', 'decoder_model': './sherpa-onnx-streaming-zipformer-en-2023-06-21/decoder-epoch-99-avg-1.int8.onnx', 'joiner_model': './sherpa-onnx-streaming-zipformer-en-2023-06-21/joiner-epoch-99-avg-1.int8.onnx', 'tokens': './sherpa-onnx-streaming-zipformer-en-2023-06-21/tokens.txt', 'sample_rate': 16000, 'feat_dim': 80, 'decoding_method': 'greedy_search', 'num_active_paths': 4, 'use_endpoint': 1, 'rule1_min_trailing_silence': 2.4, 'rule2_min_trailing_silence': 1.2, 'rule3_min_utterance_length': 20, 'port': 6006, 'nn_pool_size': 1, 'max_batch_size': 50, 'max_wait_ms': 10, 'max_message_size': 1048576, 'max_queue_size': 32, 'max_active_connections': 500, 'num_threads': 1, 'certificate': None, 'doc_root': './web'}
2023-08-11 07:13:53,899 INFO [streaming_server.py:476] No certificate provided
2023-08-11 07:13:53,909 INFO [server.py:711] server listening on [::]:6006
2023-08-11 07:13:53,912 INFO [server.py:711] server listening on 0.0.0.0:6006
2023-08-11 07:13:53,918 INFO [streaming_server.py:493] Please visit one of the following addresses:
http://0.0.0.0:6006
http://localhost:6006
http://127.0.0.1:6006
http://172.17.0.2:6006
2023-08-11 07:13:59,361 INFO [server.py:646] connection open
2023-08-11 07:13:59,363 INFO [streaming_server.py:534] Connected: ('172.17.0.1', 56670). Number of connections: 1/500
2023-08-11 07:15:14,339 INFO [streaming_server.py:513] ('172.17.0.1', 56670) disconnected
2023-08-11 07:15:14,343 INFO [streaming_server.py:518] Disconnected: ('172.17.0.1', 56670). Number of connections: 0/500
2023-08-11 07:15:14,345 INFO [server.py:268] connection closed
2023-08-11 07:15:23,890 INFO [server.py:646] connection open
2023-08-11 07:15:23,891 INFO [streaming_server.py:534] Connected: ('172.17.0.1', 57322). Number of connections: 1/500
2023-08-11 07:15:45,342 INFO [streaming_server.py:513] ('172.17.0.1', 57322) disconnected
2023-08-11 07:15:45,346 INFO [streaming_server.py:518] Disconnected: ('172.17.0.1', 57322). Number of connections: 0/500
2023-08-11 07:15:45,347 INFO [server.py:268] connection closed
2023-08-11 09:15:30,493 INFO [server.py:646] connection open
2023-08-11 09:15:30,494 INFO [streaming_server.py:534] Connected: ('172.17.0.1', 59944). Number of connections: 1/500
2023-08-11 09:15:53,406 INFO [streaming_server.py:513] ('172.17.0.1', 59944) disconnected
2023-08-11 09:15:53,409 INFO [streaming_server.py:518] Disconnected: ('172.17.0.1', 59944). Number of connections: 0/500
2023-08-11 09:15:53,411 INFO [server.py:268] connection closed
After sending the first chuck audio data, the connection was closed.
WebSocket connection opened.
127.0.0.1 - - [11/Aug/2023 14:45:46] "GET /socket.io/?EIO=4&transport=polling&t=OdZuMIH HTTP/1.1" 200 -
127.0.0.1 - - [11/Aug/2023 14:45:46] "GET /socket.io/?EIO=4&transport=polling&t=OdZuMIQ&sid=7W9D6BtYrcewzJHAAAAA HTTP/1.1" 200 -
New connection. Assigning UUID: d0d71114-3f81-4868-bab7-0cc13df133d4
127.0.0.1 - - [11/Aug/2023 14:45:46] "POST /socket.io/?EIO=4&transport=polling&t=OdZuMIP&sid=7W9D6BtYrcewzJHAAAAA HTTP/1.1" 200 -
Sending data to Server B 2023-08-11 14:45:48.372287
Len of the Data 2972
<class 'bytes'>
<class 'bytes'>
<class 'bytes'>
WebSocket connection closed. Reconnecting...
Sending data to Server B 2023-08-11 14:45:48.410658
Len of the Data 2972
<class 'bytes'>
Error sending chunk: Connection is already closed.
Yes, I'm using the latest streaming_Server.py.
No, yours is not the latest one.
def recieve_from_frontend(data):
What is the format of data?
Does https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/online-websocket-client-decode-file.py work for you?
If yes, I suggest that you check the message format you are using.
Its a -> <class 'bytes'>
I know that it is a byte array. What I am meaning is that what is the content in the byte array.
Does https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/online-websocket-client-decode-file.py work for you?
I'm utilizing the web code from the Sherpa example, and I've transmitted the audio data.
Could you please explain how can I use this - https://github.com/k2-fsa/sherpa-onnx/blob/master/python-api-examples/online-websocket-client-decode-file.py
Please see the documentation https://k2-fsa.github.io/sherpa/onnx/websocket/online-websocket.html
By the way, k2-fsa/sherpa and k2-fsa/sherpa-onnx do NOT share the same message format.
Sorry, the correct link should be
https://k2-fsa.github.io/sherpa/onnx/python/streaming-websocket-server.html
@Md-Aashiqq Have you fixed it?
Description:
I'm experiencing an issue where the connection is unexpectedly terminated when I forward audio data from my Flask server to the Sherpa server using WebSockets. When sending string data, the connection remains stable; however, with audio data (binary), the connection gets closed.
Expected Behavior:
Current Behavior:
Steps to Reproduce:
Code Snippet (if applicable):
Environment:
Additional Context:
The primary goal is to forward audio data in chunks from the frontend to the Sherpa server via the Flask intermediary server. This workflow is crucial for real-time audio processing and analysis.