QSWWLTN / DigitalLife

174 stars 69 forks source link

什么报错都没有就就在Temp/Test.wav倒出了音频就没下文了 #3

Closed Wuduo1234 closed 1 year ago

QSWWLTN commented 1 year ago

运行后检查下是否出现了Log信息，如果发送时出现了发送数据量：xxxxx的情况，请检查下服务器的Log，是否正常连接上

Wuduo1234 commented 1 year ago

运行后检查下是否出现了Log信息，如果发送时出现了发送数据量：xxxxx的情况，请检查下服务器的Log，是否正常连接上

你好，server端是显示连接上了，而且我用之前beta版里封装好的Client是能跑通的，这是server端显示的全部日志： 2023-05-24 08:50:25,245 INFO Initializing Server... 2023-05-24 08:50:25,278 INFO Initializing ASR Service... 2023-05-24 08:50:35,999 INFO Initializing ChatGPT Service... 2023-05-24 08:50:35,999 INFO chatGPT prompt: paimon35.txt 2023-05-24 08:50:36,202 INFO machine identifier: 7C:10:C9:9E:16:43 2023-05-24 08:50:37,071 INFO using custom API proxy, with rate limit. 2023-05-24 08:50:37,441 INFO API Chatbot initialized. 2023-05-24 08:50:37,441 INFO Initializing TTS Service for character_paimon... 2023-05-24 08:51:13,941 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864) 2023-05-24 08:51:13,965 INFO Initializing Sentiment Engine... 2023-05-24 08:51:19,695 INFO Server is listening on 192.168.1.131:38438... 2023-05-24 08:59:27,410 INFO Connected by ('192.168.1.131', 3452)

ue主要是没有报错 ue的log是这样的： PIE: New page: PIE会话：Medieval_Village_Demo（2023年5月24日下午4:58:42） LogOnline: OSS: Created online subsystem instance for: NULL LogOnline: OSS: TryLoadSubsystemAndSetDefault: Loaded subsystem for module [NULL] LogPlayLevel: Creating play world package: /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo LogPlayLevel: PIE: StaticDuplicateObject took: (0.031661s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/Medieval_Village/maps/Medieval_Village_Demo.Medieval_Village_Demo to /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo (0.032142s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world Medieval_Village_Demo LogPlayLevel: PIE: World Init took: (0.012116s) LogUObjectHash: Compacting FUObjectHashTables data took 0.35ms LogAudio: Display: Creating Audio Device: Id: 2, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device Headphones (High Definition Audio Device) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=2 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=2 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=2 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=2 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 2) registered with world 'Medieval_Village_Demo'. LogLoad: Game class is 'BaseGameModeBase_C' LogWorld: Bringing World /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo up for play (max tick rate 0) at 2023.05.24-08.58.42 LogWorld: Bringing up level for play took: 0.004474 LogOnline: OSS: Created online subsystem instance for: :Context_1 LogViewport: Scene viewport resized to 757x282, mode Windowed. LogViewport: Scene viewport resized to 318x403, mode Windowed. PIE: 登陆的服务器 PIE: PIE总开始时间：0.996秒。 LogViewport: Display: Viewport HideCursorDuringCapture Changed, False -> True LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> CaptureDuringMouseDown LogViewport: Scene viewport resized to 714x403, mode Windowed.

LogBlueprintUserMessages: [BaseTargetActor_C_0] D:/workplaces/ai_paimon/DigitalLife/Temp/Test.wav

kxylxx commented 1 year ago

您好，请问解决了吗？我也是走到这步，和您稍微不同的是在点击Z键输入语言时，前端log会一直显示： LogUObjectBase: Warning: NULL object 服务端log没有异样，请教一下这个该如何处理？

kxylxx commented 1 year ago

后端log： 2023-05-24 15:22:08,085 INFO Initializing Server... 2023-05-24 15:22:08,087 INFO Initializing ASR Service... 2023-05-24 15:22:10,242 INFO Initializing ChatGPT Service... 2023-05-24 15:22:10,242 INFO chatGPT prompt: paimon35.txt 2023-05-24 15:22:10,294 INFO machine identifier: 74:56:3C:71:30:9B 2023-05-24 15:22:10,365 INFO using custom API proxy, with rate limit. 2023-05-24 15:22:10,543 INFO API Chatbot initialized. 2023-05-24 15:22:10,544 INFO Initializing TTS Service for character_paimon... 2023-05-24 15:22:11,151 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864) 2023-05-24 15:22:11,166 INFO Initializing Sentiment Engine... 2023-05-24 15:22:12,726 INFO Server is listening on 192.168.1.105:38438... 2023-05-24 15:22:14,901 INFO Connected by ('192.168.1.105', 53115) 前端log： LogDebuggerCommands: Repeating last play command: 选中的视口 LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE会话：Medieval_Village_Demo（2023年5月24日下午11:37:35） LogPlayLevel: Creating play world package: /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo LogPlayLevel: PIE: StaticDuplicateObject took: (0.007372s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/Medieval_Village/maps/Medieval_Village_Demo.Medieval_Village_Demo to /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo (0.007593s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world Medieval_Village_Demo LogPlayLevel: PIE: World Init took: (0.001368s) LogUObjectHash: Compacting FUObjectHashTables data took 0.93ms LogAudio: Display: Creating Audio Device: Id: 12, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device PHL 273V7 (NVIDIA High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=12 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=12 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=12 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=12 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 12) registered with world 'Medieval_Village_Demo'. LogLoad: Game class is 'BaseGameModeBase_C' LogWorld: Bringing World /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo up for play (max tick rate 0) at 2023.05.24-15.37.35 LogWorld: Bringing up level for play took: 0.000461 LogOnline: OSS: Created online subsystem instance for: :Context_15 LogViewport: Scene viewport resized to 282x822, mode Windowed. PIE: 登陆的服务器 PIE: PIE总开始时间：0.158秒。 LogViewport: Display: Viewport HideCursorDuringCapture Changed, False -> True LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> CaptureDuringMouseDown LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogBlueprintUserMessages: [BaseTargetActor_C_0] C:/Users/xss/project/DigitalLife/Temp/Test.wav

Wuduo1234 commented 1 year ago

后端log： 2023-05-24 15:22:08,085 INFO Initializing Server... 2023-05-24 15:22:08,087 INFO Initializing ASR Service... 2023-05-24 15:22:10,242 INFO Initializing ChatGPT Service... 2023-05-24 15:22:10,242 INFO chatGPT prompt: paimon35.txt 2023-05-24 15:22:10,294 INFO machine identifier: 74:56:3C:71:30:9B 2023-05-24 15:22:10,365 INFO using custom API proxy, with rate limit. 2023-05-24 15:22:10,543 INFO API Chatbot initialized. 2023-05-24 15:22:10,544 INFO Initializing TTS Service for character_paimon... 2023-05-24 15:22:11,151 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864) 2023-05-24 15:22:11,166 INFO Initializing Sentiment Engine... 2023-05-24 15:22:12,726 INFO Server is listening on 192.168.1.105:38438... 2023-05-24 15:22:14,901 INFO Connected by ('192.168.1.105', 53115) 前端log： LogDebuggerCommands: Repeating last play command: 选中的视口 LogPlayLevel: PlayLevel: No blueprints needed recompiling PIE: New page: PIE会话：Medieval_Village_Demo（2023年5月24日下午11:37:35） LogPlayLevel: Creating play world package: /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo LogPlayLevel: PIE: StaticDuplicateObject took: (0.007372s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/Medieval_Village/maps/Medieval_Village_Demo.Medieval_Village_Demo to /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo (0.007593s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world Medieval_Village_Demo LogPlayLevel: PIE: World Init took: (0.001368s) LogUObjectHash: Compacting FUObjectHashTables data took 0.93ms LogAudio: Display: Creating Audio Device: Id: 12, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device PHL 273V7 (NVIDIA High Definition Audio) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=12 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=12 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=12 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=12 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 12) registered with world 'Medieval_Village_Demo'. LogLoad: Game class is 'BaseGameModeBase_C' LogWorld: Bringing World /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo up for play (max tick rate 0) at 2023.05.24-15.37.35 LogWorld: Bringing up level for play took: 0.000461 LogOnline: OSS: Created online subsystem instance for: :Context_15 LogViewport: Scene viewport resized to 282x822, mode Windowed. PIE: 登陆的服务器 PIE: PIE总开始时间：0.158秒。 LogViewport: Display: Viewport HideCursorDuringCapture Changed, False -> True LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> CaptureDuringMouseDown LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogUObjectBase: Warning: NULL object LogBlueprintUserMessages: [BaseTargetActor_C_0] C:/Users/xss/project/DigitalLife/Temp/Test.wav

这个没啥，就是你按住按键盘任意一个键位的时候就会报这个警告，这东西应该不会影响程序运行，我也有这个警告，但我把日志里这个警告删掉了

kxylxx commented 1 year ago

目前来看是后端问题， SocketServer.py这个是主程序按Z键开始对话后，data = self.conn.recv(1024) 运行到这边就不走了，好像是服务被挂起了，所以就没有然后了 def listen(self):

MAIN SERVER LOOP

    while True:
        self.s.listen()
        logging.info(f"Server is listening on {self.host}:{self.port}...")
        self.conn, self.addr = self.s.accept()
        logging.info(f"Connected by {self.addr}")
        logging.info(f"1")
        self.conn.sendall(b'%s' % self.char_name[args.character][2].encode())
        logging.info(f"2")

        while True:
            try:
                logging.info(f"3")
                file = self.__receive_file() #运行到这个函数

另外这两个路径要改： self.tmp_recv_file = 'tmp/server_received.wav' 改为： self.tmp_recv_file = ‘D://workplaces/ai_paimon/DigitalLife/Temp/Test.wav’ self.tmp_proc_file = 'tmp/server_processed.wav' 这个怎么改还没搞清楚

QSWWLTN commented 1 year ago

服务端的Wav路径推荐不要修改，因为可能下一次更新后Wav路径变更后无法正常读取，服务端的话可以去服务端那边去留言下，如果客户端还要其他问题可以继续和我说

QSWWLTN commented 1 year ago

运行后检查下是否出现了Log信息，如果发送时出现了发送数据量：xxxxx的情况，请检查下服务器的Log，是否正常连接上

你好，server端是显示连接上了，而且我用之前beta版里封装好的Client是能跑通的，这是server端显示的全部日志： 2023-05-24 08:50:25,245 INFO Initializing Server... 2023-05-24 08:50:25,278 INFO Initializing ASR Service... 2023-05-24 08:50:35,999 INFO Initializing ChatGPT Service... 2023-05-24 08:50:35,999 INFO chatGPT prompt: paimon35.txt 2023-05-24 08:50:36,202 INFO machine identifier: 7C:10:C9:9E:16:43 2023-05-24 08:50:37,071 INFO using custom API proxy, with rate limit. 2023-05-24 08:50:37,441 INFO API Chatbot initialized. 2023-05-24 08:50:37,441 INFO Initializing TTS Service for character_paimon... 2023-05-24 08:51:13,941 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864) 2023-05-24 08:51:13,965 INFO Initializing Sentiment Engine... 2023-05-24 08:51:19,695 INFO Server is listening on 192.168.1.131:38438... 2023-05-24 08:59:27,410 INFO Connected by ('192.168.1.131', 3452)

ue主要是没有报错 ue的log是这样的： PIE: New page: PIE会话：Medieval_Village_Demo（2023年5月24日下午4:58:42） LogOnline: OSS: Created online subsystem instance for: NULL LogOnline: OSS: TryLoadSubsystemAndSetDefault: Loaded subsystem for module [NULL] LogPlayLevel: Creating play world package: /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo LogPlayLevel: PIE: StaticDuplicateObject took: (0.031661s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/Medieval_Village/maps/Medieval_Village_Demo.Medieval_Village_Demo to /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo (0.032142s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world Medieval_Village_Demo LogPlayLevel: PIE: World Init took: (0.012116s) LogUObjectHash: Compacting FUObjectHashTables data took 0.35ms LogAudio: Display: Creating Audio Device: Id: 2, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device Headphones (High Definition Audio Device) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=2 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=2 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=2 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=2 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 2) registered with world 'Medieval_Village_Demo'. LogLoad: Game class is 'BaseGameModeBase_C' LogWorld: Bringing World /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo up for play (max tick rate 0) at 2023.05.24-08.58.42 LogWorld: Bringing up level for play took: 0.004474 LogOnline: OSS: Created online subsystem instance for: :Context_1 LogViewport: Scene viewport resized to 757x282, mode Windowed. LogViewport: Scene viewport resized to 318x403, mode Windowed. PIE: 登陆的服务器 PIE: PIE总开始时间：0.996秒。 LogViewport: Display: Viewport HideCursorDuringCapture Changed, False -> True LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> CaptureDuringMouseDown LogViewport: Scene viewport resized to 714x403, mode Windowed.

LogBlueprintUserMessages: [BaseTargetActor_C_0] D:/workplaces/ai_paimon/DigitalLife/Temp/Test.wav

Log中有出现发送数据量:xxxx之类的消息嘛？

Wuduo1234 commented 1 year ago

运行后检查下是否出现了Log信息，如果发送时出现了发送数据量：xxxxx的情况，请检查下服务器的Log，是否正常连接上

你好，server端是显示连接上了，而且我用之前beta版里封装好的Client是能跑通的，这是server端显示的全部日志： 2023-05-24 08:50:25,245 INFO Initializing Server... 2023-05-24 08:50:25,278 INFO Initializing ASR Service... 2023-05-24 08:50:35,999 INFO Initializing ChatGPT Service... 2023-05-24 08:50:35,999 INFO chatGPT prompt: paimon35.txt 2023-05-24 08:50:36,202 INFO machine identifier: 7C:10:C9:9E:16:43 2023-05-24 08:50:37,071 INFO using custom API proxy, with rate limit. 2023-05-24 08:50:37,441 INFO API Chatbot initialized. 2023-05-24 08:50:37,441 INFO Initializing TTS Service for character_paimon... 2023-05-24 08:51:13,941 INFO Loaded checkpoint 'TTS/models/paimon6k_390k.pth' (iteration 8864) 2023-05-24 08:51:13,965 INFO Initializing Sentiment Engine... 2023-05-24 08:51:19,695 INFO Server is listening on 192.168.1.131:38438... 2023-05-24 08:59:27,410 INFO Connected by ('192.168.1.131', 3452) ue主要是没有报错 ue的log是这样的： PIE: New page: PIE会话：Medieval_Village_Demo（2023年5月24日下午4:58:42） LogOnline: OSS: Created online subsystem instance for: NULL LogOnline: OSS: TryLoadSubsystemAndSetDefault: Loaded subsystem for module [NULL] LogPlayLevel: Creating play world package: /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo LogPlayLevel: PIE: StaticDuplicateObject took: (0.031661s) LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/Medieval_Village/maps/Medieval_Village_Demo.Medieval_Village_Demo to /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo (0.032142s) LogSubsystemCollection: Failed to initialize subsystem dependency (WorldPartitionSubsystem) LogChaos: FPhysicsSolverBase::AsyncDt:-1.000000 LogAIModule: Creating AISystem for world Medieval_Village_Demo LogPlayLevel: PIE: World Init took: (0.012116s) LogUObjectHash: Compacting FUObjectHashTables data took 0.35ms LogAudio: Display: Creating Audio Device: Id: 2, Scope: Unique, Realtime: True LogAudioMixer: Display: Audio Mixer Platform Settings: LogAudioMixer: Display: Sample Rate: 48000 LogAudioMixer: Display: Callback Buffer Frame Size Requested: 1024 LogAudioMixer: Display: Callback Buffer Frame Size To Use: 1024 LogAudioMixer: Display: Number of buffers to queue: 2 LogAudioMixer: Display: Max Channels (voices): 0 LogAudioMixer: Display: Number of Async Source Workers: 0 LogAudio: Display: AudioDevice MaxSources: 32 LogAudio: Display: Audio Spatialization Plugin: None (built-in). LogAudio: Display: Audio Reverb Plugin: None (built-in). LogAudio: Display: Audio Occlusion Plugin: None (built-in). LogAudioMixer: Display: Initializing audio mixer using platform API: 'XAudio2' LogAudioMixer: Display: Using Audio Hardware Device Headphones (High Definition Audio Device) LogAudioMixer: Display: Initializing Sound Submixes... LogAudioMixer: Display: Creating Master Submix 'MasterSubmixDefault' LogAudioMixer: Display: Creating Master Submix 'MasterReverbSubmixDefault' LogAudioMixer: FMixerPlatformXAudio2::StartAudioStream() called. InstanceID=2 LogAudioMixer: Display: Output buffers initialized: Frames=1024, Channels=2, Samples=2048, InstanceID=2 LogAudioMixer: Display: Starting AudioMixerPlatformInterface::RunInternal(), InstanceID=2 LogAudioMixer: Display: FMixerPlatformXAudio2::SubmitBuffer() called for the first time. InstanceID=2 LogInit: FAudioDevice initialized. LogAudio: Display: Audio Device (ID: 2) registered with world 'Medieval_Village_Demo'. LogLoad: Game class is 'BaseGameModeBase_C' LogWorld: Bringing World /Game/Medieval_Village/maps/UEDPIE_0_Medieval_Village_Demo.Medieval_Village_Demo up for play (max tick rate 0) at 2023.05.24-08.58.42 LogWorld: Bringing up level for play took: 0.004474 LogOnline: OSS: Created online subsystem instance for: :Context_1 LogViewport: Scene viewport resized to 757x282, mode Windowed. LogViewport: Scene viewport resized to 318x403, mode Windowed. PIE: 登陆的服务器 PIE: PIE总开始时间：0.996秒。 LogViewport: Display: Viewport HideCursorDuringCapture Changed, False -> True LogViewport: Display: Viewport MouseCaptureMode Changed, CapturePermanently_IncludingInitialMouseDown -> CaptureDuringMouseDown LogViewport: Scene viewport resized to 714x403, mode Windowed. LogBlueprintUserMessages: [BaseTargetActor_C_0] D:/workplaces/ai_paimon/DigitalLife/Temp/Test.wav

Log中有出现发送数据量:xxxx之类的消息嘛？

没有生成Test.wav之后就没消息了。

imloama commented 1 year ago

目前来看是后端问题， SocketServer.py这个是主程序按Z键开始对话后，data = self.conn.recv(1024) 运行到这边就不走了，好像是服务被挂起了，所以就没有然后了 def listen(self): # MAIN SERVER LOOP while True: self.s.listen() logging.info(f"Server is listening on {self.host}:{self.port}...") self.conn, self.addr = self.s.accept() logging.info(f"Connected by {self.addr}") logging.info(f"1") self.conn.sendall(b'%s' % self.char_name[args.character][2].encode()) logging.info(f"2")
        while True:
            try:
                logging.info(f"3")
                file = self.__receive_file() #运行到这个函数
另外这两个路径要改： self.tmp_recv_file = 'tmp/server_received.wav' 改为： self.tmp_recv_file = ‘D://workplaces/ai_paimon/DigitalLife/Temp/Test.wav’ self.tmp_proc_file = 'tmp/server_processed.wav' 这个怎么改还没搞清楚

我认为这个是正常的，因为后边还是while-true循环，用于等待socket数据。

代码运行的后端，使用视频下最新的apk是可以接收到语音文件的，使用代码编译的apk是无法收到文件的。

WANGRUI-ZB commented 1 year ago

我遇到同样的问题，debug后了一下发现了问题，修改后成功运行原因：SendMessageAsyncTask.cpp发送客户端音频数据线程初始化时T->IsValidLowLevel()的问题

void FSendMessageAsyncTask::DoWork() {
    UClientGameInstance* T = Cast<UClientGameInstance>(Owner);
    while (!bStop) {
        FPlatformProcess::Sleep(0.5);
        if (T->IsValidLowLevel() || Socket == nullptr || T->StopSocket) {
            break;
        }

        if (T->ClientBuffData.Num() == 0) {
            continue;
        }

        BuffSize = 0;
#if NO_PROTOBUF
        if (T->BuffDataSize > 3) {
            if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

                Buff = T->ClientBuffData;
                BuffSize = Buff.Num();
            }
        }
#else
        if (T->BuffDataSize > 3) {
            if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

                T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);
                T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);

                Buff = T->ClientBuffData;
                BuffSize = Buff.Num();
            }
        }
#endif
        if (BuffSize != 0) {
            int32 NowSize;
            if (!Socket->Send(Buff.GetData(), BuffSize, NowSize)) {
                AsyncTask(ENamedThreads::GameThread, [=] {
                    UE_LOG(SendMessageTaskLog, Error, TEXT("发送消息失败"));
                    });
            }
            AsyncTask(ENamedThreads::GameThread, [=] {
                UE_LOG(SendMessageTaskLog, Error, TEXT("发送数据量：%d"), NowSize);
                GEngine->AddOnScreenDebugMessage(-1, 1000.f, FColor::Red, FString(TEXT("发送数据量：")) + FString::FromInt(NowSize));

                T->CloseBuff();
            });
        }
    }
}

debug发现IsValidLowLevel()方法返回了True，直接跳出了循环导致发送音频文件的线程中断，网上查了下该方法的作用：IsValidLowLevel是UE5引擎中的一个函数，它的作用是检查一个UObject对象是否有效。在UE5游戏中，UObject是所有实例对象的基类，因此这个函数可以用来检查任何类型的对象是否有效。如果使用无效的对象指针，很可能会导致游戏崩溃或出现其他异常情况。 此处返回True说明该对象有效，所以不应该break，反之对象无效时break，所以只需要将该方法值取反即可，如下：

if (!(T->IsValidLowLevel()) || Socket == nullptr || T->StopSocket) {
    break;
}

希望了帮到 @kxylxx @Wuduo1234

QSWWLTN commented 1 year ago

我遇到同样的问题，debug后了一下发现了问题，修改后成功运行原因：SendMessageAsyncTask.cpp发送客户端音频数据线程初始化时T->IsValidLowLevel()的问题
void FSendMessageAsyncTask::DoWork() {
  UClientGameInstance* T = Cast<UClientGameInstance>(Owner);
  while (!bStop) {
      FPlatformProcess::Sleep(0.5);
      if (T->IsValidLowLevel() || Socket == nullptr || T->StopSocket) {
          break;
      }

      if (T->ClientBuffData.Num() == 0) {
          continue;
      }

      BuffSize = 0;
#if NO_PROTOBUF
      if (T->BuffDataSize > 3) {
          if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

              Buff = T->ClientBuffData;
              BuffSize = Buff.Num();
          }
      }
#else
      if (T->BuffDataSize > 3) {
          if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

              T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);
              T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);

              Buff = T->ClientBuffData;
              BuffSize = Buff.Num();
          }
      }
#endif
      if (BuffSize != 0) {
          int32 NowSize;
          if (!Socket->Send(Buff.GetData(), BuffSize, NowSize)) {
              AsyncTask(ENamedThreads::GameThread, [=] {
                  UE_LOG(SendMessageTaskLog, Error, TEXT("发送消息失败"));
                  });
          }
          AsyncTask(ENamedThreads::GameThread, [=] {
              UE_LOG(SendMessageTaskLog, Error, TEXT("发送数据量：%d"), NowSize);
              GEngine->AddOnScreenDebugMessage(-1, 1000.f, FColor::Red, FString(TEXT("发送数据量：")) + FString::FromInt(NowSize));

              T->CloseBuff();
          });
      }
  }
}
debug发现IsValidLowLevel()方法返回了True，直接跳出了循环导致发送音频文件的线程中断，网上查了下该方法的作用：IsValidLowLevel是UE5引擎中的一个函数，它的作用是检查一个UObject对象是否有效。在UE5游戏中，UObject是所有实例对象的基类，因此这个函数可以用来检查任何类型的对象是否有效。如果使用无效的对象指针，很可能会导致游戏崩溃或出现其他异常情况。 此处返回True说明该对象有效，所以不应该break，反之对象无效时break，所以只需要将该方法值取反即可，如下：
if (!(T->IsValidLowLevel()) || Socket == nullptr || T->StopSocket) {
  break;
}
希望了帮到 @kxylxx @Wuduo1234

哦，确实，这是我疏忽了，我可能那时候上传的时候在改东西，没有检查到，感谢你的帮助；稍晚点我会重新传下。

kxylxx commented 1 year ago

我遇到同样的问题，debug后了一下发现了问题，修改后成功运行原因：SendMessageAsyncTask.cpp发送客户端音频数据线程初始化时T->IsValidLowLevel()的问题
void FSendMessageAsyncTask::DoWork() {
  UClientGameInstance* T = Cast<UClientGameInstance>(Owner);
  while (!bStop) {
      FPlatformProcess::Sleep(0.5);
      if (T->IsValidLowLevel() || Socket == nullptr || T->StopSocket) {
          break;
      }

      if (T->ClientBuffData.Num() == 0) {
          continue;
      }

      BuffSize = 0;
#if NO_PROTOBUF
      if (T->BuffDataSize > 3) {
          if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

              Buff = T->ClientBuffData;
              BuffSize = Buff.Num();
          }
      }
#else
      if (T->BuffDataSize > 3) {
          if (T->ClientBuffData[T->BuffDataSize - 1] == '!' && T->ClientBuffData[T->BuffDataSize - 2] == '?') {

              T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);
              T->ClientBuffData.RemoveAt(T->ClientBuffData.Num() - 1);

              Buff = T->ClientBuffData;
              BuffSize = Buff.Num();
          }
      }
#endif
      if (BuffSize != 0) {
          int32 NowSize;
          if (!Socket->Send(Buff.GetData(), BuffSize, NowSize)) {
              AsyncTask(ENamedThreads::GameThread, [=] {
                  UE_LOG(SendMessageTaskLog, Error, TEXT("发送消息失败"));
                  });
          }
          AsyncTask(ENamedThreads::GameThread, [=] {
              UE_LOG(SendMessageTaskLog, Error, TEXT("发送数据量：%d"), NowSize);
              GEngine->AddOnScreenDebugMessage(-1, 1000.f, FColor::Red, FString(TEXT("发送数据量：")) + FString::FromInt(NowSize));

              T->CloseBuff();
          });
      }
  }
}
debug发现IsValidLowLevel()方法返回了True，直接跳出了循环导致发送音频文件的线程中断，网上查了下该方法的作用：IsValidLowLevel是UE5引擎中的一个函数，它的作用是检查一个UObject对象是否有效。在UE5游戏中，UObject是所有实例对象的基类，因此这个函数可以用来检查任何类型的对象是否有效。如果使用无效的对象指针，很可能会导致游戏崩溃或出现其他异常情况。 此处返回True说明该对象有效，所以不应该break，反之对象无效时break，所以只需要将该方法值取反即可，如下：
if (!(T->IsValidLowLevel()) || Socket == nullptr || T->StopSocket) {
  break;
}
希望了帮到 @kxylxx @Wuduo1234

非常感谢！

kxylxx commented 1 year ago

嗯嗯好的

QSWWLTN / DigitalLife

什么报错都没有就 就在Temp/Test.wav倒出了音频就没下文了 #3

MAIN SERVER LOOP

什么报错都没有就就在Temp/Test.wav倒出了音频就没下文了 #3