search

charlesquick / ntripcaster-containerised

Another containerised ntripcaster!
14 stars 10 forks source link

Bad handling on AMD x64 arch #2

Open PaJaSoft opened 4 months ago

PaJaSoft commented 4 months ago

Hello, I've discovered some weird issue on AMD x64 arch (AMD Ryzen 9 5950X in my case). I wasn't able to authorize my NTRIP-X station although I was sure I've provided correct creds - I've always got 'Bad Password' result.

Digging deeper into the issue and I've discovered that there is some issue related to the HTTP handling and decoding.

The correct decoding looks like:

^M[10/Jun/2024:15:19:26] [2:Connection Handler] Setting fd 6 to blocking                                                                                                                                                                                               
^M[10/Jun/2024:15:19:26] [2:Connection Handler] KUN DEBUG: read_bytes = 1 (P)                                                                                                                                                                                          

^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Encoder logging in with [POST /CZEZLITRA1 HTTP/1.1                                                                                                                                                              
Host: 192.168.111.132                                                                                                                                                                                                                                                  
Ntrip-Version: Ntrip/2.0                                                                                                                                                                                                                                               
User-Agent: NTRIP NTRIP-X/0.8.3                                                                                                                                                                                                                                        
Authorization: Basic <REDACTED>                                                                                                                                                                                                  
Cache-Control: no-store, no-cache, max-age=0                                                                                                                                                                                                                           
Connection: close                                                                                                                                                                                                                                                      

]                                                                                                                                                                                                                                                                      
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [7]: [POST] [/CZEZLITRA1 HTTP/1.1]                                                                                                                                                                  
^M[10/Jun/2024:15:19:26] [2:Connection Handler] Locking 0x563078a19530 (Source Tree Mutex) on line 730 in file utility.c by thread 2                                                                                                                                   
^M[10/Jun/2024:15:19:26] [2:Connection Handler] Locked 0x563078a19530 by thread 2                                                                                                                                                                                      
^M[10/Jun/2024:15:19:26] [2:Connection Handler] Unlocking 0x563078a19530 (Source Tree Mutex) on line 742 in file utility.c by thread 2                                                                                                                                 
^M[10/Jun/2024:15:19:26] [2:Connection Handler] Unlocked 0x563078a19530 by thread 2                                                                                                                                                                                    
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Parsing mount location file /usr/local/ntripcaster/conf/mountpos.conf                                                                                                                                           
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [6]: [Host] [ 192.168.111.132]                                                                                                                                                                      
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [5]: [Ntrip-Version] [ Ntrip/2.0]                                                                                                                                                                   
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [4]: [User-Agent] [ NTRIP NTRIP-X/0.8.3]                                                                                                                                                            
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [3]: [Authorization] [ Basic <REDACTED>]                                                                                                                      
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [2]: [Cache-Control] [ no-store, no-cache, max-age=0]                                                                                                                                               
^M[10/Jun/2024:15:19:26] [2:Connection Handler] DEBUG: Source line [1]: [Connection] [ close]                                                                                                                                                                          
^M[10/Jun/2024:15:19:26] Accepted encoder on mountpoint /CZEZLITRA1 from 192.168.111.39. 1 sources connected

however on the AMD CPU the same part looks like:

[10/Jun/2024:15:19:07] [37:Connection Handler] Setting fd 6 to blocking                                                            
[10/Jun/2024:15:19:07] [37:Connection Handler] KUN DEBUG: read_bytes = 1 (P)                                                       

[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Encoder logging in with [POST /CZEZLITRA1 HTTP/1.1                                                                                                                                                               
Host: 192.168.111.129                                            
Ntrip-Version: Ntrip/2.0                                         
User-Agent: NTRIP NTRIP-X/0.8.3                                  
Authorization: Basic <REDACTED>                                                              
Cache-Control: no-store, no-cache, max-age=0                                                                                       
Connection: close                                                

]                                                                
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [7]: [POST] [/CZEZLITRA1 HTTP/1.1]                                                                                                                                                                   
[10/Jun/2024:15:19:07] [37:Connection Handler] Locking 0x556175a19530 (Source Tree Mutex) on line 730 in file utility.c by thread 37                                                                                                                                   
[10/Jun/2024:15:19:07] [37:Connection Handler] Locked 0x556175a19530 by thread 37                                                  
[10/Jun/2024:15:19:07] [37:Connection Handler] Unlocking 0x556175a19530 (Source Tree Mutex) on line 742 in file utility.c by thread 37                                                                                                                                 
[10/Jun/2024:15:19:07] [37:Connection Handler] Unlocked 0x556175a19530 by thread 37                                                
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Parsing mount location file /usr/local/ntripcaster/conf/mountpos.conf                                                                                                                                            
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [6]: [Host] [ 192.168.111.129]                                                                                                                                                                       
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [5]: [Ntrip-Version] [ Ntrip/2.0]                                                                                                                                                                    
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [4]: [U.8.3] []                                                  
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [3]: [Aent] [ NTRIP NTRIP: Basic SGuthorization: <REDACTED>]                                                                                             
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [2]: [Cache-Control] [ no-store, no-cache, max-age=0]                                                                                                                                                
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Source line [1]: [Connection] [ close]                                                                                                                                                                           
[10/Jun/2024:15:19:07] [37:Connection Handler] KUN DEBUG: info.encoder_pass <REDACTED>                                                                                                                                                                           

[10/Jun/2024:15:19:07] Kicking source 35 [192.168.111.39] [Bad Password] [encoder], connected for 0 seconds, 0 bytes transfered. -1 sources connected                                                                                                                  
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Removing connection 35 of type 1                                                                                                                                                                                 
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: sock_close: Closing socket 6                                                 
[10/Jun/2024:15:19:07] [37:Connection Handler] DEBUG: Closing fd 6

As you can see somehow there is mangled User-Agent and Authorization headers.

I don't have a fix or further info now, the only what I know is that on Intel CPU everything is correct and works as expected. Another symptom is that when I use this chain: NTRIP-X -> ntrip-caster (on Intel CPU) -> ntrip-server -> ntrip-caster (on AMD CPU) , the ntrip-caster on AMD CPU produces a coredump when I put mountpoint, user, passwd on the command line of ntrip-server.

On each CPU platform I build ntripcaster-containerized, so it seems something related to AMD CPU. I'll try to find the issue when time permits...

PaJaSoft commented 4 months ago

After even deeper investigation, it isn't only AMD x64 arch CPU, but the source code of NTRIP Caster v1.0 relies on undefined behavior which can lead to random issues during execution, including Intel CPU.

One related issue is here because ([cite]) "Character movement is performed differently in different implementations. Thus overlapping moves may yield surprises." -> the outcome is unpredictable and in couple of runs I've got really weird result for my server_name config parameter (only!).

charlesquick commented 3 months ago

Hi, Thanks for the digging, and it does explain some weird behaviour I've seen when adding users and suddenly no one can log in. On your research did you come across a fix? Or a workaround? We're well into the weeds here, which isn't my speciality.

PaJaSoft commented 2 weeks ago

In file ntrip_string.c inside function "char splitc (char first, char *rest, const char divider)" calling "strcpy(rest, p+1);" replaced by "memmove(rest, p+1, strlen(p+1) +1);" fixes above issue for me (I'm over decade out of C programming, not sure if my approach is the best in this case, however WFM).

charlesquick commented 2 weeks ago

Added your changes to the ntripstring branch.

You may be interested to know that BKG have recently released their professional NTRIP caster free of charge https://igs.bkg.bund.de/ntrip/bkgcaster