invalid ntp_time < gst_audio_pipeline_base_time

GribouilleVert commented 10 months ago

Hi,

Source device: iPad Pro (iOS 17 beta 5) Destination device: Fedora 38 I've built and run successfully the binary.

The iPad has no issues connecting to the computer, but when playing, there is no sound or image and i get the following error in console:

*** invalid ntp_time < gst_audio_pipeline_base_time
9910.665274 ntp_time
1693253157.394135 base_time

(the numbers varies)

Do you have any idea where this could be coming from, and if it's a bug or a problem on my side.

- Vasco

fduncanh commented 10 months ago

Bonjour!

Can you test with iOS16.x?

Lets hope that iOS17 doesn't have bad changes.... UxPlay is well-tested (with iOS16 clients) on Fedora38

fduncanh commented 10 months ago

The error you see is because the audio-frame timestamp "ntp_time=9910.665274 seconds" has not correctly been converted to unix time (seconds since Jan 1, 1970), like the base_time 1693253157.394135 seconds. The timestamp should be later than the base time, evidently it is not!

Please verify that this is (or is not) an iOS17 issue.

Also svp post a debug output from "uxplay -d". (you may need to call it something like "debug.txt" to post it by dragging it to the issue window.

Below, data->ntp_time_remote is timestamp measured on the client, perhaps seconds since last client boot. data->ntp_time_local is its conversion to unix time. possibly the second one is corrupted in your example. We need to see why. The debug output may help.

extern "C" void audio_process (void *cls, raop_ntp_t *ntp, audio_decode_struct *data) {
    if (dump_audio) {
        dump_audio_to_file(data->data, data->data_len, (data->data)[0] & 0xf0);
    }
    if (use_audio) {
        if (!remote_clock_offset) {
            remote_clock_offset = data->ntp_time_local - data->ntp_time_remote;
        }
        data->ntp_time_remote = data->ntp_time_remote + remote_clock_offset;
        if (data->ct == 2 && audio_delay_alac) {
            data->ntp_time_remote = (uint64_t) ((int64_t) data->ntp_time_remote  + audio_delay_alac);
        } else if (audio_delay_aac) {
            data->ntp_time_remote = (uint64_t) ((int64_t) data->ntp_time_remote + audio_delay_aac);
        }
      audio_renderer_render_buffer(data->data, &(data->data_len), &(data->seqnum), &(data->ntp_time_remote));
    }
}

void audio_renderer_render_buffer(unsigned char* data, int *data_len, unsigned short *seqnum, uint64_t *ntp_time) {                                                                                                                                                                                                         
    GstBuffer *buffer;                                                                                                                                                                                                                                                                                                      
    bool valid;                                                                                                                                                                                                                                                                                                             

    if (!render_audio) return;    /* do nothing unless render_audio == TRUE */                                                                                                                                                                                                                                              

    GstClockTime pts = (GstClockTime) *ntp_time ;    /* now in nsecs */                                                                                                                                                                                                                                                     
    //GstClockTimeDiff latency = GST_CLOCK_DIFF(gst_element_get_current_clock_time (renderer->appsrc), pts);                                                                                                                                                                                                                
    if (pts >= gst_audio_pipeline_base_time) {                                                                                                                                                                                                                                                                              
        pts -= gst_audio_pipeline_base_time;                                                                                                                                                                                                                                                                                
    } else {                                                                                                                                                                                                                                                                                                                
        logger_log(logger, LOGGER_ERR, "*** invalid ntp_time < gst_audio_pipeline_base_time\n%8.6f ntp_time\n%8.6f base_time",                                                                                                                                                                                              
                   ((double) *ntp_time) / SECOND_IN_NSECS, ((double) gst_audio_pipeline_base_time) / SECOND_IN_NSECS);                                                                                                                                                                                                      
        return;                                                                                                                                                                                                                                                                                                             
    }

thiccaxe commented 10 months ago

on 21A5303d it works for me with both legacy pair bit 27 on and off

fduncanh commented 10 months ago

@thiccaxe is this iOS17 (or iPadOS17?)

thiccaxe commented 10 months ago

ios17 on iphone se 2

fduncanh commented 10 months ago

Ooof! .. thats a relief!

lets see what @GribouilleVert supplies.

thiccaxe commented 10 months ago

21A5303d is ios 17 beta 5, ios 17 beta 7 is out as well and I will check out that one soon. it could be that Gribouille is on the public beta 5 whereas I am on the developer beta 5, cannot tell without the build number

thiccaxe commented 10 months ago

I updated to the latest ios 17 developer beta and no issues.

thiccaxe commented 10 months ago

interestingly ios no longer shows the blue bar when airplay is active

the web drm stuff - Managed Media Source - seems to be the only change to airplay but that is airplay2 anyway

fduncanh commented 10 months ago

I'm installing public beta iPadOS17 beta5 now ....

fduncanh commented 10 months ago

@GribouilleVert

UxPlay-1.56.3 is now tested on Fedora38, with iPadOS 1.17 public beta5, and we don't see any problems,

please give more details of your issue, including the uxplay -d debug output

GribouilleVert commented 10 months ago

I will provide tht details as soon as i am home ! Thanks for the fast answers and insightfull comments :)

For more context my iPad is on the latest iPadOS 17 developper beta (21A5319a)

GribouilleVert commented 10 months ago

Hi ! Here a the logs, furthermore, after some testing, it seems that only the iPad has the issue, my iPhone doesn't have it as long as my ipad never connected.

My config file contains the following:

n "Vasco-Main"
nh
reset 0

log.txt

fduncanh commented 10 months ago

Looking at it now.

Please retry without "reset 0"

GribouilleVert commented 10 months ago

Here you go !

log.txt

fduncanh commented 10 months ago

Here is what I see in your output

Client identified as User-Agent: AirPlay/710.79.1
timing_rport = 60046
raop_ntp parse remote ip = 192.168.1.37
raop_ntp starting time
raop_ntp local timing port socket 51 port UDP 59847
raop_rtp parse remote ip = 192.168.1.37
raop_rtp_mirror parse remote ip = 192.168.1.37
eport = 37053, tport = 59847

RTSP/1.0 200 OK 
CSeq: 3 
Server: AirTunes/220.68 
Content-Type: application/x-apple-binary-plist 
Content-Length: 77 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>timingPort</key>
    <integer>59847</integer>
    <key>eventPort</key>
    <integer>37053</integer>
</dict>
</plist>

raop_ntp send time type_t=82 packetlen = 32, now = 1693335563.017768
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 be 8b 04 8c 6d 20 

Accepted IPv4 client on socket 52
Local: 192.168.1.85
Remote: 192.168.1.37
httpd receiving on socket 50
conn_request

This is what I see in the sample debug output (ipadOS 1.16.x client) in the UxPlay wiki:

Client identified as User-Agent: AirPlay/690.7.1
timing_rport = 59687
raop_ntp parse remote ip = 192.168.1.128
raop_ntp starting time
raop_ntp local timing port socket 25 port UDP 7011
raop_rtp parse remote ip = 192.168.1.128
raop_rtp_mirror parse remote ip = 192.168.1.128
eport = 7000, tport = 7011

RTSP/1.0 200 OK 
CSeq: 3 
Server: AirTunes/220.68 
Content-Type: application/x-apple-binary-plist 
Content-Length: 77 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>timingPort</key>
    <integer>7011</integer>
    <key>eventPort</key>
    <integer>7000</integer>
</dict>
</plist>

raop_ntp send time type_t=82 packetlen = 32, now = 1686755908.388390
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 34 58 c4 63 6d 7b f7 

raop_ntp receive time type_t=83 packetlen = 32, now = 1686755908.477676 t1 = 105495.240660, t2 = 105495.240807
80 d3 00 07 00 00 00 00 e8 34 58 c4 63 6d 7b f7 
83 ac 1a 97 3d 9b ec 7d 83 ac 1a 97 3d a5 8c a5 

raop_ntp sync correction = -1686650413192298815
Accepted IPv4 client on socket 26
Local: 192.168.1.6
Remote: 192.168.1.128
httpd receiving on socket 24
conn_request

Looks like you never got the client's reply for NTP synchronization...

fduncanh commented 10 months ago

@GribouilleVert You also don't have a video connection request, before the audio request.

What actually are you doing on the iPad? AHA! Looks like you are listing to ALAC audio in audio-only mode.

Please test the screen-mirroring mode instead.

I will test the audio mode with iPadOS 1.17 to see if anything changed or if anything is broken (I only tested mirror mode)

GribouilleVert commented 10 months ago

I in fact used audio-only mode on both the iPhone and iPad

Here is the video log on the ipad. video.txt

fduncanh commented 10 months ago

I checked: I have working Audio mode with iPadOS 1.17, and your video mode is same as audio mode: no recorded reply to the ntp request made to the client.

Why were you using the "reset 0" setting?

fduncanh commented 10 months ago

The -reset 0 setting switches off a check that should have terminated your session with an error message.

Please explicitly use "uxplay -reset 5" to enforce the default in case some startup file file still sets reset 0

Every 3 seconds the server sends an ntp request for a timing signal to the client, which must be received within 0.3 secs If (reset=) 5 consecutive request are ignored, the connection should be terminated as "dead".

fduncanh commented 10 months ago

At this point, I cannot understand your debug output: after the ntp time request is made,

raop_ntp send time type_t=82 packetlen = 32, now = 1693335563.017768
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 be 8b 04 8c 6d 20

in -d debug mode there should either be a message showing the response, OR a message that a timeout occurred. I dont see either in your output

I assume you compiled the latest UxPlay release from source (or are you using the RPM specfile?)

Perhaps the debug mode is not reporting all messages correctly?

We can provide a modified version of UxPlay to help debug your issue if necessary.

(lib/raop_ntp.c)


        // Send request
        uint64_t send_time = raop_ntp_get_local_time(raop_ntp);
        byteutils_put_ntp_timestamp(request, 24, send_time);
        int send_len = sendto(raop_ntp->tsock, (char *)request, sizeof(request), 0,
                              (struct sockaddr *) &raop_ntp->remote_saddr, raop_ntp->remote_saddr_len);
        if (logger_debug) {
            char *str = utils_data_to_string(request, sizeof(request), 16);
            logger_log(raop_ntp->logger, LOGGER_DEBUG, "\nraop_ntp send time type_t=%d packetlen = %d, now = %8.6f\n%s",
                       request[1] &~0x80, sizeof(request), (double) send_time / SECOND_IN_NSECS, str);
            free(str);
        }
        if (send_len < 0) {
            logger_log(raop_ntp->logger, LOGGER_ERR, "raop_ntp error sending request");
        } else {
            // Read response
            response_len = recvfrom(raop_ntp->tsock, (char *)response, sizeof(response), 0,
                                    (struct sockaddr *) &raop_ntp->remote_saddr, &raop_ntp->remote_saddr_len);
            if (response_len < 0) {
                timeout_counter++;
                char time[30];
                int level = (timeout_counter == 1 ? LOGGER_DEBUG : LOGGER_ERR);
                ntp_timestamp_to_time(send_time, time, sizeof(time));
                logger_log(raop_ntp->logger, level, "raop_ntp receive timeout %d (limit %d) (request sent %s)",
                           timeout_counter, raop_ntp->max_ntp_timeouts, time);
                if (timeout_counter ==  raop_ntp->max_ntp_timeouts) {
                    conn_reset = true;   /* client is no longer responding */
                    break;
                }
        } else {
                //local time of the server when the NTP response packet returns
                int64_t t3 = (int64_t) raop_ntp_get_local_time(raop_ntp);
                timeout_counter = 0;

                // Local time of the server when the NTP request packet leaves the server
                int64_t t0 = (int64_t) byteutils_get_ntp_timestamp(response, 8);

                // Local time of the client when the NTP request packet arrives at the client
                int64_t t1 = (int64_t) raop_remote_timestamp_to_nano_seconds(raop_ntp, byteutils_get_long_be(response, 16));

                // Local time of the client when the response message leaves the client
                int64_t t2 = (int64_t) raop_remote_timestamp_to_nano_seconds(raop_ntp, byteutils_get_long_be(response, 24));

                if (logger_debug) {
                    char *str = utils_data_to_string(response, response_len, 16);                   
                    logger_log(raop_ntp->logger, LOGGER_DEBUG,
                               "raop_ntp receive time type_t=%d packetlen = %d, now = %8.6f t1 = %8.6f, t2 = %8.6f\n%s",
                               response[1] &~0x80, response_len, (double) t3 / SECOND_IN_NSECS, (double) t1 / SECOND_IN_NSECS,
                               (double) t2 / SECOND_IN_NSECS, str); 
                    free(str);
                }

fduncanh commented 10 months ago

@GribouilleVert

I modified UxPlay's lib/raop_ntp.c in a test version for you in a branch "Grib" of UxPlay on github.

Please compile and run that (you dont need the -d option) and post the terminal output.

That may help me see what is actually happening, as I don't understand why some debug messages are not showing in your debug output

This is what a working output would be:

%uxplay -p

UxPlay 1.65: An Open-Source AirPlay mirroring and audio-streaming server.
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address 73:e3:5b:ec:be:88
Initialized server socket(s)
Accepted IPv4 client on socket 24
Local: 192.168.1.6
Remote: 192.168.1.42
Client identified as User-Agent: AirPlay/710.79.1

raop_ntp send time type_t=82 packetlen = 32, now = 1693345853.019784
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 e6 bd 05 10 8b 34 

send_len= 32
waiting for client response
client response_len = 32
raop_ntp receive time type_t=83 packetlen = 32, now = 1693345853.023218 t1 = 56162.717956, t2 = 56162.718057
80 d3 00 07 00 00 00 00 e8 98 e6 bd 05 10 8b 34 
83 ab 59 e2 b7 cb f8 fc 83 ab 59 e2 b7 d2 93 fd 

raop_ntp sync correction = -1693289690303494297
Accepted IPv4 client on socket 26
Local: 192.168.2.6
Remote: 192.168.2.242
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline

*** X11 Windows: Use key F11 or (left Alt)+Enter to toggle full-screen mode

raop_ntp send time type_t=82 packetlen = 32, now = 1693345856.023332
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 e6 c0 05 f9 14 ba 

send_len= 32
waiting for client response
client response_len = 32
raop_ntp receive time type_t=83 packetlen = 32, now = 1693345856.025715 t1 = 56165.721031, t2 = 56165.721087
80 d3 00 07 00 00 00 00 e8 98 e6 c0 05 f9 14 ba 
83 ab 59 e5 b8 95 85 37 83 ab 59 e5 b8 99 26 41 

raop_ntp sync correction = 30080
ct=8 spf=480 usingScreen=1 isMedia=1  audioFormat=0x1000000
start audio connection, format AAC-ELD 44100/2
raop_rtp starting audio

raop_ntp send time type_t=82 packetlen = 32, now = 1693345859.025836
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 e6 c3 06 9d 30 00 

send_len= 32
waiting for client response
client response_len = 32
raop_ntp receive time type_t=83 packetlen = 32, now = 1693345859.027984 t1 = 56168.723294, t2 = 56168.723391
80 d3 00 07 00 00 00 00 e8 98 e6 c3 06 9d 30 00 
83 ab 59 e8 b9 29 cd 11 83 ab 59 e8 b9 30 2a 8e 

raop_ntp sync correction = -103189

raop_ntp send time type_t=82 packetlen = 32, now = 1693345862.028104
80 d2 00 07 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 e8 98 e6 c6 07 31 cb dd 

send_len= 32
waiting for client response
client response_len = 32
raop_ntp receive time type_t=83 packetlen = 32, now = 1693345862.030538 t1 = 56171.725999, t2 = 56171.726048
80 d3 00 07 00 00 00 00 e8 98 e6 c6 07 31 cb dd 
83 ab 59 eb b9 db 0b 0c 83 ab 59 eb b9 de 4f 1c 

raop_ntp sync correction = 0

^CStopping...

GribouilleVert commented 10 months ago

I will do it tomorrow morning, thank you very much for the help, i'll keep you updated

- Vasco

fduncanh commented 10 months ago

My guess is that perhaps it will turn out that recvfrom just never returns

response_len = recvfrom(raop_ntp->tsock, (char *)response, sizeof(response), 0, (struct sockaddr *) &raop_ntp->remote_saddr, &raop_ntp->remote_saddr_len);

EDIT: but the socket has a 300000us (0.3 second) timeout, so I don't see how recvfrom could fail to return. EDIT: the last two arguments could be replaced with NULL, NULL since we dont do anything with the returned remote_saddr

fduncanh commented 10 months ago

Updated iPad to newly-available iPadOS 1.17 public beta 6.

No Uxplay problems seen.

GribouilleVert commented 10 months ago

Hi !

I've upgraded my iPad to IOS17.0 beta 8. The following logs are from a binary that i've built directly from the Grib branch, i also explicitly provided -reset 5 when running the binary.

Airplay with screen mirroring: video.txt Airplay with audio only: audio.txt

- Vasco

fduncanh commented 10 months ago

Bonjour @GribouilleVert !

OK this time I am seeing the ntp timeouts. In your audio debug you get to the 3rd timeout (the system would reset after reset=5 timeouts). (I also looked back at your earlier debug traces, and did finally see timeouts)

Please try again with the latest "Grib" version, it reports a bit more about the client, (but probably wont help, since recvfrom returns -1, presumably because the timeout after 0.3 secs happened)

The UxPlay server is sending ntp time-signal requests to the client every 3 seconds, and waits for 0.3 seconds for response before a time out. There are 3 udp ports involved in audio, the timing port, the control port and the data port.

data is certainly arriving on the audio udp data port.

you can use the uxplay -p option to fix the network ports on the server. with only -p, they are udp 7011 timing port udp 6001 audio control (resend requests) udp 6000 audio data tcp 7100 video data tcp 7000 raop and "event" port tcp 7001 airplay port (not used in legacy mode)

to diagnose your issue either the timing requests are not arriving at the client, or the client is ignoring them, or the client replies are not getting to the UxPlay server.

Your client did report it was using the NTP protocol (not a newer one that doesnt send NTP signals)

SETUP rtsp://192.168.1.85/5370894068230474712 RTSP/1.0
Content-Length: 575
Content-Type: application/x-apple-binary-plist
CSeq: 3
DACP-ID: 6D709593D8698102
Active-Remote: 1357498066
User-Agent: AirPlay/710.79.1

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>et</key>
    <integer>32</integer>
    <key>eiv</key>
    <data>
    rYT76CTTmmXI47BhKuXKQA==
    </data>
    <key>statsCollectionEnabled</key>
    <false/>
    <key>sessionUUID</key>
    <string>4A893FB2-E721-4BD8-BA1E-493098E2C84C</string>
    <key>timingProtocol</key>
    <string>NTP</string>                       <========================================HERE
    <key>diagnosticsAndUsage</key>
    <true/>
    <key>osName</key>
    <string>iPhone OS</string>
    <key>osBuildVersion</key>
    <string>21A5319a</string>
    <key>sourceVersion</key>
    <string>710.79.1</string>
    <key>timingPort</key>
    <integer>60046</integer>   <=====================client timing port
    <key>osVersion</key>      
    <string>17.0</string>
    <key>ekey</key>on 21A5303d it works for me
    <data>
    RlBMWQECAQAAAAA8AAAAAMvZXEyj0luCDPWFDkvlD7AAAAAQH24GyZPal7S0Hlybbd/d
    kov5CVSNTdOjJW53a+bVLaJfZOzM
    </data>
    <key>internalBuild</key>
    <false/>
    <key>deviceID</key>
    <string>2C:18:09:02:09:7E</string>
    <key>model</key>
    <string>iPad14,5</string>
    <key>name</key>
    <string>Vasco-Tablet</string>
    <key>macAddress</key>
    <string>A6:64:93:03:4E:E7</string>
</dict>
</plist>

So something is "wrong" with your system or network, and it can't be reproduced on iPadOS 1.17 public beta 6. @thiccaxe has tested on a 1.17 developer beta, quote "on 21A5303d it works for me"

At this point a tool like netstat would be needed to listen for activity of the UDP timing ports of client and server.

The -d output shows them:

Client identified as User-Agent: AirPlay/710.79.1
timing_rport = 60046
raop_ntp parse remote ip = 192.168.1.37
raop_ntp starting time
raop_ntp local timing port socket 51 port UDP 59847
raop_rtp parse remote ip = 192.168.1.37
raop_rtp_mirror parse remote ip = 192.168.1.37
eport = 37053, tport = 59847

fduncanh commented 10 months ago

UxPlay 1.65 also works fine with a client iPad Pro (M2, 2023) running iPadOS 16.6 This shows as iPad14,5 like @GribouilleVert 's iPad so may be the same model.

There was a strange point made earlier by @GribouilleVert

Hi ! Here a the logs, furthermore, after some testing, it seems that only the iPad has the issue, my iPhone doesn't have it as long as my ipad never connected.

Are you saying that if you start UxPlay, connect with the iPad, disconnect without stopping UxPlay, then connect with the iphone, the iphone connection is messed up, but not if the iphone connects first?.

What happens if you connect first with the iphone, then the iPad ?

EDIT: looking at the code, it seems that all the timing stuff is re-initialized when there is a new connection.

thiccaxe commented 10 months ago

I think something has definitely changed with the ipad of GribouilleVert and even my ios 17 dev beta 7 iphone, you can see on the dev beta 7,

note that both are cseq 7 with supports legacy pairing off

SETUP
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>streams</key>
        <array>
                <dict>
                        <key>timestampInfo</key>
                        <array>
                                <dict>
                                        <key>name</key>
                                        <string>SubSu</string>
                                </dict>
                                <dict>
                                        <key>name</key>
                                        <string>BePxT</string>
                                </dict>
                                <dict>
                                        <key>name</key>
                                        <string>AfPxT</string>
                                </dict>
                                <dict>
                                        <key>name</key>
                                        <string>BefEn</string>
                                </dict>
                                <dict>
                                        <key>name</key>
                                        <string>EmEnc</string>
                                </dict>
                        </array>
                        <key>latencyMs</key>
                        <integer>100</integer>
                        <key>type</key>
                        <integer>110</integer>
                        <key>streamConnectionID</key>
                        <integer>2307174609987456411</integer>
                </dict>
        </array>
</dict>
</plist>

versus the log file GribouilleVert shared:

SETUP

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>streams</key>
    <array>
        <dict>
            <key>latencyMax</key>
            <integer>88200</integer>
            <key>sr</key>
            <integer>44100</integer>
            <key>audioMode</key>
            <string>default</string>
            <key>latencyMin</key>
            <integer>11025</integer>
            <key>ct</key>
            <integer>2</integer>
            <key>spf</key>
            <integer>352</integer>
            <key>controlPort</key>
            <integer>55730</integer>
            <key>supportsDynamicStreamID</key>
            <true/>
            <key>audioFormat</key>
            <integer>262144</integer>
            <key>isMedia</key>
            <true/>
            <key>type</key>
            <integer>96</integer>
        </dict>
    </array>
</dict>
</plist>

not sure what exactly has changed but this could be messing with the timing.

thiccaxe commented 10 months ago

What I mean to say is that the data with the "timestampInfo" is missing from the GribouilleVert log. But on code inspection it seems that the "timestampInfo" stuff is not used anyway, so maybe it is something else.

fduncanh commented 10 months ago

@thiccaxe some of @GribouilleVert logs are audio-only (ALAC) music with only a type 96 setup. In Mirror mode, the type 110 video setup comes before the type 96 and the CSeq numbers change.

The examples you give are (i) type 110 (ii) type 96, so cant be compared.

EDIT: I compared @GribouilleVert 's mirror-mode iPadODE 1.7 beta dev debug trace carefully with the 1.65 example in the Wiki.

There are no significant differences I could see, except that the ntp timing requests time out 0.3 secs after being sent by the server, with no response from the client (I eventually found the error message for this in @GribouilleVert 's debug: 0.3secs is a lot of debug messages later))
there doesnt seem to be any firewall on his system, and the other two udp ports seem to be working, so its a mystery. Perhaps being on a development release has broken something on @GribouilleVert 's iPad OS ?

@thiccaxe what do you mean by "missing timestampinfo"? (the latest @GribouilleVert outputs are not debug traces, but use a modified UxPlay I provided in a "Grib" branch)

GribouilleVert commented 10 months ago

Hello !

Are you saying that if you start UxPlay, connect with the iPad, disconnect without stopping UxPlay, then connect with the iphone, the iphone connection is messed up, but not if the iphone connects first?.

No, after more testing, it seems to have been a one off, for more infos: my iPad is 2023 M2 iPad Pro 12.9in (Model Num. MNXP3NF/A), the current OS version is iPadOS 17.0 developpera beta 8 (21A5326a).

I seem to have found the source of my issue (altough i've no idea what's causing it), since you hinted that it may be my network, i tried to put my computer and iPad on my iPhone network share, and oh surpise, it worked. In my home i have two wifi access points that are on the same network, one from my freebox (my isp internet box) and one that i've plugged in on of my switches (which are connected back to my isp box), and the external one seem to do something that drop some info, it should be noted that since it's an old device and is using WPA2 TKIP instead of WPA2 AES, iPadOs and iOS show "Less secure" under the network's name, so maybe that's it.

Tell me if you want me to do any more test, you can also write me on discord if you want to do a hands-on: @vasco_

Thanks a lot for the help and time.

\ - Vasco

fduncanh commented 10 months ago

OK! @GribouilleVert So you have it working now? (Please confirm no issues are remaining, and close this issue if so).

Somewhere the UDP timing channel communications between the UxPlay Server and the client were being messed up, maybe because of two competing access points (?)

Thank you for reporting your issue. It pushed me into testing the iPadOS 1.17 public beta, and I confirmed that the older protocol used by UxPlay is still supported.
It also got me to look at the NTP time signal code, and I saw that the infrastructure for IPV6 socket addresses was not quite finished, so I fixed that (though wasn't able to get my network to hand out IPV6 rather than IPV4 addresses using DHCPv6, so I can't yet test), plus a few other code cleanups that I noticed. (The codebase of the ancestors of UxPlay are on GitHub, so it was useful to see where and when various code features originated.)

Merci!

GribouilleVert commented 10 months ago

Hi !

There are indeed no more issues.

Somewhere the UDP timing channel communications between the UxPlay Server and the client were being messed up, maybe because of two competing access points (?)

I beleive this is due to the interal routing of the access point, since it's quite old and has a lot of "enterprise features" that i'm not quite sure how they work.

Have a nice day.

- Vasco

FDH2 / UxPlay

invalid ntp_time < gst_audio_pipeline_base_time #206