why the streaming content-type is wrong ?

[philippedc]

Hi all, it is not an issue of the library, but I do not know where to find documentation to solve the issue. Most of small local radios use redirect url, for instance: 1- Radio Kankan (it is a cityname of Guinea West Africa): http://stream.zeno.fm//ab9q4v3mb The console gives:

Request access to: Radio KANKAN Request access to: stream.zeno.fm:80/ab9q4v3mb Try to reconnect... -->HTTP/1.1 302 -->location: http://stream.zenolive.com/ab9q4v3mb?zs=3rXgc_2kRwuG_ScHzZfXSQ new url fields = stream.zenolive.com + /ab9q4v3mb?zs=3rXgc_2kRwuG_ScHzZfXSQ -->Access-Control-Allow-Origin: * -->Location: http://node-29.zeno.fm/ab9q4v3mb?zs=3rXgc_2kRwuG_ScHzZfXSQ&rj-tok=AAABf7urGXYASuMmCX_O1AEP9A&rj-ttl=5 new url fields = node-29.zeno.fm + /ab9q4v3mb?zs=3rXgc_2kRwuG_ScHzZfXSQ&rj-tok=AAABf7urGXYASuMmCX_O1AEP9A&rj-ttl=5

@@>content-type: audio/mpeg icy-name: -

There are several redirection, however after all the content-type: audio/mpeg appears and the station works fine

2- Kassara Barikama (Bamako Mali): http://stream.zeno.fm/u5u4gvaxh4duv The console gives:

Request access to: Kassara Barikama Request access to: stream.zeno.fm:80/u5u4gvaxh4duv Try to reconnect... -->HTTP/1.1 302 -->location: http://stream.zenolive.com/u5u4gvaxh4duv?zs=xEwnpDv8SV2bzd5D7yTBqw new url fields = stream.zenolive.com + u5u4gvaxh4duv?zs=xEwnpDv8SV2bzd5D7yTBqw -->Access-Control-Allow-Origin: * -->Location: http://node-35.zeno.fm/u5u4gvaxh4duv?zs=xEwnpDv8SV2bzd5D7yTBqw&rj-tok=AAABf7uvXJcAUn8ykeWJtSNSQQ&rj-ttl=5 new url fields = node-35.zeno.fm + /u5u4gvaxh4duv?zs=xEwnpDv8SV2bzd5D7yTBqw&rj-tok=AAABf7uvXJcAUn8ykeWJtSNSQQ&rj-ttl=5 -->location: http://stream.zeno.fm/y8sgkhhra3quv?zs=xEwnpDv8SV2bzd5D7yTBqw new url fields = stream.zeno.fm + /y8sgkhhra3quv?zs=xEwnpDv8SV2bzd5D7yTBqw @@>Content-length: 90 Cache-Control: no-cache Connection: close Content-Type: text/html

400 Bad request

Try to reconnect... Try to reconnect... This station - and many others - return a bad content-type. But both stations works fine with a PC and modzilla. So something is missing in the request to access to the url because there is reason it does not work on the ESP8266 webradio example Here is the code: // si perte du signal if( !client.connected() ) { Serial.println("Try to reconnect..."); headerSearch = true; bufferNumber = 0; if( client.connect(urlStation, portStation.toInt()) ) { client.print(String("GET ") + pathStation + " HTTP/1.1\r\n" + "Host: " + urlStation + "\r\n" + "Connection: close\r\n\r\n"); } // end of client.connect test } // end of !client.connected() test // si tout va bien :) if( client.available() && !pause ) { // look foward url rediction type 302 if( headerSearch ) { headerSearch = false; String header = client.readStringUntil('\n'); // read the header until \n Serial.print("-->"); Serial.println(header); while((header.indexOf("HTTP/1.0 302") >= 0)||(header.indexOf("HTTP/1.1 302") >= 0)) { // it may have several while((header.indexOf("Location:") < 0) && (header.indexOf("location:") < 0)) { // new url to reach header = client.readStringUntil('\n'); Serial.print("-->"); Serial.println(header); } // get fields separated by "/" // Example=> Location: http://node-28.zeno.fm/20mdfd30wqzuv?rj-ttl=5&rj-tok=AAABfhD5HCcAg9QOMJ5yH8jRjA String newPortStation = "80"; String newUrlStation = ""; String newPathStation = "/"; if( header[14] == 's' ) newPortStation = "443"; byte index = 0; while( header[index] != '/' ) index ++; index +=2; while( header[index] != '/' ) { newUrlStation += header[index]; index ++; } index ++; for( byte k=index ; k < header.length() ; k++ ) newPathStation += header[k]; Serial.print("new url fields = "); Serial.print(newUrlStation); Serial.print(" + "); Serial.println(newPathStation); if( client.connect(newUrlStation, newPortStation.toInt()) ) { client.print(String("GET ") + newPathStation + " HTTP/1.1\r\n" + "Host: " + urlStation + "\r\n" + "Connection: close\r\n\r\n"); } // end of client.connect test header = client.readStringUntil('\n'); // read the header until \n } // end of while on header } // end of test on headerSearch if( ++bufferNumber < 30 ) { Serial.print("@@>"); client.read(mp3buff, buffSize); for( int i=0; i

[Dr-Dawg]

I haven't tried the code, but could it be an issue of cutting of the new path too early? Maybe some String does not provide the correct length?

My first idea would be to change

for( byte k=index ; ...

into

for( int k=index ; ...

[Dr-Dawg]

..or -another blind guess- could it be that the String keeps data over the redirect, so that Content-Type: text/html belongs to the old header of the redirecting webpage?

[philippedc]

I've updated the code, for a better view of the header to the console, and to adapt either HTTP/1.0 or HTTP/1.1 depending of the header received... no change. However, what I've noticed from the attached screenshot and the console message from my first message:

1- there is a client.connect/client.print to the radio station, 2- a first 302 answer gives a 2nd url - the one ending by "9tGOg" 3- the second url answers the 302 again for a 3rd url 4- the 3rd url answers the 302 again for a 4th url, that is identical to the 2nd 5- to connect again to the 2nd url answers a bad request. Something is missing.

What is surprising me, the same radio provider can deliver either radios in streaming which works fine after on or 2 url redirect, and radios which need something more to connect correctly....

[philippedc]

It is not a problem of the lengh of the path:

/ab9q4v3mb?zs=Zk2aWhTHQFCOwWNJLnwobg&rj-tok=AAABf70H5hsAwNC2BwJhjknfGA&rj-ttl=5
/62v1f6u70mzuv?zs=zu04ijnTS3K72ZAcI3pHsA&rj-tok=AAABf70QxboATmSDtJXPtttZEQ&rj-ttl=5
/u5u4gvaxh4duv?zs=psfJsXxGSyqvedvKya4TKA&rj-tok=AAABf70JTdoAAeK8dAVHJT2ZTQ&rj-ttl=5
/csp5pw3stvduv?zs=vGoivvEMQ8ea8J6n447K9Q&rj-tok=AAABf70KKZ0AKdvsyVd3E1ryBA&rj-ttl=5

the 2 first temporary paths was working, not the 2 last.

[Dr-Dawg]

I'll test the code and double check the station with my own redirect implementation .. this may take a few days ;-)

[Dr-Dawg]

After some trouble, I was able to connect with the station using my redirect implementation with the feedback given below.

Looks like there are 5 redirects before anything happens. I now do suspect the problems with this station could be a matter of the station expecting cookies?

----------------------------------------------------------
---------------Kassara Barikama---------------
----------------------------------------------------------
Connecting to stream.zeno.fm
Requesting stream: /u5u4gvaxh4duv
buffering
HTTP/1.1 302 
location: http://stream.zenolive.com/u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ
access-control-allow-origin: *
cache-control: no-cache
content-length: 0
date: Fri, 25 Mar 2022 16:22:08 GMT
connection: close

Ende des Headers
REDIRECT, Host: stream.zenolive.com
Path: /u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ
----------------------------------------------------------
Connecting to stream.zenolive.com
Requesting stream: /u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ
HTTP/1.0 302 Found
Access-Control-Allow-Origin: *
Location: http://node-07.zeno.fm/u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrySoAdZKVNKBDtjQxVQ&rj-ttl=5
Content-Length: 0
Connection: close
Set-Cookie: rj-listener-cookie=9cqasw5q81uf; Domain=zeno.fm

Ende des Headers
REDIRECT, Host: node-07.zeno.fm
Path: /u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrySoAdZKVNKBDtjQxVQ&rj-ttl=5
----------------------------------------------------------
Connecting to node-07.zeno.fm
Requesting stream: /u5u4gvaxh4duv?zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrySoAdZKVNKBDtjQxVQ&rj-ttl=5
HTTP/1.0 302 Found
location: http://stream.zeno.fm/y8sgkhhra3quv?zs=h-zghE-PTPyxhEdwvrepPQ

Ende des Headers
REDIRECT, Host: stream.zeno.fm
Path: /y8sgkhhra3quv?zs=h-zghE-PTPyxhEdwvrepPQ
----------------------------------------------------------
Connecting to stream.zeno.fm
Requesting stream: /y8sgkhhra3quv?zs=h-zghE-PTPyxhEdwvrepPQ
HTTP/1.1 302 
location: http://stream.zenolive.com/y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ
access-control-allow-origin: *
cache-control: no-cache
content-length: 0
date: Fri, 25 Mar 2022 16:22:09 GMT
connection: close

Ende des Headers
REDIRECT, Host: stream.zenolive.com
Path: /y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ
----------------------------------------------------------
Connecting to stream.zenolive.com
Requesting stream: /y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ
HTTP/1.0 302 Found
Access-Control-Allow-Origin: *
Location: http://node-11.zeno.fm/y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrzQwAJ9dYbSKyLBZXlA&rj-ttl=5
Content-Length: 0
Connection: close
Set-Cookie: rj-listener-cookie=o6cpxoju1kx; Domain=zeno.fm

Ende des Headers
REDIRECT, Host: node-11.zeno.fm
Path: /y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrzQwAJ9dYbSKyLBZXlA&rj-ttl=5
----------------------------------------------------------
Connecting to node-11.zeno.fm
Requesting stream: /y8sgkhhra3quv?zs=EyXx4VCwRIWZ1KP8VyYWYw&zs=h-zghE-PTPyxhEdwvrepPQ&rj-tok=AAABf8HrzQwAJ9dYbSKyLBZXlA&rj-ttl=5
HTTP/1.1 200 OK
content-type: audio/mpeg
icy-name: -

Ende des Headers
SyncBytefolge: 255, 251, 144, 68
MPEG 1 Layer 3, Bitrate 128, Sample Frequency 44100, Pad Bit 1, Frame Length 417

[philippedc]

is it the right way to declare a cookie?

       if( client.connect(newUrlStation, newPortStation.toInt()) ) {
          client.print(String("GET ") + newPathStation + " " + protocol + "\r\n" +
                  "Host: " + newUrlStation + "\r\n" + 
                  "Cookie: " + cookie + "\r\n" +
                  "User-Agent: Mozilla/5.0\r\n" +
                  "Connection: close\r\n\r\n");

with

    if( header.indexOf("Set-Cookie:") >= 0) {
            for( int k=12; k < header.length(); k++ ) cookie += header[k];
            Serial.print("Cookie found: "); Serial.println(cookie);
      }

because this does not solve the "HTTP/1.1 400 Bad request" answer from the server

[Dr-Dawg]

I have no idea (yet). Actually I never worked with cookies. Hence I'm as curious about the outcome..

Anyway, I'm going to test the code you posted in the first place. After three tries with my code it's always that the 5th redirect does work, so it should do the same with your code, even without cookies.

[philippedc]

hi @Dr-Dawg here is the code. Thanks

// si perte du signal
  if((!client.connected()) || stationChange ) {
    Serial.print(F("Try to connect to: ")); Serial.println(nomStation);
    Serial.print(F("http://")); Serial.print(urlStation); Serial.print(":"); 
    Serial.print(portStation.toInt()); Serial.print("/"); Serial.println(pathStation);
    Serial.print(F("with the cookie: ")); Serial.println(cookie);
    Serial.println(F("-----------------------------"));

    stationChange = false;
    headerSearch = true;
    bufferNumber = 0;
    if( client.connect(urlStation, portStation.toInt()) ) {
      client.print(String("GET ") + pathStation + " " + protocol +"\r\n" +
                  "Host: " + urlStation + "\r\n" +
                  "Cookie: " + cookie + "\r\n" +
                  "User-Agent: Mozilla/5.0\r\n" +
                  "Connection: close\r\n\r\n");
    }  // end of client.connect() test
  }    // end of !client.connected() test

// si tout va bien :)
  if( client.available() && !pause ) {

    // look after url redirect type 302, it may have several occurancy
    if( headerSearch ) {
      headerSearch = false;
      String header = client.readStringUntil('\n');   // read the header until \n
      String newLocation = "";
      String cookie = "rj-listener-cookie=160e020z6m36v; Domain=zeno.fm";
      Serial.print(F("1-->")); Serial.println(header);
      while((header.indexOf("HTTP/1.0 302") >= 0) || (header.indexOf("HTTP/1.1 302") >= 0)) {  // redirect url
        byte i=1;
        do {
          header = client.readStringUntil('\n');
          Serial.print(++i); Serial.print(F("-->")); Serial.println(header);
          if((header.indexOf("Location:") >= 0) || (header.indexOf("location:") >= 0)) {      // look after new url
            newLocation = header;
            Serial.print(F("Redirect url found: ")); Serial.println(newLocation);
          }
          else if( header.indexOf("Set-Cookie:") >= 0) {
            for( int k=12; k < header.length(); k++ ) cookie += header[k];
            Serial.print(F("Cookie found: ")); Serial.println(cookie);
          }
        } while( header.indexOf("") >= 0 );  // wait for the client.readStringUntill timeout

    // url fields search with "/" separator
    // example: Location: http://node-28.zeno.fm/20mdfd30wqzuv?rj-ttl=5&rj-tok=AAABfhD5HCcAg9QOMJ5yH8jRjA
        String newPortStation = "80";
        String newUrlStation  = "";
        String newPathStation = "/";
        if( newLocation[14] == 's' ) newPortStation = "443";
        int index = 0;
        while( newLocation[index] != '/' ) index ++;
        index +=2;
        while( newLocation[index] != '/' ) {
          newUrlStation += newLocation[index];
          index ++;
        }
        index ++;
        for( int k=index; k < newLocation.length(); k++ ) newPathStation += newLocation[k];

    // connect to new url
        Serial.print(F("\nRedirect to new location: ")); Serial.println(nomStation);
        Serial.print(F("http://")); Serial.print(urlStation); Serial.print(":"); 
        Serial.print(portStation.toInt()); Serial.print("/"); Serial.println(pathStation);
        Serial.print(F("with the cookie: ")); Serial.println(cookie);
        Serial.println(F("-----------------------------"));

        if( client.connect(newUrlStation, newPortStation.toInt()) ) {
          client.print(String("GET ") + newPathStation + " " + protocol + "\r\n" +
                  "Host: " + newUrlStation + "\r\n" + 
                  "Cookie: " + cookie + "\r\n" +
                  "User-Agent: Mozilla/5.0\r\n" +
                  //Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:98.0) Gecko/20100101 Firefox/98.0
                  "Connection: close\r\n\r\n");

        }    // end of client.connect test
        header = client.readStringUntil('\n');   // read the header until \n
        Serial.print(F("1-->")); Serial.println(header);
      }      // end of while on header HTTP
    }        // end of test on headerSearch

    if( ++bufferNumber < 5 ) {
      Serial.print(F("@@@>"));
      client.read(mp3buff, buffSize);
      for( int i=0; i<buffSize; i++ ) Serial.write( mp3buff[i] );
      Serial.println();
    }
  }      // end of client.available test

[philippedc]

I've tried to trace HTTP dialog with https://www.rexswain.com/httpview.html but I have not notice anything special.

With Modzilla Firefox, if I delete all cookies and run one "non-working" radio station from a private windows, I first have a timeout error message. Only the very first time. Unfortunately I cannot reproduce this error with the tool above, it is working each time

[Dr-Dawg]

hmm, would be easier with the definitions, I take, it's something like

char *nomStation = "Ankerherz";
char *urlStation = "217.160.184.16";
char *pathStation = "/Ankerherz";
String portStation = "8001";
String cookie = "";
char *protocol = " HTTP/1.1";
int buffsize = 64;
bool stationChange = true;
bool headerSearch = false;
int bufferNumber;

Still, I receive some compiling errors starting with

RedirectWebRadioDemo:220:65: error: no matching function for call to 'WiFiClient::connect(String&, long int)'
         if( client.connect(newUrlStation, newPortStation.toInt()) ) {

By the way, it seems that you are definig a String cookie in if( client.available() && !pause ) while using a String cookie outside the if statement. Note that the compiler handles this as two different variables, I'm not sure whether this was your intention.

[philippedc]

Hi @Dr-Dawg you're right. Here is the full test code for a .ino file:

byte VOLUME = 75;       
#define buffSize 128           // audio streams buffer size
String protocol = "HTTP/1.0";  // default http protocol

// librairies
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <DNSServer.h>     // requis pour WifiManager.h
#include "VS1053.h"        // https://github.com/baldram/ESP_VS1053_Library 
#include <WiFiManager.h>   // https://github.com/tzapu/WiFiManager

#define VS1053_CS     D0
#define VS1053_DCS    D3 
#define VS1053_DREQ   D4
byte Tone[4] = { 10, 10, 7, 2 };
VS1053 player(VS1053_CS, VS1053_DCS, VS1053_DREQ);

WiFiClient client; 

String nomStation  = "Kassara Barikama";
String portStation = "80";
String urlStation  = "stream.zeno.fm";
//String pathStation = "/ab9q4v3mb";     // OK
String pathStation = "/u5u4gvaxh4duv"; // non OK
String cookie = "";
bool stationChange = true;

bool pause = false;     // bouton de pause 
uint8_t mp3buff[buffSize];
bool headerSearch = false;
int bufferNumber = 0;

//
// SETUP
//_____________________________________________________________________________________________

void setup () {
  delay(1000);
  Serial.begin(250000);
  Serial.println(F("\n\nWiFi Radio is starting up :)"));

// connect to wifi
  WiFiManager monwifi;
  if(!monwifi.autoConnect("AutoConnectAP")) Serial.println(F("Wifi to setup"));
  else {
// Connect to Wi-Fi network with SSID and password
    Serial.print(F("connecting to wifi "));               
    while(WiFi.status() != WL_CONNECTED) Serial.print(".");
  }

  SPI.begin();
  player.begin();
  player.switchToMp3Mode();
  player.setVolume(VOLUME);  
  player.setTone(Tone);

  // check if chip is really a VS1053 for the firmware update
  if(player.getChipVersion() == 4) {
    player.loadDefaultVs1053Patches();  // Only perform an update if we really are using a VS1053, not. eg. VS1003
    Serial.println(F("\nChip is a VS1053 - so aac can be decoded\n"));
  }
  else Serial.println(F("\nChip is a VS1003 - aac format decoder unavailable\n"));

// set WiFiClient client.read function timeout
  client.setTimeout(500);          // default is 1s

}    // end of setup

//
// LOOP
//____________________________________________________________________________________________

void loop() {

  if((!client.connected()) || stationChange ) {
    stationChange = false;
    headerSearch = true;
    bufferNumber = 0;

    Serial.print(F("\nTry to connect: ")); Serial.println(nomStation);
    Serial.print(F("http://")); Serial.print(urlStation); Serial.print(":"); 
    Serial.print(portStation.toInt()); Serial.println(pathStation);
    Serial.println(F("-----------------------------"));

    if( client.connect(urlStation, portStation.toInt()) ) {
      client.print(String("GET ") + pathStation + " " + protocol +"\r\n" +
                  "Host: " + urlStation + "\r\n" +
                  "Connection: close\r\n\r\n");
    }  // end of client.connect() test
  }    // end of !client.connected() test

// si tout va bien :)
  if( client.available() && !pause ) {

    // look after url redirect type 302, it may have several occurancy
    if( headerSearch ) {
      headerSearch = false;
      String header = client.readStringUntil('\n');   // read the header until \n
      String newLocation = "";
      Serial.print(F("1-->")); Serial.println(header);

      while((header.indexOf("HTTP/1.0 302") >= 0) || (header.indexOf("HTTP/1.1 302") >= 0)) {  // redirect url
        byte i=1;
        do {
          header = client.readStringUntil('\n');
          Serial.print(++i); Serial.print(F("-->")); Serial.println(header);
          if((header.indexOf("Location:") >= 0) || (header.indexOf("location:") >= 0)) {      // look after new url
            newLocation = header;
          }
          else if( header.indexOf("Set-Cookie:") >= 0) {
            for( int k=12; k < header.length(); k++ ) cookie += header[k];
          }
        } while( header.indexOf("") >= 0 );  // wait for the client.readStringUntill timeout

    // url fields search with "/" separator
        String newPortStation = "80";
        String newUrlStation;
        String newPathStation = "/";
        if( newLocation[14] == 's' ) newPortStation = "443";
        int index = 0;
        while( newLocation[index] != '/' ) index ++;
        index +=2;
        while( newLocation[index] != '/' ) {
          newUrlStation += newLocation[index];
          index ++;
        }
        index ++;
        for( int k=index; k < newLocation.length(); k++ ) newPathStation += newLocation[k];

    // connect to new url
        Serial.print(F("\nRedirect to http://"));
        Serial.print(newUrlStation); Serial.print(":"); 
        Serial.print(newPortStation.toInt()); Serial.println(newPathStation);
        if( cookie != "" ) {
          Serial.print(F("with the cookie: ")); Serial.println(cookie);
        }
        Serial.println(F("-----------------------------"));

        if( client.connect(newUrlStation, newPortStation.toInt()) ) {
          client.print(String("GET ") + newPathStation + " " + protocol + "\r\n" +
                  "Host: " + newUrlStation + "\r\n" + 
                  "Cookie: " + cookie + "\r\n" +
                  "User-Agent: Mozilla/5.0\r\n" +
                  "Connection: close\r\n\r\n"); 
        }    // end of client.connect test

        header = client.readStringUntil('\n');   // read the header until \n
        Serial.print(F("1-->")); Serial.println(header);

      }      // end of while on header HTTP
    }        // end of test on headerSearch

    if( ++bufferNumber < 5 ) {
      Serial.print(F("@@@>"));
      client.read(mp3buff, buffSize);
      for( int i=0; i<buffSize; i++ ) Serial.write( mp3buff[i] );
      Serial.println();
    }
  }      // end of client.available test
}        // end of loop

[Dr-Dawg]

I had to do some minor modifications, in order to make the code work with ESP32 and without WiFiManager.h

I also added the following else statement, so you can actually hear music after reading the header

    if( ++bufferNumber < 5 ) {
      Serial.print(F("@@@>"));
      client.read(mp3buff, buffSize);
      for( int i=0; i<buffSize; i++ ) Serial.write( mp3buff[i] );
      Serial.println();
    }
    else
    {
     uint8_t bytesread = client.read(mp3buff, buffSize);
     player.playChunk(mp3buff, bytesread);
    } 

For a 'normal' station there is a header output and music, for Kassara Barikama I got stucked after the 8th header line in the client.readStringUntil command in the following loop:

        do {
          header = client.readStringUntil('\n');
        ...

It's not an infinite loop, it gets stuck in client.readStringUntil('\n'), possibly it reads the mp3 stream that does not provide "\n"?