Maintain connection in lock screen / standby

[michaelhunziker]

Dear all

I just deployed the Sample App to my Android phone. MQTT messages can be published and received as long as the app is active. When I leave the phone alone it will go to lock screen/standby and after about 15 minutes it loses connection to the MQTT Broker. As far as I understand the app uses a background service (MqttService.java) that will maintain a constant connection to the MQTT Broker. Am I wrong with this assumption?

How could I achieve a constant connection to the MQTT Broker even if the phone is in standby mode? I would like to implement an app similar to Whatsapp or Facebook Messenger. These apps are able to receive messages in standby mode.

Michael

[danki]

I'd believe notifications are transported with Push, when Messenger / Facebook are opened, connection with broker is established.

The reason why i believe this is cause i've been trying to do exact same as you for past 3 months and I just can not keep the damn connection alive. Ive pretty much tried everything here but no success... at all.

However there seems to be ONE tiny thing that might keep connection alive for a bit longer. From your Android client, put a timer to publish to a random topic. This keeps it going for a bit longer (sometimes) but still is not solid method.

I see no reason why there would be a MqttService, since that thing seems to live its own life. It doesnt even publish last will. I also went for my own Service that would keep things in rotation... connect -> when ever disconnected -> reconnect but i always run into Exception , invalid client handlers or some other stuff thats not cleared by MqttService and AndroidClient...

Frustrating...

[danki]

Last night i actually seemed to establish what i needed. I made some changes i my service and re installed app on phone and left it over night (8h+) and when checking back this morning, i was connected.

I'll post a example here for anyone to use see or improve or what ever.

I know it is messy and probably not the correct way doing, but as long it works for me as hobby-project I'm fine.

package com.package;

import android.app.Activity;
import android.app.Notification;
import android.app.PendingIntent;
import android.app.Service;
import android.content.Intent;
import android.os.Binder;
import android.os.Build;
import android.os.CountDownTimer;
import android.os.Handler;
import android.os.IBinder;
import android.os.Vibrator;
import android.util.Log;

import com.google.firebase.iid.FirebaseInstanceId;

import org.eclipse.paho.android.service.MqttAndroidClient;
import org.eclipse.paho.client.mqttv3.IMqttActionListener;
import org.eclipse.paho.client.mqttv3.IMqttDeliveryToken;
import org.eclipse.paho.client.mqttv3.IMqttToken;
import org.eclipse.paho.client.mqttv3.MqttCallback;
import org.eclipse.paho.client.mqttv3.MqttConnectOptions;
import org.eclipse.paho.client.mqttv3.MqttException;
import org.eclipse.paho.client.mqttv3.MqttMessage;
import org.json.JSONException;
import org.json.JSONObject;

import java.util.ArrayList;
import java.util.Date;

/**
 * Created by deko on (v29) 2017-07-18.
 */

public class MQTTConHandler extends Service {

  private static String TAG = "ComHandler";

  public static final String MQTT_STARTED = "com.mqtt.service.started";
  public static final String MQTT_CONNECTED = "com.mqtt.connected";
  public static final String MQTT_DISCONNECTED = "com.mqtt.disconnected";
  public static final String MQTT_CONNECT_FAIL = "com.mqtt.connect.failed";
  public static final String MQTT_SUBSCRIBED_OK = "com.mqtt.subscribed.success";
  public static final String MQTT_SUBSCRIBED_NOK = "com.mqtt.subscribed.failed";
  public static final String MQTT_DATA = "com.mqtt.indata";
  public static final String APP_DATA = "some_data";

  public static final String NOTIFICATIONS = "ar4.notification.RECEIVED";

  private static MQTTConHandler _instance;
  private static MqttAndroidClient client;
  private static Boolean mConnected = false;
  public static String ClientID;
  private MqttConnectOptions options;
  private MqttConnectOptions mqttConnectOptions;
  private ArrayList <String> subs = new ArrayList <>();
  private Activity mainActivity;

  Notification serviceRunningNotification;

  public static String clientID = "non_"+Math.floor( Math.random()*1000 ) ;

  // Keep a interval for sending data, kind of allows connection to be alive
  Handler handler = new Handler();
  Runnable runnableCode = new Runnable() {
    @Override
    public void run() {
      if ( _instance == null ) {
        return;
      }
      if ( client.isConnected() ) {
        Date now = new Date();
        JSONObject json = new JSONObject();
        try {
          json.put( "d", now.toString() );
          json.put( "s", now.getTime() );
        } catch ( JSONException e ) {
          e.printStackTrace();
        }
        publish( TOPICS.TOPIC_1 + ClientID + "/ping", json.toString() );
        handler.postDelayed( runnableCode, 10000 );
      } else {
        log( "mq not connected." );
      }
    }
  };

  public static MQTTConHandler getInstance() {
    return _instance;
  }

  @Override
  public void onCreate() {
    super.onCreate();
    log( "on create" );
    options = createMqttConnectOptions();
    client = createMqttAndroidClient();
  }

  @Override
  public int onStartCommand( Intent intent, int flags, int startId ) {
    log( "start, connected: " + isConnected() );
    _instance = this;
    broadcastUpdate( MQTT_STARTED );
    return START_STICKY;
  }

  @Override
  public IBinder onBind( Intent intent ) {
    log( "bind, connected: " + isConnected() );
    log( "Instance == " + ( this == _instance ) );
    if ( isConnected() == false ) {
      this.connect();
    }
    return null;
  }

  @Override
  public void onTaskRemoved( Intent rootIntent ) {
    log( "task removed" );
    mainActivity = null;
    restartServiceInBG();
    super.onTaskRemoved( rootIntent );
  }

  /**
  * Some attempt to re-establish connection when disconnected
  */
  public void restartService() {
    log( "stopself" );
    broadcastUpdate( MQTT_DISCONNECTED );
    client.unregisterResources();
    client.close();
    try {
      client.disconnect();
    } catch ( MqttException e ) {
      e.printStackTrace();
    }
    _instance = null;
    stopSelf();
    stopForeground( true );
  }

  /**
  * Vibration
  */
  public void vibrate( Long time ) {
    Vibrator vib = ( Vibrator ) getSystemService( VIBRATOR_SERVICE );
    vib.vibrate( time );
  }

  /**
   * MainActivity instance
   */
  public void setMainActivity( Activity _activity ) {
    mainActivity = _activity;
  }

  /**
   * Attempt for restarting service and trying to maintain connection
   */
  public void restartServiceInBG() {
    // restart the service with a notification
    Intent notificationIntent = new Intent( getApplicationContext(), this.getClass() );
    notificationIntent.addFlags( Intent.FLAG_ACTIVITY_CLEAR_TOP | Intent.FLAG_ACTIVITY_SINGLE_TOP );
    PendingIntent pendingIntent = PendingIntent.getActivity( this, 0, notificationIntent, 0 );

    serviceRunningNotification = new Notification.Builder( this )
            .setContentTitle( "Service" )
            // ConextText part bugs, Notification maysay disconnected despite valid connection and vice versa
            .setContentText( isConnected() ? "Connected" : "Disconnected" )
            .setSmallIcon( R.drawable.ic_extension_black_24dp )
            .setContentIntent( pendingIntent )
            .setTicker( "ticker ?" )
            .addAction(R.drawable.ic_new_releases_black_24dp,"Open",
                    PendingIntent.getActivity( getBaseContext(), 0,
                            new Intent( "android.intent.category.LAUNCHER" ).setClassName( "com.package", "com.package.MainActivity" ),
                            PendingIntent.FLAG_UPDATE_CURRENT ) )
            .build();

    startForeground( 10, serviceRunningNotification );
    connect();
    log( "Starting in BG" );
  }

  /**
   * Connection options
   */
  private MqttConnectOptions createMqttConnectOptions() {
    mqttConnectOptions = new MqttConnectOptions();
    mqttConnectOptions.setUserName( "username" );
    mqttConnectOptions.setPassword( "password".toCharArray() );
    mqttConnectOptions.setCleanSession( true );
    mqttConnectOptions.setKeepAliveInterval( 10 );

    /*
    * NOTE
    *   After commenting out auto reconnect,
    *   app seemed to be connected with broker at least 8h.
    *   After good night sleep, i checked the status on morning and it was still going strong
    */
    //mqttConnectOptions.setAutomaticReconnect( true );
    ClientID = "app_" + Build.DEVICE + "_" + Math.random();
    mqttConnectOptions.setWill( "/app/will", ( "uex " + "asd" ).getBytes(), 2, false );
    return mqttConnectOptions;
  }

  /**
   * Creates android client
   *
   * @return Android client
   */
  private MqttAndroidClient createMqttAndroidClient() {
    log( "Create new client" );
    client = new MqttAndroidClient( this, "tcp://URL:PORT", ClientID );
    return client;
  }

  /**
   * Disconnect from broker
   */
  public void disconnect() {
    log( "disconnect..." );
    try {
      client.unregisterResources();
      //client.close(); // causes exceptions
      client.disconnect();
      client = null;
    } catch ( MqttException e ) {
      log( "1disconnect exception: " + e.toString() );
    } catch ( NullPointerException ne ) {
      log( "2disconnect exception:: " + ne.toString() );
    }
    broadcastUpdate( MQTT_DISCONNECTED );
  }

  /**
   * Connect to broker.
   */
  public void connect( final MqttAndroidClient client, MqttConnectOptions options ) {
    log( "Connect" );
    try {
      if ( client != null ) {
        IMqttToken token = client.connect( options );
        client.setTraceEnabled( true );
        token.setActionCallback( actionCallback );
        client.setCallback( mqttCallback );
      }
    } catch ( MqttException e ) {
      //handle e
      Log.e( TAG, "connect: " + e.getMessage(), e );
    } catch ( NullPointerException ne ) {
      Log.e( TAG, "connect: " + ne.getMessage(), ne );
    } catch ( Exception uknw ) {
      Log.e( TAG, "connect: " + uknw.getMessage(), uknw );
    }
  }

  /**
   * Connect to broker.
   * If client is not null, it will be reused instead of creating new one
   */
  public void connect() {
    log( "Connect: " + client );
    if ( client == null ) {
      client = createMqttAndroidClient();
    }
    this.connect( client, options );
  }
  /**
   * On connect
   */
  private IMqttActionListener actionCallback = new IMqttActionListener() {
    @Override
    public void onSuccess( IMqttToken asyncActionToken ) {
      log( "onconnect" );
      handler.post( runnableCode );
      try {
        // subscribe to topic
        client.subscribe( TOPICS.TOPIC_3, 0 );

      } catch ( MqttException e ) {
        e.printStackTrace();
      }
      broadcastUpdate( MQTT_CONNECTED );

    }
    @Override
    public void onFailure( IMqttToken asyncActionToken, Throwable exception ) {
      log( "onconnect fail " + exception.toString() );
      broadcastUpdate( MQTT_CONNECT_FAIL );
    }
  };

  /**
  * On connectionlost and messages received
  */
  private MqttCallback mqttCallback = new MqttCallback() {
    @Override
    public void connectionLost( Throwable cause ) {
      broadcastUpdate( MQTT_DISCONNECTED );
      client.unregisterResources();

      if ( null != cause ) {
        Log.d( TAG, "connection lost: " + cause.getMessage() + " " );
        Log.e( TAG, "connectionLost: ", cause );
      }
      if ( _instance.mainActivity == null ) {
        stopForeground( true );
        vibrate( 100L );
      }

      new CountDownTimer( 2000, 1000 ) {
        public void onTick( long millisUntilFinished ) {}
        public void onFinish() {
          restartServiceInBG();
        }
      }.start();
    }

    @Override
    public void messageArrived( String topic, MqttMessage message ) throws Exception {
      Log.d( TAG, topic + " " + message.toString() + " dupe: " + message.isDuplicate() );
      broadcastUpdate( MQTT_DATA, topic, msg );
    }

    @Override
    public void deliveryComplete( IMqttDeliveryToken token ) {
      log( "delivery complete " + token.toString() );
    }
  };

  /**
   * Broadcast updates
   *
   * @param action What to broadcast
   * @param topic Which topic received data
   * @param data What's the data
   */
  private void broadcastUpdate( String action, String topic, String data ) {
    final Intent intent = new Intent( action );
    intent.putExtra( "topic", topic );
    intent.putExtra( "data", data );
    sendBroadcast( intent );
  }

  /**
   * Broadcast updates
   * Also sets mConnected, since broadcastUpdate( ) with Action only 
   * Indicates mostly connectivity states
   *
   * @param action
   */
  private void broadcastUpdate( String action ) {
    log( "Broadcast: " + action );
    final Intent intent = new Intent( action );
    if ( action.equals( MQTT_CONNECTED ) || action.equals( MQTT_DATA ) ) {
      mConnected = true;
    } else if ( action.equals( MQTT_CONNECT_FAIL ) || action.equals( MQTT_DISCONNECTED ) ) {
      mConnected = false;
    }
    sendBroadcast( intent );
  }

  /**
   * Is connected
   *
   * @return boolean
   */
  public boolean isConnected() {

    return mConnected;
  }

  /**
   * Subscribe to a topic
   *
   * @param topic String Topic to subscribe to
   */
  public void subscribe( String topic ) {
    try {
      client.subscribe( topic, 0 );
    } catch ( Exception e ) {
      log( "Subscribe exception: " + e.toString() );
    }
  }

  /**
   * Publish method
   *
   * @param topic
   * @param msg
   * @return
   */
  public boolean publish( String topic, String msg ) {
    if ( client == null ) {
      return false;
    }
    return publish( topic, msg, false );
  }

  /**
   * Publish method
   *
   * @param topic
   * @param msg
   * @param retain
   * @return
   */
  public boolean publish( String topic, String msg, boolean retain ) {
    if ( client == null ) {
      return false;
    }
    if ( client.isConnected() ) {
      MqttMessage mMsg = new MqttMessage( new byte[ 0 ] );
      if ( msg == null ) {
        mMsg.setRetained( true );
      } else {
        mMsg.setRetained( retain );
        mMsg.setPayload( msg.getBytes() );
      }
      try {
        client.publish( topic, mMsg );
        return true;
      } catch ( MqttException e ) {
        log( "Publish exception " + e.toString() );
      }
    }
    return false;
  }

  /**
   * Binder
   */
  public class LocalBinder extends Binder {
    public MQTTConHandler getService() {
      log( "getService: " + ( MQTTConHandler.this == _instance ) );
      return MQTTConHandler.this;
    }
  }

  /**
   * Log.debug
   *
   * @param string
   */
  private static void log( String string ) {
    Log.d( TAG, "log: " + string );
  }
}

[jpwsutton]

Thanks for posting your solution, unfortunately this restriction is due to the doze feature introduced in Android O (https://github.com/eclipse/paho.mqtt.android/issues/195). At this point the advice is to something like this, or to wait for maintenance windows to reconnect and send messages. Mainstream applications that want to get the best of both worlds are using MQTT when in the foreground, but GCM (Google Cloud Messaging) in the background to wake the app up when there is a message waiting for it. Sadly I don't think that there is anything we can do within the client as all of these solutions depend on your own code as well as a number of other factors.

[chintan-mishra]

@danki does this solution work everywhere and can it be generalized? If so, then I believe it needs to surface more in searches.

[Stefan300381]

@danki any chance you provide a full project sample?

[chintan-mishra]

@Stefan300381 I was using a custom solution built on the above for my app, 8hoot. We started noticing complaints from some OnePlus, Huawei, and Xiaomi users on Andorid 8+ that the OS shows the app stopped working even if the app is not being used.

IMO the best solution is to wake up the device using FCM and start your MQTT connection. With recent restriction it is quite hard(if not impossible) to keep the connection open when the app is in background. Even with high priority FCM this sometimes fails.

[Stefan300381]

hmm, this is really bad news. I was planning to write an mqtt doorbell for my own purpose. my phone is xiaomi redmi note 6 :-/