bernaferrari / ChangeDetection

Automatically track websites changes on Android in background.
Apache License 2.0
703 stars 98 forks source link
android android-app android-application androidx architecture-components carousel dagger2 diff jsoup kotlin kotlin-android livedata material-design meyer paging-library pdf room room-persistence-library viewmodel

ChangeDetection

Change Detection

This app tracks changes on websites you otherwise would visit frequently to see if there is something new. Use cases:

This app also showcases all the Android Architecture Components working together: Room, ViewModels, LiveData, Paging, WorkManager and Navigation.

<img src="https://play.google.com/intl/en_us/badges/images/generic/en-play-badge.png" alt="Get it on Google Play" height="80"> <img src="https://f-droid.org/badge/get-it-on.png" alt="Get it on F-Droid" height="80">

GIF

Screenshots

Main Screen Text Diff PDF Diff Settings
First Sec Third Fourth

Introduction

Features

This app contains the following screens:

Presentation layer

This app is a Single-Activity app, with the following components:

The app uses a Model-View-ViewModel (MVVM) architecture for the presentation layer. Each of the fragments corresponds to a MVVM View. The View and ViewModel communicate using LiveData and general good principles.

Data layer

The database is created using Room and it has two entities: a Site and a Snap that generate corresponding SQLite tables at runtime. There is a one to many relationshiop between them. The id from Site is a foreign key on Snap. Snap only contains the snapshot metadata, all the data retrieved from the http request (body response) is stored in Android's own File storage.

To let other components know when the data has finished populating, the ViewModel exposes a LiveData object via callbacks using interfaces (inspired from this todo app). This could be, eventually, easily extended to work with server and sync. The app also makes use of Kotlin's Coroutines to deal with some callbacks.

Simple comparison process

The app works like this:

  1. Make http request and store the body response in a byteArray.
  2. Retrieve most recent stored file for that site, if any.
  3. Convert to string, clean up Jsoup and compare them. If same, don't do anything else. If different, add the new byteArray to storage and create a new entry on Snap table. When this happens in background, a notification is created to warn the user.
Inside the App Outside the App
inside outside

Diff Process for text files

After a change is detected and user taps to see it, a byte to byte comparision wouldn't be readable, so it makes sense to make a text comparison.

That's why this app makes extensive use from java-diff-utils. In fact, part of the library was converted to Kotlin and is now working perfectly on Java 6 (the original library makes use of Streams, which is only supported on Java 8). All the diff process is made using Myer's diff algorithm, and the result, for performance reasons, is put on a RecyclerView.

When this diff process happens, the app will use jsoup with a relaxed whitelist to remove all the useless tags from html to avoid pages that generate them at every request. Example: pages that make use of Google Analytics and pages that were made in WordPress. The app will also use jsoup to unescape "<" and ">" from html.

Diff Process for image and pdf files

It makes no sense to compare images and visual files using strings, so there is a carousel to compare them. PDF's are rendered to an imageView, while images are rendered with support for tiling, which is great for ultra-heavy pictures - in case user is tracking changes for a 20mb photo.

How each Architecture Component is used

Third Party Libraries Used

Reporting Issues

Issues and Pull Requests are welcome. You can report here.

License

Copyright 2018 Bernardo Ferrari.

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.