external storage management on android kitkat

[graychan]

The application sometimes fails on Android 4.4 (maybe other versions of Android 4). This is the result that Android changes its way of managing persistent storage (internal & external storage). As stated in the Android External Storage Technical Information guide,

Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. This enables apps to manage their package-specific directories on external storage without requiring they hold the broad WRITE_EXTERNAL_STORAGE permission. For example, the app with package name com.example.foo can now freely access Android/data/com.example.foo/ on external storage devices with no permissions. These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.

This has to be addressed to cope with a few new changes introduced in various versions of Android 4.

[andriosrobert]

Is not this feature related only to the managing of the package directories ? I think that the working way stayed the same for the other folders of the OS where the data is being stored. On this material they are saying something related to it: http://www.doubleencore.com/2014/03/android-external-storage/

"There are two main changes here in KitKat: Read access now requires the READ_EXTERNAL_STORAGE permission (for apps that didn’t already have the pre-existing write permission granted), and data stored in the managed application directories on external storage (i.e. /Android/data/[PACKAGE_NAME]) require no permissions at all if you are the application that owns those files."

[graychan]

A very good find. Perhaps, the issue is that we did not give the application READ_EXTERNAL_STORAGE. I observed in a few occasion that the application crashes on a fresh AVD. My casual study seems to suggest that it was a result of file access permission problem. I did not study it in depth, but wanted to have an issue to investigate it further.

Your investigation is timely. We may conduct a few tests, i.e., running the app on a freshly created AVD with the 4.4.2 Android to see if crashes. If it does, we can give the app the READ_EXTERNAL_STORAGE permission to see if it fixes issues.

In addition, we may investigate future, for newer version android, if we want to save the data in an application specific directory. I suspect that one motivation for Android to change its management of external storage in this way, is data security and privacy -- one app cannot access another app's private data. However, our data is not something we want to protect.

You may continue to investigate to see if the above plan sounds reasonable since I again did not study it in-depth.

On Fri, Jun 6, 2014 at 5:16 PM, andriosr notifications@github.com wrote:

Is not this feature related only to the managing of the package directories ? I think that the working way stayed the same for the other folders of the OS where the data is being stored. On this material they are saying something related to it: http://www.doubleencore.com/2014/03/android-external-storage/

"There are two main changes here in KitKat: Read access now requires the READ_EXTERNAL_STORAGE permission (for apps that didn’t already have the pre-existing write permission granted), and data stored in the managed application directories on external storage (i.e. /Android/data/[PACKAGE_NAME]) require no permissions at all if you are the application that owns those files."

— Reply to this email directly or view it on GitHub https://github.com/sysnetlab/SensorDataCollector/issues/78#issuecomment-45385299 .

[andriosrobert]

Thank you for the hint of conducting the tests on new devices, I did not thought about that. I conducted tests on some different new ADVs and on the actual devices (with the app being previously removed), and the error could not be reproduced. According to our AndroidManifest file, it seems that we already have the READ_EXTERNAL_STORAGE permission settled, as can be seen in the image bellow:

The changes in Kitkat on managing the files are actually related only to the package folders, where no permission is required to read or write to it, and to the introduction of the READ_EXTERNAL_STORAGE, which is automatically granted to the apps that have the WRITE_EXTERNAL_STORAGE.

http://developer.android.com/reference/android/Manifest.permission.html#READ_EXTERNAL_STORAGE:

Any app that declares the WRITE_EXTERNAL_STORAGE permission is implicitly granted this permission

So, this issue should not be related to the granting of accesses.

One thing that could be happening is that, when running the app, another higher priority task locks the access to the storage resource. For instance, when trying to load up the app to the actual device through USB, if the "USB Mass Storage" option is active, it is not going to be possible to run the app. It happens because the storage resources are locked due to the higher priority task of exchanging data between computer and device.

I can not think of another situation where the resource could be locked during a regular phone use, which may bring to the cause of this issue. However, it would be a further issue to be faced, where a long experiment is being run and the user wants to exchange data with the computer, it will make the app crash due to the lost of the storage resource.

About saving the data on the package folder, even though we do not want to protect our data, I think that it would be worth to store it on the app specific folders, once that when the app is uninstalled, the system will take care of deleting the files contained on it (saving us of this role when dealing with the uninstallation of the app) and it is also a recommended by the Android documentation:

http://developer.android.com/reference/android/os/Environment.html#getExternalStorageDirectory()

Applications should not directly use this top-level directory, in order to avoid polluting the user's root namespace. Any files that are private to the application should be placed in a directory returned by Context.getExternalFilesDir, which the system will take care of deleting if the application is uninstalled. Other shared files should be placed in one of the directories returned by getExternalStoragePublicDirectory(String).

[graychan]

That is an excellent point that other threads may lock the external device. In discussion with @dcwalter , another possibility is that Android has not finished mounting the external SD card when the app is already started.

Since you were not able to reproduce them, it is very likely the two cases above, which are generally rare.

Based on your research, I recommend,

• In a future milestone, implementing a logic that checks if external storage is available and if not asks user to check, such as if a SD card is installed and to wait a few seconds after Android boots, which should alleviate the problem instead of throwing an exception and quitting in an ugly manner.
• In a future milestone, we can implement new external storage method introduced in Android 4.

For now, I am moving this issue out of this milestone, unless you believe that you can complete the first bullet and are more interested in working on this issue than others in this milestone.

[andriosrobert]

Ok, we can work on that later.

[graychan]

Perhaps, this is related to this. Observed a crash report on a user's device,

java.lang.RuntimeException: Unable to start activity ComponentInfo{sysnetlab.android.sdc/sysnetlab.android.sdc.ui.SensorDataCollectorActivity}: java.lang.RuntimeException: SimpleFileStore::SimpleFileStore(): failed to create directory /mnt/sdcard/SensorData
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1647)
at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1663)
at android.app.ActivityThread.access$1500(ActivityThread.java:117)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:931)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:123)
at android.app.ActivityThread.main(ActivityThread.java:3683)
at java.lang.reflect.Method.invokeNative(Native Method)
at java.lang.reflect.Method.invoke(Method.java:507)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:864)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:622)
at dalvik.system.NativeStart.main(Native Method)
Caused by: java.lang.RuntimeException: SimpleFileStore::SimpleFileStore(): failed to create directory /mnt/sdcard/SensorData
at sysnetlab.android.sdc.datastore.SimpleFileStore.<init>(SimpleFileStore.java:68)
at sysnetlab.android.sdc.datastore.SimpleXmlFileStore.<init>(SimpleXmlFileStore.java:71)
at sysnetlab.android.sdc.datastore.StoreSingleton.getInstance(StoreSingleton.java:30)
at sysnetlab.android.sdc.ui.SensorDataCollectorActivity.onCreate(SensorDataCollectorActivity.java:54)
at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1047)
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1611)
... 11 more

The information about the deivce and the version of the app is,

Application version   0.10
Android version  Android 2.3.3 - 2.3.71
Device  Optimus One (thunderc)

[SagarPandit01]

This issue was moved to sysnetlab/SenSee#20