Rewrite of AppData? "Appen behöver uppdateras" issue.

[walle89]

For the full story, read issue #39.

Short version

Recently Swedbank have updated the minimum required mobile apps version that's allowed to use the API. SwedbankJson is affected by these changes because the client is using the mobile API to do requests to Swedbank. In the light of this I may need to look into another solution for AppData.

Please read the proposals and leave some feedback. I would really appreciate it!

Long version

The basic concept behind SwedbankJson is to mimic how the mobile apps do requests to the mobile API. In the same way a end user needs to update there app in order to continue to use mobile banking services, this library needs also be updated in order to mimic the new app versions. This is done by changing the AppID and user-agent in AppData.php with the same appID and user-agent that the apps are using.

The AppData class have worked quite well, mainly because this file have not needed to be updated once per 1 to 2.5 years. This have been done via a patch and updated via Composer. However in the recent events of this needed to be updated less than a month, have made me reevaluate the current design of AppData. Maybe it's not a good idea to update AppID and User-agent as of a path via Composer. This should maybe only be reserved to only actual changes in code and not to update a few strings, even if it's essential in order to make SwedbankJson to function.

Currently I don't know the reason for why Swedbank have updated the minimum app version requirement more often than before. This may just be temporary and it was circumstances this happens 3 times for a short period. Or this could be an internal policy change at Swedbank to force there customers to have the latest app version. Or some other reason.

It's currently too early to say this will become an issue for foreseeable feature, but I think I need at least have a plan to rewrite how AppData needs to be handled and rewrite the code according to it.

Proposals

1. Automatic AppData update

The idea to create a static JSON file with the latest AppData configuration in order to make the API tick. I plan to host it in this Github repo and use Github raw file function (eg. https://raw.githubusercontent.com/walle89/SwedbankJson/master/appdata.json). I will update this file manually each time there is a new Ios app release by Swedbank. SwedbankJson will download this file and save it in a local cache before the first request to the Swedbank API. The cache will remain until it expires or the client API have detected minimum version error.

Pros

• We can keep using the Swedbank app API and keep all the endpoints that we have today.
• Probably minimal changes to your project code.
• Overall low risk changes in the library.

Cons

• Not a 100% automatic process. To my knowledge there is no way to get this data directly from Swedbank automatically. Someone (likely me) needs to update the appdata.json file manually.
• Third party connection and privacy concerns. Because of the potential sensitive information the API is sending and receiving, I want to stick to my zero knowledge policy. I can't be hacked and leak data that I don't have in the first place. This the main reason for why I will never design the library to send any data no other than Swedbank API servers. It's up to you to implement whatever third party services you want to use. It's a chance that the host (Github in this case) logs the IP numbers that access the appdata.json file. Even if this can be partially be solved by allow download the file and add it to library manually, this still gives out more information of the use of the SwedbankJson than the current solution does.
• Potential source of support issues. In order for this to automatically update to work you need to allow the script write to disk.

2. Swedbank web API

Currently the client API is using the API used by the mobile apps. But Swedbank have also have a web version of it called "Nya Internatbanken" (https://online.swedbank.se/). It's using the same technical base as the API for apps and looks and behaves very similar in the API request. Because the AppID is far more accessible (could in theory be fetched automatically) and the user-agent most likely doesn't need to be changed because most likely there is no "minimal version" check for web at all.

Pros

• With this is we may no longer need to select the bank type, both Swedbank and Sparbankerna shares the same web portal. But will needs to be investigated
• It's much easier to contribute to new endpoints for SwedbankJson. You don't need to setup and configure HTTP proxy and don't need to have an IOS device or an older Android device. Just open the network tab in your console for your favorite web browser and you will see the API requests and responses.
• Access to features that are not in the apps...

Cons

• ...and maybe lose access app only exclusive features. I need to investigate how this affects
• It's a bit risky to implement. Because I don't know this is possible or not. I'm 95% sure it should work in theory.
• Changes are less predicable to monitor. There are no releases like in App store or Play store.

3. Keep as is

Don't change anything. Meaning that each time Swedbank changes there minimum requirements, you have to update the library with Composer update.

Pros

• Business as usual.

Cons

• Not an automatic process. Someone (likely me) needs to update AppData.
• Forced to update the library to the latest version. This may entail your code needs to be updated in order to support the new APIs of the library, even if you only needed to update the AppData. You should never need to change any code that is downloaded with Composer (vendor/ folder).

Feedback and suggestions

Leave feedback and suggestions below and tell me what you think.

[Leatherface75]

This sounds good to me.

[walle89]

@Leatherface75 Can I assume that you are rooting for the "1. Automatic AppData update" suggestion? I'm basing that on your previous comment in issue #39 :

Your first idea if possible sounds like a good idea.

[Leatherface75]

Yes automatic appdata update seems the best way to do it.

[NisseDILLIGAF]

no. 1 Because I don't think QuickBalance will work in 'nya Internetbanken'... right? or no. 3, just keep it as is, and update code when needed.

[walle89]

I have just released a pre-alpha release of version 1.0.0. The primary focus of this release is to test the current implementation of the new App data with the auto updater.

The reason for I'm releasing this as a pre-alpha is the following:

1. I want you to have more time to test the auto updater before I release the rest of the features in 1.0.0. Mainly to have done a filed tested update of the AppData.json file and maybe catch & fix some errors in cetacean edge cases. But also give you the chance to give early feedback.
2. I plan to move Swedbank API from version 4 to version 5. Should be simple, It's mostly testing all endpoints and document changes from version 4 that takes time.
3. Current way to add arguments to AppData() and auth classes have some downsides in aspects of feature updates and features, like implementing PSR-18. The idea is to replace optional parameters with a config array. Like how Guzzle have implemented the Request() class.

How to test

1. Change the composer.json file to this:

"walle89/swedbank-json": "dev-pre-alpha-1.0.0" and run composer update if you have an existing project. Otherwise run composer require walle89/swedbank-json:dev-pre-alpha-1.0.0

2. Modify the code

I have updated the documentation and how to use the auto updater (Mobile ID example, with side-by-side changes, and AppData documentaiton).

3. Test

If you have the standard configuration, look for the AppData.json file after the frist successfull run and see if it contains some json data. If you manage to use any mehod in Swedbankjson() class and get the file, it's working.

Questions or feedback?

Feel free to reach out. I'm more than happy to assist if you have some trouble with the upgrade or installation of this alpha version. If you have issues, it's likely someone have or will have it too.

Feedback is much appreciated. It's hard to use someone as a sounding board when working as a team of one. :)

[NisseDILLIGAF]

Seems to be working... only two small things... I couldn't run 'composer upgrade' I had to run 'php composer.phar update' I'm not very familiar with Composer..

And I had some errors at first run... AppData.json couldn't be created so I had to create a folder and chown the folder to www-data, then point the cachePath there.

Thanx for great work!!

[walle89]

Thank you for the feedback @NisseDILLIGAF.

I changed the instruction from composer upgrade to composer update, a typo from my part.

The reason for you need to use php composer.phar is because you run the local composer installation. In order to use the composer command you need to do follow the global installation instruction. But you don't have to do that if you won't to, it's mainly a shortcut to do the same thing but with less typing. :)

Creating a cache folder or change file permissions is pretty much expected. This is one of the downsides with this new method compared to have it hard coded. That's one of the reasons for I added a way to disable cache and just download AppData.json each time authentication is needed.

[walle89]

Swedbank have released new update of its apps, that means it's time for a live test.

I have updated the appdata.json file, and all you need to do is to use SwedbankJson as you normally would do. What you need to do is to see if your cache file have been updated. A easy way to tell if that's the case is to compare the date in updated with the current remote appdata.json file (https://raw.githubusercontent.com/walle89/sbj-resources/master/src/AppData.json).

Note that the standard cache expiration time is 1440 minutes (=24 hours), which means that it can take up 1440 minutes before it will check and download the remote appdata.json file. If needed, you can adjust the cache expiration time. How you do that in documented in the AppData documentation

Let me know if it worked for you or not.

[NisseDILLIGAF]

working as expected here.... great work!!

[walle89]

Nice to hear that Nisse!

A quick update. I have moved the source of AppData.json.

From

https://raw.githubusercontent.com/walle89/SwedbankJson/files/appdata.json

To

https://raw.githubusercontent.com/walle89/sbj-resources/master/src/AppData.json

I have mainly done so mainly for it makes more sense to have this as a separate git project repo. First of, starting with a new clean git history for AppData.json and its tools makes more sense then a giant delete commit. But also from a perspective of maintenance and ability to make pull request in order to contribute to AppData.json is now far more easier. Also make it easier for others to use AppData.json and the tools for it in there on projects.

The new project is called SBJ Resources and you can find it here.

The old URL will work for a while, but I'm no longer maintain it and all feature updates will be done in the SBJ Resources project.

How to update

Option 1 - Composer update

I have updated pre-alpha-1.0.0 branch with the new URL as standard. Unless a custom cache URL is used (see AppData documentation), all you need to do is to do a Composer update:

composer update

Option 2 - Set custom cache URL

Change the code to the following (more details in AppData documentation):

$cachePath = __DIR__.'/AppData.json'; // Full path to cache file
$cacheExpiration = 1440; // 24 hours in minutes
$remoteUrl = 'https://raw.githubusercontent.com/walle89/sbj-resources/master/src/AppData.json';

$appData  = new SwedbankJson\AppData($bankApp, $cachePath, $cacheExpiration, $remoteUrl);

[NisseDILLIGAF]

how does a chljsFile look like, and how do you make it? just curious, I like to know how things work :)

[walle89]

Excellent question! I have planed to publish documentation how to use the AppData build tool including how to get the raw data from the Swedbank apps.

But in short, in Charels proxy there is an option to export a session in .chlsj format. This file is in JSON format and can be decoded by a standard JSON parser such as json_decode() in PHP. If you want to know more about .chlsj, i suggest to do an export and explore the file in an editor you would to use to open a json file. :)

[NisseDILLIGAF]

Tjena! Tänkte jag skriver här istället för att öppna upp en ny Issue. Tror att AppData.json behöver uppdateras...

Appen beh\xc3\xb6ver uppdateras. Du har en \xc3\xa4ldre version av appen installerad p\xc3\xa5 din telefon. F\xc3\xb6r att komma vidare beh\xc3\xb6ver du uppdatera till den senaste versionen.

Någon som har Charles installerat och kan fånga upp 'appID' och 'useragent' till swedbank? Har inte kvar Charles och jag har nyare mobil som vad jag har förstått gör avlyssningen mycket svårare.

[walle89]

I have updated AppData and my tests indicates that the API is working.

[walle89]

Version 1.0.0 related with the new AppData implementation released!

If you already running pre-alpha-1.0.0, change SwedbankJson version from pre-alpha-1.0.0 to ^1.0 then run composer update. Otherwise, read the installation instructions.

Also note that some braking changes has been introduced with version 1.0.0 that isn't related to AppData. Please read read the change log for more information.