Feature: support new API sign mechanism (HMAC-SHA265)

nalajcie commented 5 years ago

Adding new API signage mechanism compatible with current (3.9.0) TuyaSmart App. This enables You to use API with clientId/token used in official apps - which means it's possible to continue using official app and read/control device from the cloud simultaneously.

For details regarding reverse-engineering process see: tuya-sign-hacking repo. You can also find there the credentials retrieved from the official app (I didn't include them here on purpose in case there will ever be any takedown action from Tuya).

Things to consider:

there is no clear difference between old/new API in terms of submitted data (I've decided to use apiEtVersion which hopefully will change if the API will change in the future; moreover, the API is versioned on per-action basis (each action has it's own version field) which adds another level of confusion)
I don't know if the distinction between using old and new API is clear enough (providing different options to constructor). Maybe there should be whole new class (Cloud2? but we don't even know if that's version 2). Maybe it should be made more explicit by adding e.g. useHMACSign option to the constructor?

What do You think?

nalajcie commented 5 years ago

One more note - I've added tests for new API but don't have the credentials which will pass all the tests at the same time (the credentials from the official app will return SING_VALIDATE_FALED_4 with the old API). Maybe there should be 2 sets of credentials for the tests?

Unfortunately I don't know how to create encrypted keys.json to be used in Travis.

codetheweb commented 5 years ago

That walkthrough of the reverse engineering process was really interesting, thank you for taking the time to share that and patch @tuyapi/cloud.

I updated the encrypted keys file so tests should pass now.

A few questions:

What does apiEtVersion stand for? Would just having version as a parameter make more sense?
Similar question for loginEx: what does ex stand for? Is it just included as a login function for the new API? If so, it might be better to move that logic into the standard login function, which then checks if this.apiEtVersion is set to use the correct version.

Edit: tests aren't passing because Travis disables sensitive environment variables on PR builds. They should pass after a merge to master.

nalajcie commented 5 years ago

Hi, thanks for the positive reply, regarding Your questions:

What does apiEtVersion stand for? Would just having version as a parameter make more sense?

I don't know if we should call the oldAPI version 1 and the new version 2 - there is no indication of API version in mobile app. And in the docs each endpoint is being versioned separately (in the docs all have 1.0 but maybe there are newer versions used by the mobile app).

The apiEtVersion is the only parameter sent with all requests from the mobile app which may point to api version being used (currently hard-coded to 0.0.1, maybe it will change in the future with changes in API?). Et may also be a shortcut from ETag which is sent with each of the request but not being used right now.

Maybe we should add some option like signAlgorithm and export constants SIGN_MD5, SIGN_HMAC_SHA256? The only downside is that it may prevent us from differentiating between two versions of API in the future if the signage mechanism would not change. But it may never happen :).

Similar question for loginEx: what does ex stand for? Is it just included as a login function for the new API? If so, it might be better to move that logic into the standard login function, which then checks if this.apiEtVersion is set to use the correct version.

Actually, the login should be possible using both variations of login action (they are described in the docs). It's written that it's the same version (1.0) of this API endpoint but with different input parameters. It's probably possible to use the "more secure" version in old API, bu I haven't tested that (so it should not depend on apiEtVersion, maybe we can add optional parameter to login, eg. ifencrypt used in docs to probably differentiate the input parameters). I've experimentally found out that it's not possible to log in using "less secure" method when providing Tuya's clientId.

The Ex stands for extended as this login requires two requests to the API (and some RSA crypto) as opposed to simple login function. I left it there because it's much simpler to port it to other language if anyone would need to do it.

codetheweb commented 5 years ago

Ok, everything you said makes sense.

I'm good with merging (after a few touch ups to the README) if everything looks good to you.

Edit: finished and ready to merge if it looks good. 👍

nalajcie commented 5 years ago

Everything looks good to me, you can proceed with the merge. 👍

codetheweb commented 5 years ago

Squashed and merged. ~~Pending test results.~~

Tests pass. 🎉 Thanks for your hard work.

nalajcie commented 5 years ago

That's great! It might be a good idea to bump the package version (and release to npm?) to better track any problems with the new implementation - it's up to You.

If there would be any issues with the new API - please mention me in the bugs (I'll try to help) - I'm watching your repo right now, but may miss something in the future :).

Thanks for Your work!

codetheweb commented 5 years ago

Totally forgot to, thanks for the reminder.

Now published and bumped to v0.4.0.

TuyaAPI / cloud

Feature: support new API sign mechanism (HMAC-SHA265) #3