EliasKotlyar / Monsieur-Cuisine-Connect-Hack

167 stars 21 forks source link

Recipes #6

Open dgliosca opened 5 years ago

dgliosca commented 5 years ago

Hi I was wondering if you ever explored how to create new recipes and feed them to the collection.

toolsche commented 4 years ago

Hi,

Thanks for the explanation. Why does the cable have to be angled 90°? I have a USB A-Male Y Cable like this. Can it be used? I would have to put the device next to my stationary pc anyways if I want to use flash with Win10.

Right now I'm working on a React App for displaying the recipes similar to the MC website. My plan was once I have the basics down (listing, filter) to extend it with a possibility to create custom recipes with original JSON structure which could then be displayed (simulated) like in the MC2-App and export it to SQL format for importing in the database.

bobstar-fr commented 4 years ago

Hello No, your cable is made to link a USB hard drive with Mini USB to a computer in USB A with the possibility of powering the HDD via the 2nd USB A socket. For my angled USB A socket, a picture will be more explicit.

20200521_045509

My current problem is programming to integrate my excel update file into the SQL recipes file. Until I find the solution, I manually import (by drag and drop) the new data and it works.

You should know that in the database is a bit complicated. The number of tables is constant (it makes sense) and that the number of elements per table varies according to several parameters: 1) the number of steps in the recipe, 2) the number of ingredients in the recipe, 3) the number of machine actions ("ramp", "scales", "cooking", .....) 4) the number of submenus on the MCC screen, .... This makes the structure different from one recipe to another.

Look at this explanatory file (sorry it's in French) : Chronologie.xlsx

PatrizioR commented 4 years ago

Hello together, would be nice if we all can work together on this.

My success was:

  1. ADB enable on mcc (so that I don't need twrp)
  2. close the mc2 app
  3. cp Recipes into /sdcard for having access
  4. pull Recipes
  5. change sqlite locally (add recipe works)
  6. push Recipes to /sdcard
  7. cp Recipes back to origin
  8. chmod
  9. start app works

To add images, you also need to copy to /files/image_recipes the version small- and large- jpg.

Problem at the moment is the delte elias mentioned:

Recipes server. When device is connected to internet, it asks server for recipe's ids for current language. If database has recipe with id not listed in server's response - it will be deleted

So unsubscribe device should do the trick, so it will not sync

I will start to write a .net core server api app with blazor ui for creating recipes and sync to the machine. Could work on a raspberry pi zero for example

bobstar-fr commented 4 years ago

Hello I have the same problem as you .... a new recipe and its images disappear from the file after a connection to the server Regarding CHMOD: You apply it to which file and what value do you give? Thank you

PatrizioR commented 4 years ago

The most granted 777 ;) To databases/Recipes and image_recipes

bobstar-fr commented 4 years ago

Hello A fake server will undoubtedly facilitate the addition / modification of recipes but will not prevent the deletion of the added recipes when the MCC connects to the real server.

The solution to use existing values of recipes for the RECIPE table in the fields UPLOADED, VALID_FROM and VALID_TO does not work for a new recipe (it is nevertheless erased during connection) My EXCEL file and finally developed and works well. It allows you to easily add recipes by generating SQL code which is added to the existing file in the MCC. I noted a lot of subtleties on the use of the MODE of the STEP table. I can upload my EXCEL file here but I warn that it is in French.

toolsche commented 4 years ago

Hi,

UPDATE. Header X-Recipe-Type: [default | live] defines recipe type. If no type specified - default is used by server.

Besides 'default' and 'live' there is another recipe type: 'beta'.

All beta recipe ids:

curl -H "Accept-Language: de" -H "X-Recipe-Type:beta" https://mc20.monsieur-cuisine.com/mcc/api/v1/recipe/ids

All beta recipes in German:

curl -H "Accept-Language: de" -H "X-Recipe-Type:beta" https://mc20.monsieur-cuisine.com/mcc/api/v1/recipe/all

Another interesting method I found:postAllMachineUsageLog() It posts an Object of Type 'MachineInfoData' to the usage api:

URL: https://mc20.monsieur-cuisine.com/mcc/api/v1/machine/
bobstar-fr commented 4 years ago

Hello Very interesting. Thank you. To prevent recipes manually added to the "Recipes" file disappearing after a while, do you know an effective way. I am blocked because my personal recipes disappear after a while even with the MCC disconnected from the MCC account. If I put the MCC's wifi on OFF, it automatically returns to ON when restarted and the new personal recipes disappear. Apparently by blocking the MCC's wifi on the router, the personal recipes does not disappear, even after restarting. This means that even without an account connected, the MCC automatically connects to the server at startup. Thank you.

toolsche commented 4 years ago

Ah that maybe explains why the recipe api is not secured by a user token. I also noticed that if you change the language on the device it deletes all recipes and reloads it with the new language setting from the server. In addition, not all recipes are loaded at once, but rather in packages from update to update.

Example: recipe-fetch-01 recipe-fetch-02

bobstar-fr commented 4 years ago

Hello Somewhere it is logical that the server erases personal recipes because the automatic update could not be done in a global and uniform way for all the users of the same country. In fact, to add a personal recipe we increment several "id" in the database, the server during an update of the recipes cannot increment correctly the individual databases of the MCCs if other people do it too. .. The automatic update should not be global for all users but individualized for each database.

I confirm that you must cut OFF the wifi at the router level to prevent any connection from the MCC to the server. Even by putting the wifi OFF this one comes back to ON when restart and connects the MCC. The fact of not indicating a user account is useless .... even without an account the MCC connects to the server if the wifi is ok. Obviously without a user account the MCC does not update automatically but still deletes the personal recipes added to the database.

LilaQ commented 4 years ago

What about some MitM for the DB requests? We could just re-route the DB requests (maybe even to the MCC itself), have some Python script check if this is a custom recipe or not (maybe even by adding a new field to the sqliteDB, like personal or so), have it relay the request to the official MC servers if necessary, otherwise block it.

Oh, also, I don't have my USB cable yet. If someone could post a working link to the sqliteDB and/or the Discord again, I would really appreciate it. Sounds like a fun project! :)

LilaQ commented 4 years ago

For the sake of collecting data, here is a single formatted JSON of a Recipe: https://pastebin.com/CNNFZRwb

jnsc commented 4 years ago

I worked on this quite a bit this weekend, and here is what I have done.

So there is still a lot of thing to (make everything dynamic, support more api calls, work on the Gutenberg blocks for creating the data, multi language support, and a ton more). But it's at least a proof of concept, for now, everything from code to the notes I took is really to messy to share.

Thanks to @kovz for the curl command and for the way to get the apk, it really was the starting point of all my investigations.

LilaQ commented 4 years ago

@jnsc Nice work! Are you planning to make your work public? A github repo or something? And do you happen to have an idea how to obtain the APK when I don't have a USB cable yet? wink wink

PatrizioR commented 4 years ago

@jnsc awesome can you provide the steps for creating the certificate and decompile recompile the app? I am totally interested in this.

So a raspberry pi as recipe server would work with this. Do you have a preferred communication platform can contact you?

Many thanks to your progress

jnsc commented 4 years ago

@jnsc Nice work! Are you planning to make your work public? A github repo or something? And do you happen to have an idea how to obtain the APK when I don't have a USB cable yet? wink wink

Of course I will make it public, I only have to get it in a quite decent state. All the info for the APK is here, https://github.com/EliasKotlyar/Monsieur-Cuisine-Connect-Hack/issues/1#issuecomment-627331182

jnsc commented 4 years ago

@jnsc awesome can you provide the steps for creating the certificate and decompile recompile the app? I am totally interested in this.

So a raspberry pi as recipe server would work with this. Do you have a preferred communication platform can contact you?

Many thanks to your progress

for now, I would not recommend to anyone to do it on a real device, so the first step would be to setup an emulator.

The app looks if the certificate if signed by a valid CA, so if you want to host it on you Raspi, you will need at least a domain name and a Certificate. But by chance they choose the Let's Encrypt CA which means you can get the certs for free.

PatrizioR commented 4 years ago

@jnsc since I have the backup of the rims and two devices I am willing to do it on one of my devices.

Decompiling and change the url, what did you use for decompiling and recompiling? Own domain and own certificate won't be the problem :) But I am sure you also can omit the CA check if you can decompile the app right? :)

jnsc commented 4 years ago

@jnsc since I have the backup of the rims and two devices I am willing to do it on one of my devices.

Decompiling and change the url, what did you use for decompiling and recompiling? Own domain and own certificate won't be the problem :) But I am sure you also can omit the CA check if you can decompile the app right? :)

Ok, but please be carefull. So for the decompilation I used apktool here is the install process used https://ibotpeaches.github.io/Apktool/install/

Decompile apktool d MCLauncher-release-1.1.17-226.apk

Find url references grep -r -l 'https://mc20.monsieur-cuisine.com/mcc/api/v1/'

Rebuild after changing the url's apktool b MCLauncher-release-1.1.17-226

create a signing keystore keytool -genkey -keystore apktool.keystore -validity 10000 -alias apktool

sign the apk jarsigner -keystore apktool.keystore -verbose MCLauncher-release-1.1.17-226/dist/MCLauncher-release-1.1.17-226.apk apktool

Zipalign zipalign -v 4 MCLauncher-release-1.1.17-226/dist/MCLauncher-release-1.1.17-226.apk MCLauncher-release-1.1.17-226/dist/MCLauncher-release-1.1.17-226-signed.apk

Install adb install MCLauncher-release-1.1.17-226/dist/MCLauncher-release-1.1.17-226-signed.apk

As I did a lot of test, I also had to edit the manifest before I got the apk working, but it might not be necessary.

LilaQ commented 4 years ago

I thought the idea to gather in Discord to exchange info was good, so I created a new Discord with a working invite link. https://discord.gg/mKEMEPv

PatrizioR commented 4 years ago

Rebuild after changing the url's apktool b MCLauncher-release-1.1.17-226

Just decompile and directly recompile does not work (and so any other changes in between) kovz already mentioned apktool has a problem. These lines makes unable to use apktool for compile again:

MCLauncher-release-1.1.17-226\AndroidManifest.xml:34: Tag attribute name has invalid character ':'. W: MCLauncher-release-1.1.17-226\AndroidManifest.xml:35: Tag attribute name has invalid character ':'.

Thank you for sharing your steps :)


Update: Removed both lines and got it compiled, but then stucked on installing to vm first, "Failure [INSTALL_FAILED_SHARED_USER_INCOMPATIBLE]"...

Update: Got it, signing the app with the right key did the trick: Certificate fingerprint (MD5): 8D:DB:34:2F:2D:A5:40:84:02:D7:56:8A:F2:1E:29:F9

needed to build it with openssl by myself :/ took me 6 hours to solve this.

PatrizioR commented 4 years ago

@jnsc I got the app installed on mcc, but no luck with the url. Did you use a non https url and also had a local server running in your lan?

I tried:

Will try now:

But if you have any advice, it would be faster, still diging in the dark

Main problem is login url seems to be also one of the 11 urls saved in the apk, how did you skip login and got the recipes anyway?

jnsc commented 4 years ago

@jnsc I got the app installed on mcc, but no luck with the url. Did you use a non https url and also had a local server running in your lan?

I tried:

* https Remote url with certificate valid by root cert

* https LAN Computer name (with self signed certificate, but guess won't work)

Will try now:

* http LAN Computer name

* http LAN IP

* https LAN IP

* http Remote url

But if you have any advice, it would be faster, still diging in the dark

Main problem is login url seems to be also one of the 11 urls saved in the apk, how did you skip login and got the recipes anyway?

I used a remote webserver with a valid Let's encrypt SSL certificate (As it's what the App checks). If you want to try I can give you my apk (which points to my server) but you will only get 2 recipes in french.

PatrizioR commented 4 years ago

If you can rename your app, than I can install it (beside the original, in my opinion a good idea) Or just give me the url (but would prefer not here, just text mit in github or discord), so I can compile with this url and we can see if it works :)

(Just need to know which urls you replaced, because the one for login I guess you didn't provide a login api?)

jnsc commented 4 years ago

e the original, in my opinion a good idea) Or just give me the url (but would prefer not here, just text mit in github or discord), so I can compile with this url and we can see if it works :)

I sent you a mail.

PatrizioR commented 4 years ago

It works :)

So lets create some nice and general solution if you like.

@all no registration needed in mcc (and actual not possible), but I made it as a seperated app, so no problem at all

jnsc commented 4 years ago

It works :)

So lets create some nice and general solution if you like.

@ALL no registration needed in mcc (and actual not possible), but I made it as a seperated app, so no problem at all

I implement the login (every email and every password works), the registration should soon be implemented too.

PatrizioR commented 4 years ago

@jnsc works like a charm, can you identify the device by time of login? Recognized I am directly logged in without any further action. Favorites can be realised by device identification

bobstar-fr commented 4 years ago

Hello It's a good job you've done. You have modified the apk for connecting to a personal server .... But I don't understand how you manage to modify or add a personal recipe to the existing list in the machine and how will the automatic update be done as new recipes arrive on the Lidl server. Can you explain in a few words? Thank you.

jnsc commented 4 years ago

Hello It's a good job you've done. You have modified the apk for connecting to a personal server .... But I don't understand how you manage to modify or add a personal recipe to the existing list in the machine and how will the automatic update be done as new recipes arrive on the Lidl server. Can you explain in a few words? Thank you.

Basically we substitute our personal server to the one of lidl. In fact we will not have access anymore to the recipes of lidl unless someone imports them to the personal server (which certainly would not be allowed due to copyright restrictions). The second solution would be to proxy requests if a recipe is not found on the personal server.

bobstar-fr commented 4 years ago

OK, thanks. I see that what you are doing is the same as my solution .... The original server will no longer update the "Recipes" file and only yours will be able to do it .... My solution is to add the news recipes at the end of the "Recipes" file and keep the MCC offline. The apk is not changed ... if I want an automatic update by the original server, I connect the wifi, my recipes are deleted, the update is done and I only have to submit my recipes at the end of the new "Recipes" file. I think this solution is less complicated than yours and you must have a good level of programming to realize it. I created an excel file which adds the tables and fields necessary for a new recipe and a macro which generates the SQL to inject in the original "Recipes" file ...and that's it... Regards

jnsc commented 4 years ago

OK, thanks. I see that what you are doing is the same as my solution .... The original server will no longer update the "Recipes" file and only yours will be able to do it .... My solution is to add the news recipes at the end of the "Recipes" file and keep the MCC offline. The apk is not changed ... if I want an automatic update by the original server, I connect the wifi, my recipes are deleted, the update is done and I only have to submit my recipes at the end of the new "Recipes" file. I think this solution is less complicated than yours and you must have a good level of programming to realize it. I created an excel file which adds the tables and fields necessary for a new recipe and a macro which generates the SQL to inject in the original "Recipes" file ...and that's it... Regards

But my whole point is to have an easy way to edit the recipes, like a web frontend. The other point would be to have a centralized source where people could contribute there recipes (like Wikipedia). So you would be able to get a lot more recipes "by the community"

bobstar-fr commented 4 years ago

Let me have a big doubt on the community .... If we are 20 around the world to think about this, to want and be able to modify the MCC to improve it, the others will not want to touch the system by complaint to lose the warranty or for fear of not succeeding. We have the same phenomenon with phones .... How many users have routed their smartphones? (compared to the number of smartphones sold) ... Do you have control over the "Recipes" file? Do you know the overlaps of tables and fields to develop a personal recipe. If yes, ok ... if not : I can share my experience on this subject.

PatrizioR commented 4 years ago

Since we can have two MC app installation in parallel I don't see any disadvantages. Have two APK installed on my mcc one original and the other with private server. If someone wants he could run a private server with lidl recipes as well. But we can't have a public recipe server with that

jnsc commented 4 years ago

Let me have a big doubt on the community .... If we are 20 around the world to think about this, to want and be able to modify the MCC to improve it, the others will not want to touch the system by complaint to lose the warranty or for fear of not succeeding. We have the same phenomenon with phones .... How many users have routed their smartphones? (compared to the number of smartphones sold) ... Do you have control over the "Recipes" file? Do you know the overlaps of tables and fields to develop a personal recipe. If yes, ok ... if not : I can share my experience on this subject.

For my cooking has always be something where you want to put in your touch (adding more ingredients, removing some, etc.) So not being able to do so (editing the recipes) stopped me from getting a Monsieur Cuisine Connect.

For the Recipe, that's really a big part of the work, and if you are willing to help, it would be great. I think we need to document all fields an possible values of the json, for this I would propose using https://json-schema.org/, having the json part documented would also simplify further developments.

toolsche commented 4 years ago

@jnsc Working with JSON schema is currently my daily work. So I would like to help with defining the schema. As far as I analyzed the code of the MCLauncher it does not contain a schema. They are reading the JSON file directly with Gson.fromJson() into model objects. => see /db/ImportRecipesAndImagesTask.smali and /mcapi/json/Parser.smali

I already created a small app which reads a recipe json file and lists all recipes in it by name. The list can be filtered and the results are displayed as Material UI Cards. I am planning to extend it by editing features.

toolsche commented 4 years ago

I found a nice tool written in Python which can generate a json schema from a given json input: GenSON Generate one, unified JSON Schema from one or more JSON objects and/or JSON Schemas.

Here you can see the generated schema with a correctly validated recipe json: recipe-schema.json => recipe.json

You can even seed the generator with a json schema to tweak the output.

bobstar-fr commented 4 years ago

Hello You are too strong for me .... I know nothing about JSON, I manage with SQL and that is why I operated with Excel which allows me to create the recipe and create the SQL to add to the existing "Recipes" file. The principle is simple:

LilaQ commented 4 years ago

So, I messed around with the app, and managed to get it to communicate with my local server (private IP) with a valid SSL certificate that I was able to force.

I can see every request the app makes to the server now, and can deliver responses (only stubbed out for the moment, the certification took a bit of time).

I dumped a few recipes as JSON data, and will soon deliver them from my fakeserver to the app. The fakeserver can then easily pull new recipes from the official servers, since I can set up my own sorting structure of official and handmade recipes (which then are supposed to be addable with a nice WebUI)

This is all done on a AVD/Emulator for now, since I don't have my USB cable yet, but this should all apply to the MCC itself.

jnsc commented 4 years ago

ate a json schema from

Yes, I used a similar tool https://jsonschema.net/home, I'm not a JSON expert, but from what I see there is still a lot to do especially on the descriptions, and also this automated tools would say that complexity is a string, but the MCC will only accept some value like "Easy, Medium, Hard"

"weight": { "$id": "#/items/anyOf/0/properties/data/properties/guidedCooking/properties/steps/items/anyOf/0/properties/machineValues/properties/weight", "type": "integer", "title": "The weight schema", "description": "An explanation about the purpose of this instance.", "default": 0, "examples": [ 150 ] },

LilaQ commented 4 years ago

Easter Egg While inspecting the MCC app, I found that there is a complete episode of "PINGU" on it - please don't ask me why! 😆

Proof

bobstar-fr commented 4 years ago

jnsc write :

Yes, I used a similar tool https://jsonschema.net/home, I'm not a JSON expert, but from what I see there is still a lot to do especially on the descriptions, and also this automated tools would say that complexity is a string, but the MCC will only accept some value like "Easy, Medium, Hard"

"weight": { "$id": "#/items/anyOf/0/properties/data/properties/guidedCooking/properties/steps/items/anyOf/0/properties/machineValues/properties/weight", "type": "integer", "title": "The weight schema", "description": "An explanation about the purpose of this instance.", "default": 0, "examples": [ 150 ] },

hello For the description and nesting of the tables of "Recipes" file look at my excel files ... everything is documented.

toolsche commented 4 years ago

Yes, I used a similar tool https://jsonschema.net/home, I'm not a JSON expert, but from what I see there is still a lot to do especially on the descriptions, and also this automated tools would say that complexity is a string, but the MCC will only accept some value like "Easy, Medium, Hard"

Sure, the enums can't be created automatically I guess but it is a skeleton we can use and add the required values. Maybe that is a thing for which we can use the seed schema feature.

jnsc commented 4 years ago

I got my notes into a readable shape, happy to share them, and happy if other peoples could also contribute their notes. I have setup a private (as there are some infos that are not anonymized like ip's, etc) gitlab repo for now. If you want me to grant you access please let me know your gitlab username.

toolsche commented 4 years ago

Gitlab or Github?

jnsc commented 4 years ago

Gitlab or Github?

Lab, I'm more used to lab

toolsche commented 4 years ago

Ok, registered with same name.

toolsche commented 4 years ago

Awesome work! 😎👍🏼

PatrizioR commented 4 years ago

Ok now it's my turn:

I will create an app, that are capable of:

-> Bootstrap mobile first

Any other requests? If someone is good at designing (Create pictures how the app could look like, I would appreciate that, not a good designer at all ;) )

acabala commented 4 years ago

@LilaQ @PatrizioR @jnsc and others :)

Just 2 cents from me. I have already started developing PWA a few months ago. Some basic functionality is working, but a lot needs to be done still. (VUE + firebase + algolia) Anyway, I'm pretty open to sharing it with you if you would be interested in making it work together. please check live demo here: https://lidlomix.web.app/

img1 img2 img3