No mac needed anymore

[niggowai]

Combination of that project plus some others (biemster/FindMy, beeper/pypush and Dadoum/anisette-v3-server) to work without a mac.

A "real" Apple Device or Hackintosh is still needed for the first time Registration

[dchristl]

Hello @niggowai,

thank you for your merge request. I will have a deeper look, when I have more time. I am currently observing the numerous changes gaining momentum in the openhaystack project, and I have already set up a server completely "MAC-less" in the cloud. My modifications are still experimental at the moment because I am still hoping to completely do away with the MAC. For this, it seems that the original authentication needs to be rebuilt into SMS2FA, and no MAC (neither physical nor virtual) will be required. biemster and the JJTech (the maitainer of pypush) are currently working on it. If this will work all together, I will release a new version and update the documentation. Currently, I can't say when this will hapen.

Kind regards, Danny

[CappyT]

@niggowai @dchristl Sorry for pinging you all, but seems that pypush now can also generate the access token, if i'm not mistaken Could be feasible to port everything in the app itself? (no server needed)

[biemster]

the code in biemster/FindMy should be just a drop-in here, and there is no need for an Apple device or hackintosh for initial registration either.

[niggowai]

Yes @biemster, it should work like in a02bbda16fee9b17d97e8609ae285cd3a44b6d1a where I already recycled your code. It may be also good to hear @dchristl 's opinion to the fact that I transferred his application (at least in some point of view) to a docker stack. Since anisette also runs in python, the whole project can be refactored to only one webserver without more than one port open and without docker. But since with docker, development from anisette doesn't disturbe our project, I think using docker is more convenient and the users can control what services they want to run

[dchristl]

Hello @niggowai ,

sorry for the late reply, I'm a liitle bit busy recently. I consider it a great idea to containerize all of this in a Docker container. Even though I haven't worked much with Docker yet, my experience has been more on the user side than the maintainer side. I must have somehow missed that the project now runs completely without a Mac. That's great news, thanks @biemster. I can't pinpoint exactly when I'll get to it, but the plan is definitely to integrate this and make it extremely straightforward for non-Mac owners

Kind regards, Danny

[supaeasy]

Oh Boy!

Does that mean there will be absolutely no need for a macOS anymore at all? Not even virtualized? Cannot wait to sell the Mac mini I bought a week ago for this project only (two actually, thanks eBay greediness..) and host this entirely on my Synology NAS in Docker.

Keep up the fantastic work, this project is pure awesomeness!

[biemster]

@supaeasy On x86 (and maybe aarch64) there is no need for macOS anymore. Not even virtualized. I'm not sure if https://github.com/Dadoum/anisette-v3-server or https://github.com/Dadoum/pyprovision work on your Synology NAS though, you could check that before you sell your minis? (and report back here?)

[dchristl]

I have currently running a proof of concept with anisette-v3-server and the headless-haytsack-server in the oracle free cloud (without docker and with "old" code, that needs a MAC for 1st time init). It works like a charm and I think I will pack the anisette and the headless-haystack-server in one docker image. I think this will be the most convenient solution.

[CappyT]

If I can contribute, I will provide a deployment file for k8s (I run most of my workloads over kubernetes). Maybe an helm chart? If you guys prefer..

@dchristl I would suggest not to run multiple processes in a single docker container as is strongly against the idea behind docker and could lead to unexpected results. A quick Google search for "running multiple processes inside a container" would give you more context if you're curious. You can use docker compose for these types of setups (or k8s if you're brave)

@supaeasy not all Synology NAS(es) are cool running all type of containers. Please check first.

[dchristl]

Thank you for answer @CappyT . You see I#m not so deep into Docker ;) I don't think Kubernetes is the best idea, because it needs extra resources and installation steps, which is a disadvantage on smaller systems.

[supaeasy]

@supaeasy not all Synology NAS(es) are cool running all type of containers. Please check first.

@CappyT I got a DS1621+ with 32 GB Ram. It did run all docker images I tested so far (except for a macOS Virtualization) so im quite confident. What do you think?

[niggowai]

@dchristl At this Time, I'm like you quite busy, so I don't know if I can continue working on this in the next few months, but I would help providing Docker Containers and a compose file for easy Deployment. But I think I may need some Help with CI/CD on github :sweat_smile: . But I share the opinion of @CappyT to use multiple Containers, like that for example:

• anisette
• headless-haystack
• haystack-webserver
• reverse-proxy (maybe, to provide a simple Way to add ssl)

@biemster I got myself a Mac mini too, but obviously only to run linux on it, so I could check for aarch64 as well. Currently my Setup runs on a x64 VPS...

@supaeasy FYI: This is quite off-topic, but if you got yourself an apple sillicon device and don't like MacOS like me you can run them with AsahiLinux as they are quite energy efficient

Hope we can get It to work soon :rocket:

[supaeasy]

Nope lucky enough I went for the bare minimum with a 2014 MacMini since this project was its only use case.

[CappyT]

@niggowai Tried your code, but got 400 error from apple gateway. I made sure to use the same device.json from anisette to generate the auth.json.

Sadly, no luck. Am I missing something?

[dchristl]

@niggowai I will let you know, if I need some help. Thank you. I think I will start with a little bit easier approch. From my perspective, only two containers are needed: the Anisette server and the Headless Haystack server. The UI (i.e., the web server) can be used directly from GitHub and would be more of a nice-to-have feature. Similarly, the reverse proxy is probably a bit too much and complicates the setup. My server in the Oracle Cloud runs with SSL (Let's Encrypt) and is relatively simple to install separately. I'm also not entirely sure if this would work seamlessly in a Docker container (i.e., only Let's Encrypt without a reverse proxy). Once that is in place (since we both don't have much time ;) ), we can consider extensions later on.

[niggowai]

@dchristl Sure, the two containers are the ones only needed. I thought about a compose file which containes all these containers, but the reverse proxy and the webserver can be commented out, so only anisette and the haystack-container will be started by default. This would allow a small footprint but also the possibility to run the container with ssl/webserver without much additional effort.

[niggowai]

@biemster I can confirm this working for aarch64 but I ran into issues with the anisette-v3-server, so I switched to the anisette-server image and I had to include an additional dependency (libffi-dev) in the token-generator. Fetching a location should work, I don' recieve an Error, but I can't try right now, since I don't own an iphone and I don't live in a town where dozens of volunteers offer their Internet connection to upload my trackers location ;-)

[niggowai]

@niggowai Tried your code, but got 400 error from apple gateway. I made sure to use the same device.json from anisette to generate the auth.json.

Sadly, no luck. Am I missing something?

@CappyT would you open an issue in my repo with all the steps you did

[CappyT]

@biemster I can confirm this working for aarch64 but I ran into issues with the anisette-v3-server, so I switched to the anisette-server image and I had to include an additional dependency (libffi-dev) in the token-generator. Fetching a location should work, I don' recieve an Error, but I can't try right now, since I don't own an iphone and I don't live in a town where dozens of volunteers offer their Internet connection to upload my trackers location ;-)

I can provide a working test tag flashed on esp32. (Location is already found on apple's servers) Ping me on telegram (link is in my profile)

For the error, it's something on my end and I managed to resolve. Thanks anyway for the help

[dchristl]

@niggowai, @biemster
I'm currently trying out your codes and try to integrate it, but I'm getting errors: The register-process always raises KeyError: 'service-data'

and if I use my already registered data I always get an empty repsonse ending in json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This also happens with unchanged FindMy-project. Any ideas?

[biemster]

KeyError: 'service-data' indicates your login was unsuccessful. After Beeper Mini launched this week (which is based on pypush), Apple quickly blocked these and both are not working at the moment.

Oddly enough pypush still works for me with a config.json that was generated before, so there is still investigation needed what exactly is happening. You could join the pypush discord to check the progress on this.

[dchristl]

Thanks for your answer @biemster I've found the problem of the first issue, I always get the answer: {'dsid': '21355750913', 'delegates': {'com.apple.mobileme': {'status': 1, 'status-message': 'A server problem is blocking Apple ID sign in. Try signing in later.'}}, 'status': 0}

Oddly enough pypush still works for me with a config.json that was generated before, so there is still investigation needed what exactly is happening.

I have exactly the same behaviour and no problems with old accounts and the initial pypush-version. But if I copy one to the new code (which is not really different at this part) I get the empty result.

You could join the pypush discord to check the progress on this.

I'm already there, but there is going on too much ;) I have missed your solution to the SMS2FA, too.

[dchristl]

Sorry to bother you @biemster , but are there any news about the login issues? It seems Beeper Mini is running again, but I can't find any work-a-round on Discord and I get the same error like before (A server problem is blocking Apple ID sign in. Try signing in later)

[biemster]

@dchristl that's odd, the part that did not work in beeper and pypush is actually not even used in the code I took from them. Can you still log in on appleid.apple.com? Can you reset/reprovision your anisette source?

[dchristl]

Login to appleid.apple.com is possible. I've freshly cloned your FindMy-repo (main), generated a key and requested the reports. I'm currently using Anisette instead of pyprovision. I'm currently unable to compile pyprovision. Is this needed or should it work with anisette, too?

[biemster]

pyprovision is an application that provides anisette data. Are you using anisette-v3-server now for the anisette data? That's basically just a server around libprovision, and pyprovision is just a python wrapper around libprovision. So any of these three should be fine, and are the same. You could remove your adi.pb and device.json to reset your anisette source.

[dchristl]

Are you using anisette-v3-server now for the anisette data?

Yes, the one you link in your readme, but the docker version.

You could remove your adi.pb and device.json to reset your anisette source.

There are no such files. Are they inside the docker container? I will try to reset it.

[biemster]

The are in the docker container, in $HOME/.config/anisette-v3/

[dchristl]

I've resetted the server, but with no luck. The only difference was that the 2FA was triggered. The result was the same:

pyprovision is not installed, querying http://localhost:6969 for an anisette server pyprovision is not installed, querying http://localhost:6969 for an anisette server 2FA required, requesting code pyprovision is not installed, querying http://localhost:6969 for an anisette server Enter 2FA code: XXXXXX 2FA successful pyprovision is not installed, querying http://localhost:6969 for an anisette server pyprovision is not installed, querying http://localhost:6969 for an anisette server pyprovision is not installed, querying http://localhost:6969 for an anisette server {'dsid': '21355750913', 'delegates': {'com.apple.mobileme': {'status': 1, 'status-message': 'A server problem is blocking Apple ID sign in. Try signing in later.'}}, 'status': 0} Traceback (most recent call last): File "/home/danny/projects/FindMy/./request_reports.py", line 73, in auth=getAuth(regenerate=args.regen, second_factor='trusted_device' if args.trusteddevice else 'sms'), File "/home/danny/projects/FindMy/./request_reports.py", line 35, in getAuth j = {'dsid': mobileme['dsid'], 'searchPartyToken': mobileme['delegates']['com.apple.mobileme']['service-data']['tokens']['searchPartyToken']} KeyError: 'service-data'

The line numbers can differ from your version slightly, I've added some outputs

[dchristl]

I've tried a little bit and it seems with an "old" account everything is working fine. For the new implementation I created a complete new account without a connected device and SMS2FA only. The new account raises this error on fetching reports, the old one not. Maybe this is a restriction of Apple to have at least one real device? The virtual device shiows up after the 2FA at the account:

Any other ideas?

[biemster]

Apple IDs have a "score" indeed, and a low score might block you from certain services. New accounts without any devices have low scores, adding (even virtual) machines to it does increase the score. Also adding payment info helps (you don't have to actually purchase anything). I've heard rumors that new ids created via Apple Music (on android?) start of with a higher initial score than ones created on the web. Those are things I would try, although I only ever created a single account via appleid.apple.com and added 2 VMs to it. And that one account still works for this.

[dchristl]

Tahnks for your reply, this is a good starting point for testing out. I will use my other account and use the new one for error handling

[dchristl]

Quick update. I've integrated the new code into a new branch (macless). I'm not finished yet and will continue working on it when time permits, but currently, it works great in my tests. I've restructured some things, such as extracting the config, improving logging and error handling, and renaming files. The main file is now named 'mh_endpoint' (macless haystack endpoint), so don't be surprised. Feel free to give it a try when you have a chance.

[dchristl]

Hello @niggowai,

thanks for your work. I have partially integrated your code to my project. With the latest release this project is running completely without a mac.

I will close this pull request.

Kind Regards, Danny