search

BlueBubblesApp / bluebubbles-server

Server for forwarding iMessages to clients within the BlueBubbles App ecosystem
https://bluebubbles.app
Apache License 2.0
509 stars 42 forks source link

Manual and automatic refresh of contacts is non-functional and/or unreliable #634

Open brianjcohen opened 4 months ago

brianjcohen commented 4 months ago

Steps to reproduce:

  1. My Mac is configured to synchronize contacts automatically from my Google Account to 'Contacts'
  2. From my Android phone, I add a contact to my Google Account

Expected behavior:

The new contact should, within a reasonable amount of time, become available to Bluebubbles Server and the clients that connect to it.

Actual behavior:

The new contact never becomes available to Bluebubbles on its own.

Additional troubleshooting and information:

SpaceSaver commented 4 months ago

I think BB manages its own copy of the contact book iirc

brianjcohen commented 4 months ago

@SpaceSaver I also believe it does, but it should be importing entries from the Mac's Contacts periodically, and on-demand when you use Contacts > Manage > Refresh Contacts but it does not seem to be working that way any longer.

SpaceSaver commented 3 months ago

Are you able to use the export contacts feature in the settings of the Android app?

brianjcohen commented 3 months ago

@SpaceSaver I haven't tried that because it didn't seem relevant to troubleshooting.

zlshames commented 3 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server
  3. Pulled from your Mac's Address Book
    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time.

It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

brianjcohen commented 3 weeks ago

@zlshames correct, I am expecting Bluebubbles server to automatically pull in new entries from the Mac AddressBook, which in turn is currently synchronizing with Google Contacts. However the actual behavior is that new entries from Google Contacts that make their way into AddressBook do not reliably get automatically pulled into BlueBubbles.

zlshames commented 3 weeks ago

@zlshames correct, I am expecting Bluebubbles server to automatically pull in new entries from the Mac AddressBook, which in turn is currently synchronizing with Google Contacts. However the actual behavior is that new entries from Google Contacts that make their way into AddressBook do not reliably get automatically pulled into BlueBubbles.

If it's not grabbing the latest, then it's likely an issue with the node-mac-contacts library we use.

Here is where we refresh the contacts: https://github.com/BlueBubblesApp/bluebubbles-server/blob/master/packages/server/src/server/api/lib/ContactsLib.ts#L46-L57

Which should get called when you use the Refresh button in the UI.

And this is the package: https://www.npmjs.com/package/node-mac-contacts

Do you know if restarting the server will allow it to propagate or show in the Contacts tab?

brianjcohen commented 3 weeks ago

@zlshames as per my original bug report, in this state the Refresh button does not import the new contacts but restarting BlueBubbles will:

Contacts > Manage > Refresh Contacts does not import the new contact. Restarting Bluebubbles server on the Mac does successfully import the new contact.

cameronaaron commented 3 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server

  3. Pulled from your Mac's Address Book

    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time.

It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

Would it be possible to have it sync like at least once a day or something automatically?

zlshames commented 3 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server

  3. Pulled from your Mac's Address Book

    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time. It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

Would it be possible to have it sync like at least once a day or something automatically?

im trying to implement it, but am having difficulties with Google's OAuth flow... If i figure it out, sure. But it's not looking good...

Basically, for technical folks. The OAuth flow that I'm currently using requests a response_type of "token", which will return the access_token in the callback. This allows for authorized use of the token until the expiration occurs. The downside to this method is there is no "refresh token", so when the access token expires, the user needs to re-consent for us to get a new access token.

In order to get a refresh token, I need to use the (response_type) "code" authorization flow. In this flow, instead of the full access token being returned from the callback, a "code" is returned. Which I can then use with either the client secret or challenge verification string. Since I do not want to store the client secret on the server app, the OAuth authorization flow w/ PKCE is the preferred method. The PKCE flow says that I (the client) can generate a "challenge" that gets sent to Google. That challenge will be hashed on Google's side, and will be used when I request the access token using the "code". That way, we can verify that the requestor is who received the callback.

Anyways... in my testing, the API is erroring out when I provide the "code", "client ID", and "challenge verification string", saying that I still need to include the "client_secret". From my understanding, the PKCE flow should not require a client secret. So something is broken/wrong and I'm not sure what.

zlshames commented 3 weeks ago

@zlshames as per my original bug report, in this state the Refresh button does not import the new contacts but restarting BlueBubbles will:

Contacts > Manage > Refresh Contacts does not import the new contact. Restarting Bluebubbles server on the Mac does successfully import the new contact.

my fault. Missed that. I think this may be a bug in the node-mac-contacts library code. As in, it doesnt detect any new contacts, so it's using a cached version of the contacts list, not including the new contact.

Theoretically, when node-mac-contacts detects a change, it should clear the cache, and then emit the event to the JS code: https://github.com/codebytere/node-mac-contacts/blob/main/contacts.mm#L715

My hunch is that because the change is happening via a Google Contacts sync rather than an iCloud sync, the callback is not firing, and not clearing the cache. So the cached values continue to be used.

That said, I'm not sure how I would fix the issue without modifying/forking the node-mac-contacts library and making the changes myself to force a refresh or something.

zlshames commented 2 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server

  3. Pulled from your Mac's Address Book

    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time.

It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

Would it be possible to have it sync like at least once a day or something automatically?

Not possible due to this: https://stackoverflow.com/questions/78673050/is-a-client-secret-required-for-google-oauth-2-0-using-the-pkce-authorization-fl

cameronaaron commented 2 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server

  3. Pulled from your Mac's Address Book

    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time. It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

Would it be possible to have it sync like at least once a day or something automatically?

Not possible due to this: https://stackoverflow.com/questions/78673050/is-a-client-secret-required-for-google-oauth-2-0-using-the-pkce-authorization-fl

I would be willing to have a developer account and provide a secret would it be possible to just have that option somwhere?

zlshames commented 2 weeks ago

There are a few methods to get contacts into the BlueBubbles Server:

  1. From your Android App (manual or automatic export)
  2. Manual VCF upload to the server

  3. Pulled from your Mac's Address Book

    • You can also sync Google Contacts to your Mac's Address Book, with the caveat of when BlueBubbles pulls from it, avatars are excluded (this is just the way things are)

Soon (next update), there will be a 4th. Sync directly from your Google Account to the server using a button on the Contacts page of the server. This will include avatars as well. This sync process is manual and won't automatically sync every x amount of time. It sounds like your issue is that you are expecting the server to automatically pull new entries in. If you are syncing through AddressBook's integration with Google Contacts, it should.

Would it be possible to have it sync like at least once a day or something automatically?

Not possible due to this: https://stackoverflow.com/questions/78673050/is-a-client-secret-required-for-google-oauth-2-0-using-the-pkce-authorization-fl

I would be willing to have a developer account and provide a secret would it be possible to just have that option somwhere?

just makes things pretty complicated from a UX and backend standpoint...

cameronaaron commented 2 weeks ago

Yeah fair thank you for trying hopefully this will be possible in the future