search

Azure / azure-notificationhubs-dotnet

.NET SDK for Azure Notification Hubs
MIT License
70 stars 119 forks source link

[BUG] Bulk ImportCreateRegistrations fails for every registration due to bad format, format is correct #322

Closed laura-vuilleumier closed 1 day ago

laura-vuilleumier commented 3 weeks ago

Describe the bug I am looking to move registrations from one notification hub to another. I am doing a simple bulk export to a blob file, then using that file as the import file for the bulk import into the new notification hub. I am not manipulating the exported registrations at all. However after the import is completed, all registrations fail to import with the message BadRequest,Bad Format. There is no explanation as to why, and the format has not been changed since exporting. I have run this exact code in the past (2021) and it was successful, but it no longer works.

To Reproduce

  1. Upload a file to Azure storage account container containing the following (data has been anonymised, but this is a snippet of the result of our export):

    <RegistrationDescription xmlns:i="http://www.w3.org/2001/XMLSchema-instance" i:type="GcmRegistrationDescription" xmlns="http://schemas.microsoft.com/netservices/2010/10/servicebus/connect"><ETag>1</ETag><ExpirationTime>9999-12-31T23:59:59.9999999Z</ExpirationTime><RegistrationId>9189319367137501945-6806091655136799927-2</RegistrationId><Tags>tag1,tag2,tag3</Tags><GcmRegistrationId>c-kTRr22RFu7G_ZBASRJBk:APA21bG73DUPKupRKAz2jEOMaT1PNnldSx2MFLJhh6n9rGcAlNLT5Icou4YgAdLWh5QVPMenZDx7aUhq44FMtTNCGX0Lb23yNCEOfZuGr5E_amMRXefE-bPTP93eWcmproUJMnLigTrX</GcmRegistrationId></RegistrationDescription>
<RegistrationDescription xmlns:i="http://www.w3.org/2001/XMLSchema-instance" i:type="GcmRegistrationDescription" xmlns="http://schemas.microsoft.com/netservices/2010/10/servicebus/connect"><ETag>1</ETag><ExpirationTime>9999-12-31T23:59:59.9999999Z</ExpirationTime><RegistrationId>9206851291504901378-1769790979570585279-1</RegistrationId><Tags>tag1,tag2,tag3,tag4,tag5</Tags><GcmRegistrationId>f_LWbNZ-TL-qy2n7JuJljA:APA21bFY7FsAEs7ZBWCIauhrdc8RoJkZyYIoVlfAdPCLKZ4gik_dv61gkZK5E5c7w6k7hFBPcdEzVrb35J-9p7lJlVVJDpoh8UIZmyAhZ5FrYb7UM7BprauhaEJjCbkBqg_h2nBAHoDS</GcmRegistrationId></RegistrationDescription>
<RegistrationDescription xmlns:i="http://www.w3.org/2001/XMLSchema-instance" i:type="FcmV1RegistrationDescription" xmlns="http://schemas.microsoft.com/netservices/2010/10/servicebus/connect"><ETag>1</ETag><ExpirationTime>9999-12-31T23:59:59.9999999Z</ExpirationTime><RegistrationId>24163815222695650-2672019681024503802-2</RegistrationId><Tags>tag1,tag2,tag3</Tags><FcmV1RegistrationId>cL624fevRMKVFMZuPC1K_S:APA21bE-j7oXgYuL0UAs40-CpgnCUmfUW-UUNDEJ-6DalyEZ6NofTAnxQ1QeUeEtJsiQmClZj7RU4DCr0YWK8VtBmgki4iKLQZC5qta5kv2EyinVi177NnFXh6GEp5wLUAbayOf-6xcN</FcmV1RegistrationId></RegistrationDescription>
<RegistrationDescription xmlns:i="http://www.w3.org/2001/XMLSchema-instance" i:type="FcmV1RegistrationDescription" xmlns="http://schemas.microsoft.com/netservices/2010/10/servicebus/connect"><ETag>1</ETag><ExpirationTime>9999-12-31T23:59:59.9999999Z</ExpirationTime><RegistrationId>57377756869098668-9176389869569878913-6</RegistrationId><Tags>tag1,tag2,tag3,tag4,tag5</Tags><FcmV1RegistrationId>f51S60qERl6ACetBGdvrZ3:APA92bGToxQZMamtiDaLZuJU7qbgLuzeKB1uNHdo8bt5gAJ-W96uJXAOQdSFi1b9bx-hO18X7MbzAf-Zl2Q9qeSP23YokzBUwkcTw28TTxUYT898f8LewXXBMVNZ2LBG6-zc5NuQfh0j</FcmV1RegistrationId></RegistrationDescription>

  2. Create a new job of type ImportCreateRegistrations using the above file as the import file, and wait to complete.

  3. Navigate to output container. Ouput.txt is empty, and Failed.txt contains all above registrations which failed to create with error BadRequest,Bad Format.

Code snippet I'm using code ripped directly from the README of this project.

Export code:

var job = new NotificationHubJob
{
    JobType = NotificationHubJobType.ExportRegistrations,
    OutputContainerUri = new Uri(containerUri),
};

job = await nhc.SubmitNotificationHubJobAsync(job);

Import code:

var job = new NotificationHubJob
{
    JobType = NotificationHubJobType.ImportCreateRegistrations,
    ImportFileUri = new Uri(inputFileUri),
    OutputContainerUri = new Uri(containerUri)
};

job = await nhc.SubmitNotificationHubJobAsync(job);

Expected behavior I expect registrations to create successfully when imported, as the format is correct.

Setup (please complete the following information):

laura-vuilleumier commented 3 weeks ago

Update on this, removing the ExpirationTime element from each registration allows the import to work. This was definitely not an issue in the past though, why is it happening now? We've had to export, manually remove all the dates, then import to get it working which is less than ideal.

angularsen commented 2 weeks ago

Thank you @laura-vuilleumier !

I had the same problem with bulk delete using NotificationHubJobType.ImportDeleteRegistrations. Removing ExpirationTime elements from the XML worked.

I tried several attempts removing other seemingly redundant data for deletion, but ExpirationTime seems to be the culprit. Not sure if relevant, but all my values were <ExpirationTime>9999-12-31T23:59:59.9999999Z</ExpirationTime>.

Docs should be updated if this is not a bug or will not be fixed: https://learn.microsoft.com/en-us/azure/notification-hubs/export-modify-registrations-bulk

dtap001 commented 2 weeks ago

Hi @laura-vuilleumier, I cannot make the export and then reimport work. I have tried eliminating registration id, expiration date. But there is no obvious error message anywhere. The job simply creates an empty Output.txt and Failed.txt on the destination storage blob container.

Do you have any hint to guide me?

Thanks in advance!

angularsen commented 2 weeks ago

@dtap001 I have not tried this, but I earlier found this SO answer that describes import using: <RegistrationId i:nil="true" />

You may need to specify the i XML namespace too, if not already described in your XML.

I'd give it a try.

dtap001 commented 2 weeks ago

Found it. If there is any error during Job execution. The error wont be printed out. I solved it by invoking the action that I want to run on the notification hub client e.g.: hubClient.CreateRegistrationAsync(dto);

In my case I had invalid chars in one of the registration tag key: '$'

dtap001 commented 2 weeks ago

@angularsen Thanks for the prompt reaction!

lomagdal2 commented 1 day ago

i should clarify here that importing a registration with an installationId tag won't work. we don't currently support importing / exporting installations. it seems the community has the found issue - attempting to import registrations with an expirationtime will result in failure. we're closing this out as the OP has resolved their issue with importing.

angularsen commented 1 day ago

@lomagdal2 Just to clarify once more for future reference.

Not sure about importing, but you can export installations, right?

In my experience, exporting registrations includes installations, since installations seem to be backwards compatible with registrations, but adds some extra tags like $InstallationId:{id}, $UserId:{id} etc. Is my understanding correct?

dtap001 commented 1 day ago

You cannot use $ in the keys of tags when importing. Honestly I don't understand what @lomagdal2 saying, because this workflow is working 100%: