Closed Kedar782 closed 2 weeks ago
Hi @Kedar782 , thanks for your question. We'll take a look and get back to you.
Thanks Eva.
On Mon, Jul 22, 2024 at 12:05 PM Eva Chen @.***> wrote:
Hi @Kedar782 https://github.com/Kedar782 , thanks for your question. We'll take a look and get back to you.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2243629042, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVGE5OCC5X7TWQROPRTZNVJYRAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGYZDSMBUGI . You are receiving this because you were mentioned.Message ID: @.***>
Hi @evachen96,
Do you have any update on this?
Hi @Kedar782,
I would recommend testing FHIR Loader on a FHIR server that does not have SMART on FHIR first, so that we can see if there are any issues there and remove other factors. Then, once we validate that works, we can try with a FHIR server that does have SMART on FHIR. Please let me know how that goes!
I will do that now.
But here when I use my existing FHIR server without SMART and try to access it using POSTMAN with client credentials flow I get 401 unauthorized error.
How can i fix it?
I need to give client id and client secret for the fhir loader installation process.
Kedarnath
On Wed, Jul 31, 2024 at 3:49 PM Eva Chen @.***> wrote:
Hi @Kedar782 https://github.com/Kedar782,
- We recommend using the "Deploy to Azure" button to deploy FHIR Loader via Portal deployment. It is located on the main page of the repo https://github.com/microsoft/fhir-loader
image.png (view on web) https://github.com/user-attachments/assets/32f3175f-6a1e-4d8a-8fc0-1388969f2761
- Where do I get the FHIR service service URL, FHIR audience URL? You can find these in Azure Portal in the info for your FHIR server. They should be the same URL, and you can find this in your Azure Portal under Settings > Authentication > Audience
- Also in the installation process it is asking me for the MSI or SP? If using the "Deploy to Azure" button, you can choose between MSI or SP.
I would recommend testing FHIR Loader on a FHIR server that does not have SMART on FHIR first, so that we can see if there are any issues there and remove other factors. Then, once we validate that works, we can try with a FHIR server that does have SMART on FHIR. Please let me know how that goes!
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2261596727, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVBOKPJL2QBVTWKYFWTZPFSZDAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRRGU4TMNZSG4 . You are receiving this because you were mentioned.Message ID: @.***>
I have created a new FHIR Service as you suggested but am still receiving an error on the last tab of the template.
[image: image.png]
On Wed, Jul 31, 2024 at 3:49 PM Eva Chen @.***> wrote:
Hi @Kedar782 https://github.com/Kedar782,
- We recommend using the "Deploy to Azure" button to deploy FHIR Loader via Portal deployment. It is located on the main page of the repo https://github.com/microsoft/fhir-loader
image.png (view on web) https://github.com/user-attachments/assets/32f3175f-6a1e-4d8a-8fc0-1388969f2761
- Where do I get the FHIR service service URL, FHIR audience URL? You can find these in Azure Portal in the info for your FHIR server. They should be the same URL, and you can find this in your Azure Portal under Settings > Authentication > Audience
- Also in the installation process it is asking me for the MSI or SP? If using the "Deploy to Azure" button, you can choose between MSI or SP.
I would recommend testing FHIR Loader on a FHIR server that does not have SMART on FHIR first, so that we can see if there are any issues there and remove other factors. Then, once we validate that works, we can try with a FHIR server that does have SMART on FHIR. Please let me know how that goes!
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2261596727, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVBOKPJL2QBVTWKYFWTZPFSZDAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRRGU4TMNZSG4 . You are receiving this because you were mentioned.Message ID: @.***>
Hi @Kedar782 , I'm not seeing the image, can you upload it again?
Also, please submit a separate issue at https://github.com/Azure-Samples/azure-health-data-and-ai-samples/issues regarding Postman and include details on there.
Hello @evachen96 Do you have any update on this?
Hi @Kedar782 , we have fixed the issue that occurs when using service principal for both the bash script and the Deploy to Azure button. Let me know if that works!
Closing this issue, let me know if there are still any questions!
Hello @evachen96 ,
I'm still receiving error got using button and bash script. I'm using azure health data without smart. I'm using the same client id and client secret which I used for POSTMAN client credentials flow as a SP client ID and client secret in the bash script but I'm getting error. I'm using the same FHIR server and audience URL. It is working in POSTMAN but I'm getting error for the bash script. (I intentionally changed the subscriptionid nad username here in the error I cannot post original here) Error is below:-
When I use button option to deploy the fhir loader project I get below error
{ "code": "DeploymentFailed", "target": "/subscriptions/c4cbd8a0-245d-489c-990b-694ac852/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Resources/deployments/role-assign-fhir", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "InvalidPrincipalId", "message": "A valid principal ID must be provided for role assignment." } ] }
Please guide me what I'm doing wrong. I'm following all your instructions for deploying the fhir loader project.
Thanks for the new information. We will take a look.
Hi @Kedar782 - team has reviewed and would like to provide the following information -
Attaching some screenshots as well to show how the deployment typically looks like using the Deploy to Azure button. We followed the same sequence (first accessed the FHIR service using Postman and used same app registration while deploying the FHIR Loader using Service Principal):
Regards, Kedarnath Shinde
On Mon, Sep 9, 2024 at 1:27 PM Eva Chen @.***> wrote:
Hi @Kedar782 https://github.com/Kedar782 - team has reviewed and would like to provide the following information -
- Deploy to Azure button: The error message that you shared, "A valid principal ID must be provided for role assignment," typically occurs when the "existing service principal" option is selected, but no principal ID is provided. Currently, there is no validation to ensure that this field is filled. Please ensure that the correct service principal is specified, along with the appropriate secret value.
- Deployment using Bash script: We assume the Bash script is being executed in Azure Cloud Shell – Bash Shell after logging in with valid user credentials. The "assignee" parameter in the az role assignment command refers to the user account executing the script. Could you verify if the script is being run using the same user account that was used to log into Azure Cloud Shell – Bash Shell? Additionally, ensure the script is being executed within the same tenant as that user.
Attaching some screenshots as well to show how the deployment typically looks like using the Deploy to Azure button. We followed the same sequence (first accessed the FHIR service using Postman and used same app registration while deploying the FHIR Loader using Service Principal):
btndeploy1.png (view on web) https://github.com/user-attachments/assets/4b771886-4787-4694-9853-066ac4f25ea1 btndeploy2.png (view on web) https://github.com/user-attachments/assets/2db749ca-55f8-400a-9ae4-8b447e60f05e btndeploy3.png (view on web) https://github.com/user-attachments/assets/53f952fd-fc14-41c9-903e-b03f863647dd btndeploy4.png (view on web) https://github.com/user-attachments/assets/33091269-9533-4638-85dd-6c3bc51f01d7 btndeploy5.png (view on web) https://github.com/user-attachments/assets/834182fa-2587-48d2-8de6-528d12228ec1 btndeploy6.png (view on web) https://github.com/user-attachments/assets/5a17a02b-ef4f-4be5-965f-71fbb0a74f17
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2339023956, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVETRWNLIRGCN2H36R3ZVYADHAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZZGAZDGOJVGY . You are receiving this because you were mentioned.Message ID: @.***>
@evachen96
Do you have any update on this?
Please let me know if you need any more information from me.
Hi Kedar - the team is still working on trying to reproduce your issue. In the meantime, we have added hopefully more clear instructions on how to deploy the FHIR loader using Azure Portal and more clear prompts in the Bash script in the latest PR (https://github.com/microsoft/fhir-loader/pull/73). Please try https://github.com/microsoft/fhir-loader/blob/main/docs/portaldeployment.md and https://github.com/microsoft/fhir-loader/blob/main/scripts/deployFhirBulk.bash and let us know if you're able to resolve your issue.
@evachen96
I tried again with the detail description documents you shared but still getting the same errors.
2.Using the script I'm getting error which is
Cannot find user or service principal in graph database for 'live.com#XXXXXi@XXXXXXX.com'. If the assignee is an appId, make sure the corresponding service principal is created with 'az ad sp create --id live.com#XXXXXi@XXXXXXX.com'.
@Kedar782 , could we get the following details?
@evachen96, 1.I tried again using button to send you the error screenshot adn it worked this time. I did not get any error. When I try to add a bundle file in bundles container on a storage container that file is not going in bundlesprocessed or bundleserr container. So data is not going in the fhir data service. Please note that I manually created bundlesprocessed and bundleserr container. Installation process only created the bundles container.
Please let me knoe if you need any more information from us.
@evachen96
I added a bundle file in the bundles container but still it is in bundles container.
Hello Eva,
Do you have any update? Please let me know if you need any information from me.
Regards, Kedarnath
On Wed, Sep 18, 2024 at 1:02 PM Eva Chen @.***> wrote:
@Kedar782 https://github.com/Kedar782 , could we get the following details?
- A complete screenshot and additional details regarding which resource failed during deployment when using the "Deploy to Azure" button.
- Regarding the Bash script error: Is the user (assignee) shown in the error is an invited user created via an email invitation, or if the user was created directly within Entra ID? Thanks!
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2359303236, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVB7Q7666PMENQRTMWTZXHL3PAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNJZGMYDGMRTGY . You are receiving this because you were mentioned.Message ID: @.***>
Hi @Kedar782 -
Based on the information provided in the GitHub issue, we recommend the following steps:
Deploy to Azure Button:
It appears that the deployment was completed, but the setup is corrupted and wasn’t properly configured during the process. Can you perform a fresh setup using a new resource group?
Bash Script:
To determine if the user is a native or invited user in the directory, you can navigate to Entra ID > Users and check the User Principal ID of the account used for deployment: A native user in the tenant is typically formatted like: XXXXXX@XXXXXX.com An invited user is formatted like: XXXX#EXT#@XXXXXX.XXXX.com For now, we suggest that you create a new user account directly in Entra ID, assign the necessary roles, and attempt the deployment again.
Thanks!
@evachen96
I tried to deploy it on brand new resource and now I'm getting the same error again.(Using the deploy to Azure button)
Please let me know if you need any more information from me.
I will try it with bash script and let you know.
@evachen96
I have only one user in my azure ad account and i can pass all inferno test using this user. Also, I can insert data in fhir data service using the postman with this user. It does has a EXT in it but it's User Type is a Member. I added FHIR Loader on my old fhir server (Azure API for FHIR) using the same user. I have deadline for this project which is approaching very fast. Can you please help me with this? I have tried everything.
@evachen96
I created a new user in azure AD and assign a role of owner for this user in the subscription where my fhir data service is located. I also assigned a role of owner in my fhir data service resource for this user. I also assigned a role of fhir data contributor for this user in my fhir data service. Then using the bash script I successfully deployed the fhir Loader project. In the deployment process I selected SP in MSI/SP option and I used the client id and clinet secret of the app where I can successfully insert and retirve the data using the postman. When I upload a bundle in the bundles container I get error response in the bundleserr container stating Authentication failed. If I use the same client in POSTMAN I can insert data.
Please advised.
Hi @Kedar782 , comments from the team:
Deploy to Azure Button: It appears that the deployment failed during role assignment to the storage account, as the storage account was not found. We couldn’t reproduce the issue on our end. However, to assist us in troubleshooting, could you provide the following details?
Bash Script: Could you provide additional details on how the user was created or invited? We will replicate the user creation process and investigate further. In the meantime, we will continue to investigate this issue.
Hello @evachen96 ,
Right now I have two users in my azure account. Originally I had only one user in my account which was of the type Guest. In Entra ID this user has a role of Global Administrator. In subscription level this user has a role of Owner.
After you asked me to create a brand new member user, I created a new Member user in Entra ID(without inviting) and it has a role of Application Administrator and Application Developer role in Entra ID. It has a Owner role at the subscription level.
Both of this users has owner role on the Fhir data service resource group and FHIR Data contributor role on the FHIR Data Service.
If I use both above users on the deploy to azure button I get the same error that I posted.(role assignment to the storage account). Yes using both users the storage account was created in the resource group.
Bash Script:
If I use bash script using the new user I created (Member) I can able to deploy the FHIR Loader but when I add a bundle in bundles container I get error for that bundle stating authentication failed. I'm using Service Principle for the deployment process and I'm using the same application that I use for POSTMAN clinet credential flow to insert data into my fhir server and it works fine.
If I use GUEST user I get error in the deployment process stating role assignement failed. I posted this error in the issue.
Please let me know if you need any more information from me.
We tried to reproduce the issue in-house on multiple tenants but couldn’t replicate it, nor have we encountered this specific problem before. We successfully deployed the FHIR Loader sample using both a bash script and the "Deploy to Azure" button with a new Member user who has the same roles mentioned by you.
You may experience certain deployment issues that could be specific to Azure infrastructure and may not be persistent. We recommend redeploying the sample using the redeploy button if you encounter any problems. Also, please check if the following role assignments are granted to the new Member user in your environment:
Note: Ensuring all prerequisites are met and following the documented deployment steps correctly should make it work.
@evachen96 Can you please let me know what storage account I need to assign a role oif owner? As I said before, I already have owner role on the FHIR data service resource. So every storage account in this resource group has a role of owner for the user.
@evachen96 @erikhoward I confirmed that I have above roles assigned to the user. Still I'm getting the same error again and again.
@erikhoward @evachen96
Do you have any update on this issue?
I tried to deploy it again and I'm getting the same error.
{ "code": "DeploymentFailed", "target": "/subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-XXXXc852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Resources/deployments/role-assign-storage", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "ResourceNotFound", "message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix" }, { "code": "ResourceNotFound", "message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix" }, { "code": "ResourceNotFound", "message": "The Resource 'Microsoft.Storage/storageAccounts/bulkvw67stor' under resource group 'calmedfhirproduction12-rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix" } ] }
Error is saying that storage account bulkvw67stor is not found under the resource calmedfhirproduction12-rg which is the resource for fhir data service. This storage account bulkvw67stor is created under new resource I create in the custom deployment process for the fhir loader app.
Hi @Kedar782 -
From the screenshot, it appears that the resource group used for deployment differs from the resource group where the error occurred. The FHIR Loader template deploys the Storage Account and assigns roles within the resource group specified during deployment. This suggests that an incorrect configuration might be causing the error you're encountering. We recommend reviewing the deployment configuration you're using. Please refer to the deployment guide for detailed instructions. Additionally, if possible, please share the exact steps and screenshots of your deployment process so we can attempt to reproduce the issue on our side.
Additionally, can you please try with completely new setup including creating new FHIR service also? Create new resource group > Create FHIR service in it > Use this resource group for FHIR loader resources deployment. So finally, FHIR service and all other resource would be in same resource group. The FHIR Loader should be deployed in the same resource group as the FHIR server.
@evachen96 @erikhoward
I will try to explain.
I'm creating a new resource group in the deployment process called bulk121212. calmedfhirproduction12-rg is the resource group where my fhir service is at. I'm deploying fhir bulk loader in the brand new resource bulk121212.
Is it wrong? I alos tried to deploy it in the fhir service resource and i'm still experiencing issues.
Please let me know if anything is wrong in the deployment process.
Got the same error again.
Hi @Kedar782 , you should deploy the FHIR Loader in the same resource group that your FHIR service is located in.
@evachen96
Is it mandatory?
@evachen96 @erikhoward
If I try to deploy it in the same resource group which calmedfhirproduction12-rg(Resource group for the fhir service). I get different error now which is below
{ "code": "Endpoint validation", "message": "Destination endpoint not found. Resource details: resourceId: /subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-694ac852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Storage/storageAccounts/bulki4x4stor. Resource should pre-exist before attempting this operation. Activity id:1731b373-f64f-4966-844f-e1af4639f103, timestamp: 10/3/2024 7:14:33 PM (UTC)." }
{ "code": "Endpoint validation", "message": "Destination endpoint not found. Resource details: resourceId: /subscriptions/c4cbd8a0-XXXX-XXXX-XXXX-694ac852f414/resourceGroups/calmedfhirproduction12-rg/providers/Microsoft.Storage/storageAccounts/bulki4x4stor. Resource should pre-exist before attempting this operation. Activity id:fa0834c9-7a0c-462d-acd9-0292be4f5968, timestamp: 10/3/2024 7:14:33 PM (UTC)." }
Please advice.
Hi @Kedar782, we are investigating the error message that you shared to see why that error may be coming up when you use Deploy to Azure button in the existing resource group where the FHIR service is located. In the meantime, can you try the new setup process from scratching following the steps below and ensuring both the FHIR service and FHIR Loader are deployed in the same resource group?
Entra ID role: Global Administrator
Subscription level: Owner
Thanks!
@evachen96
Can you please give me link to the documentation to publish HDS workspace and FHIR service? Last time Ideployed it using the ONC g10 instructions from the azure ai samples repo.
Kedarnath
Hi @Kedar782 - you can deploy AHDS workspace and FHIR service directly from Azure Portal. Please note that you will need to deploy a AHDS workspace first, and then inside of the AHDS workspace, deploy a FHIR service. Here is the info: https://learn.microsoft.com/en-us/azure/healthcare-apis/healthcare-apis-quickstart (Creating AHDS workspace in Azure Portal) https://learn.microsoft.com/en-us/azure/healthcare-apis/fhir/deploy-azure-portal (Deploy FHIR service in the new AHDS workspace that you just created).
Hope that helps!
@evachen96
It worked.
Why it is not working on my old fhir data service? My instinct is saying because I do not use SMART on FHIR for this new FHIR data service.
Can you please let me know what I'm doing wrong?
Kedarnath
Hi @Kedar782 - we are trying to investigate why that might be the case and will get back to you if we find anything.
@erikhoward @evachen96
Did you guys find anything? Have you guys tried this with the server which has ONC smart on fhir?
Please guide me. I have a deadline and I'm working on this from long time.
Hi @Kedar782 , we are still working on it and will let you know when we have an update.
Could you able to run fhir loader on azure fhir data service with SMART on fhir?
On Fri, Oct 18, 2024 at 12:56 PM Eva Chen @.***> wrote:
Hi @Kedar782 https://github.com/Kedar782 , we are still working on it and will let you know when we have an update.
— Reply to this email directly, view it on GitHub https://github.com/microsoft/fhir-loader/issues/70#issuecomment-2423141960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU43YVD6I7D2VBDZXONZCP3Z4FRVXAVCNFSM6AAAAABLI4WH6KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRTGE2DCOJWGA . You are receiving this because you were mentioned.Message ID: @.***>
@Kedar782 We do not support SMART on FHIR and FHIR Loader running on the same FHIR server, as SMART on FHIR only supports reads, not writes (see note here).
However, if you choose to still use SMART on FHIR and FHIR Loader together on the same FHIR server, you would need to set up FHIR Loader to interact with the FHIR server directly, not through SMART on FHIR. So, when you set up FHIR Loader, you would need to use the normal audience URL of the FHIR server, not the APIM endpoint that is set up with SMART on FHIR. We have provided screenshot on how to get the FHIR audience on the other Github issue.
We have also fixed the issue you brought up earlier where deploying FHIR Loader in a different resource group than the FHIR server using Service Principal was giving problems in this PR https://github.com/microsoft/fhir-loader/pull/75
@erikhoward @evachen96
I tried to create a brand new fhir server and then I added fhir loader project on it. It is working. However, after that I need to add smart on fhir on my server and pass the inferno test. I followed every step from this. https://github.com/Azure-Samples/azure-health-data-and-ai-samples/blob/main/samples/patientandpopulationservices-smartonfhir-oncg10/docs/deployment.md
But my first test is failing after it opens a context app stating 502 bad gateway.
So I have two scenarios with two fhir server(Please note that I tried above after long time because I have not acheived anything)
and the second scenario for which I created this issue.
I have a smart on fhir server on second fhir data service and when I try to add fhir loader without the smart url I get error that I shared in this issue.
@evachen96 @erikhoward
Hello Eva,
I tried to create a new fhir data service and then I' trying to add smart on fhir it for ONC. But I'm getting error for it. I created a new issue for it. Please look through it. If it works i can add a new fhir loader sample on it.
https://github.com/Azure-Samples/azure-health-data-and-ai-samples/issues/247
It is long time I'm working on this. I'm very afraid for our deadline. Please advise.
Adding answer here and closing this issue as we have confirmed over email that the following resolves the issue:
Team has investigated and resolved the issue. Ask from you is to deploy FHIR loader with latest update Also with this new deployment, if you encounter failure of ndjsoncreated or bundlecreated deployments, please follow the steps below:
Go to the resource group where you are trying to deploy the FHIR Loader. Navigate to Settings > Deployments. Select the most recent failed deployment. Click on the Redeploy button as shown below.
The Custom Deployment Wizard will open with pre-populated values. Fill in the required fields correctly, as indicated in the screenshot below.
Click on Review + Create.
Please check if the redeployment process resolves the deployment issue.
Post Deployment actions:
After deployment completes successfully, please verify below configurations before using FHIR Loader functionality.
Go to resource group where FHIR Loader is deployed. Go to Function app resource Navigate to Settings > Environment Variables, check values of below variables are correct FS-CLIENT-ID = Service principal Client ID FS-SECRET = Service principal secret FS-RESOURCE = FHIR Service Audience URL. (FHIR Service > Settings > Authentication > Audience) FS-URL = FHIR Server URL ( FHIR Service > Overview > FHIR metadata endpoint without ‘/metadata’)
Once above steps are completed and validated try uploading the bundle in required container.
I want to add a FHIR Loader project on my new FHIR Data service. https://github.com/microsoft/fhir-loader
For that, can you please guide me with URLs.
WHere do I get the FHIR service service URL, FHIR audience URL? Also in the installation process it is asking me for the MSI or SP?
If I decide to go with the SP then how can I create a SP on my fhir service for the bulk loader. I'm using the FHIR service for ONC g10 and We will be adding new data every day through the bulk data in our FHIR service and clients can access this data using the standalone patient apps and EHR apps.