Can't override AZURE_ResourceTypes

[oyemeLife]

I can't override the value of the AZURE_ResourceTypes, I do it via environment settings.

I always got all resources.

Name: AZURE_ResourceTypes
Value: {"Patient", "Observation", "Encounter"}

I have run the app, but I also wasn't able to override with the system env the value of the model

 [JsonProperty("resourceTypes")]
        public List<string> ResourceTypes { get; set; } = new List<string>{
                "Account",
                "ActivityDefinition",
                "AdverseEvent",
                "AllergyIntolerance",
                "Appointment",
                "AppointmentResponse",
                "AuditEvent",
                "Basic",
                "Binary",
                "BiologicallyDerivedProduct",
                "BodyStructure",
                "Bundle",
                "CapabilityStatement",
                "CarePlan",
                "CareTeam",
                "CatalogEntry",
                "ChargeItem",
                "ChargeItemDefinition",
                "Claim",
                "ClaimResponse",
                "ClinicalImpression",
                "CodeSystem",
                "Communication",
                "CommunicationRequest",
                "CompartmentDefinition",
                "Composition",
                "ConceptMap",
                "Condition",
                "Consent",
                "Contract",
                "Coverage",
                "CoverageEligibilityRequest",
                "CoverageEligibilityResponse",
                "DetectedIssue",
                "Device",
                "DeviceDefinition",
                "DeviceMetric",
                "DeviceRequest",
                "DeviceUseStatement",
                "DiagnosticReport",
                "DocumentManifest",
                "DocumentReference",
                "EffectEvidenceSynthesis",
                "Encounter",
                "Endpoint",
                "EnrollmentRequest",
                "EnrollmentResponse",
                "EpisodeOfCare",
                "EventDefinition",
                "Evidence",
                "EvidenceVariable",
                "ExampleScenario",
                "ExplanationOfBenefit",
                "FamilyMemberHistory",
                "Flag",
                "Goal",
                "GraphDefinition",
                "Group",
                "GuidanceResponse",
                "HealthcareService",
                "ImagingStudy",
                "Immunization",
                "ImmunizationEvaluation",
                "ImmunizationRecommendation",
                "ImplementationGuide",
                "InsurancePlan",
                "Invoice",
                "Library",
                "Linkage",
                "List",
                "Location",
                "Measure",
                "MeasureReport",
                "Media",
                "Medication",
                "MedicationAdministration",
                "MedicationDispense",
                "MedicationKnowledge",
                "MedicationRequest",
                "MedicationStatement",
                "MedicinalProduct",
                "MedicinalProductAuthorization",
                "MedicinalProductContraindication",
                "MedicinalProductIndication",
                "MedicinalProductIngredient",
                "MedicinalProductInteraction",
                "MedicinalProductManufactured",
                "MedicinalProductPackaged",
                "MedicinalProductPharmaceutical",
                "MedicinalProductUndesirableEffect",
                "MessageDefinition",
                "MessageHeader",
                "MolecularSequence",
                "NamingSystem",
                "NutritionOrder",
                "Observation",
                "ObservationDefinition",
                "OperationDefinition",
                "OperationOutcome",
                "Organization",
                "OrganizationAffiliation",
                "Parameters",
                "Patient",
                "PaymentNotice",
                "PaymentReconciliation",
                "Person",
                "PlanDefinition",
                "Practitioner",
                "PractitionerRole",
                "Procedure",
                "Provenance",
                "Questionnaire",
                "QuestionnaireResponse",
                "RelatedPerson",
                "RequestGroup",
                "ResearchDefinition",
                "ResearchElementDefinition",
                "ResearchStudy",
                "ResearchSubject",
                "RiskAssessment",
                "RiskEvidenceSynthesis",
                "Schedule",
                "ServiceRequest",
                "Slot",
                "Specimen",
                "SpecimenDefinition",
                "StructureDefinition",
                "StructureMap",
                "Subscription",
                "Substance",
                "SubstancePolymer",
                "SubstanceProtein",
                "SubstanceReferenceInformation",
                "SubstanceSpecification",
                "SubstanceSourceMaterial",
                "SupplyDelivery",
                "SupplyRequest",
                "Task",
                "TerminologyCapabilities",
                "TestReport",
                "TestScript",
                "ValueSet",
                "VerificationResult",
                "VisionPrescription"
            };

I tried to set in

AZURE_ResourceTypes={"Device"}
AZURE_ResourceTypes=["Device"]
AZURE_ResourceTypes='Device'

No luck

[evachen96]

Hi @oyemeLife , thanks for submitting this issue. We will investigate and get back to you!

[oyemeLife]

Hi @oyemeLife , thanks for submitting this issue. We will investigate and get back to you!

To address this, I forked the repository to implement changes and override the migrationOptions binding model with an empty list. Since it is not possible to override a list of strings directly, the workaround involves overriding an existing index. For instance, setting AZURE_ResourceTypes__0 = Device will assign "Device" to the first index. This method supports multiple resources by adding settings via AZURE_ResourceTypes__[index].

[evachen96]

Hi @oyemeLife , do you mind sharing more on your use case? Is there a reason that you are only migrating certain resource types? Also, are you able to share which company you are representing?

[oyemeLife]

Hi @oyemeLife , do you mind sharing more on your use case? Is there a reason that you are only migrating certain resource types? Also, are you able to share which company you are representing?

We are currently using two stores. Some resources point to the new FHIR API, while others point to the old one. Our original plan was to migrate resources one by one. I can share the company name privately.

[evachen96]

Hi @oyemeLife, thanks for the info. To clarify - by two stores, is that the older Azure API for FHIR and the new Azure Health Data Services FHIR? And for the migration, is your source FHIR server the Azure API for FHIR, and your destination FHIR server the Azure Health Data Services FHIR? Just want to make sure we understand your FHIR setup correctly.

Also, if you could send me the company name to FHIRMigrationTool@microsoft.com, that would be helpful for our customer tracking, thank you!

[oyemeLife]

Yes, that's correct. Migrating from Azure API for FHIR to Azure Health Data Services FHIR.

[oyemeLife]

The same issue persists when setting the Azure_SurfaceCheckResources environment variable while testing the end-to-end function. Luckily, the SurfaceCheckResources array only uses the patient index for the first element in the migration options, public List<string>? SurfaceCheckResources { get; set; } = new List<string> { "Patient" }; so I can set Azure_SurfaceCheckResources__{INDEX}

This is what I did.

[evachen96]

Hi @oyemeLife , thanks for the info. Can you share your rationale on why you are planning to migrate resource types one by one?

We do not recommend migrating resource types one by one, because many of these resources are interlinked. It may be the case that you migrate data into AHDS FHIR that is supposed to be linked to another resource, but that resource type does not exist in the FHIR server yet. We recommend migrating all the of the resource types, which is why we have provided the superset of resource types.

If you are hoping to either test the migration tool or migrate smaller chunks of data, I would suggest that you explore using smaller chunks (https://github.com/Azure/apiforfhir-migration-tool/tree/main/FHIR-data-migration-tool-docs#export-in-the-migration-tool) by configuring AZURE_ExportChunkDuration, or, you can adjust the AZURE_StartDate to set the first date and continue to start-stop the migration tool as you test the migration tool. This way, resources are still migrated in chronological order, with all the resource types, and the links between the various resources can be maintained.

Looking forward to hear more about your use case. Thanks!

[oyemeLife]

The original plan was to migrate resources one by one. Some of the resources point to new ADHD, some of the resources point to both ADHD and Azure API, and some point to old Azure API. d For example AuditEvent: ✗ The totals are different: OLD: 40840455 NEW: 7689342

Device: OLD: 456 NEW: 0

Practitioner: OLD: 844 NEW: 0

Consent: OLD: 948851 NEW: 186580

I believe then, it should be removed from the readme and env settings to have the ability of overriding the resource types as it's currently not recommended and not supported

[evachen96]

Hi @oyemeLife - I see, yes, we would still recommend migrating all resource types (if you'd like smaller chunks, you can change the dates in the migration tool), and then using the new AHDS FHIR server as your single source of truth. If you migrate resource types one at a time, there is a possibility that the references between resources may get broken.

We will go ahead and update the documentation. Our original intention was to provide all the R4 resource types, and customers could update that list if they, for example, knew that their FHIR server only had a certain subset of these resource types. However, the idea was to still migrate all the resource types that are in a given FHIR server. We will update the documentation to make this more clear!

Other than that, were there any other issues with your migration?

[evachen96]

Closing this issue as we have updated the documentation to remove the wording about customizing the resource types. Please feel free to reopen or create a new issue if there are any other issues with the migration!