search

microsoft / azure-data-manager-for-energy-experience-lab

Experience Lab is a set of utilities that assist in creating instances of Microsoft Azure Data Manager for Energy, performing data loads, and performing basic management operations.
MIT License
13 stars 7 forks source link

Unable to load data into ADME instance by using TNO Data load Container instance which created as part of the Experience Lab Bicep template #34

Closed sreedharguda closed 1 year ago

sreedharguda commented 1 year ago

Hi Team,

Thank you for your contribution to provide us the Bicep and ARM scripts to create the Experience Lab, ADME instance and other resources. When we ran the TNO data load ACI to load data into the ADME instance its not working and seeing the errors, but when i create TNO data load by using the other repo https://github.com/Azure/osdu-data-load-tno and run it, it works. Please let us know which one we have to load the data into ADME. Thank you.

Regards, Sreedhar

Miller-Kyle commented 1 year ago

Hi @sreedharguda, the fix is in the latest release https://github.com/microsoft/azure-data-manager-for-energy-experience-lab/releases/tag/v0.8

Redeploy experience lab and you'll get the fix

sreedharguda commented 1 year ago

Hi @Miller-Kyle, Thank you for the update. I see one more issue that while i am running the template spec to upload the data to ADME instance getting the access denied error. Please help me how to mitigate this issue.

2023-07-05 06:06:20,558 [Dataload ] [INFO ] args: Namespace(dir='/app/open-test-data/datasets/documents', output='output/loaded-documents-datasets.json', subparser='datasets') 2023-07-05 06:06:20,614 [Dataload ] [INFO ] Amount of data to transfer: 26.22MB 2023-07-05 06:06:20,614 [Dataload ] [INFO ] Request worker count: 50 2023-07-05 06:06:20,614 [Dataload ] [INFO ] Max Chunk size: 32 MB 2023-07-05 06:06:20,654 [Dataload ] [INFO ] Total number of files to upload: 9 2023-07-05 06:06:20,870 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FP.pdf with response <Response [403]> 2023-07-05 06:06:20,884 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/osduka_demo_texts.txt with response <Response [403]> 2023-07-05 06:06:20,977 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov22.pdf with response <Response [403]> 2023-07-05 06:06:21,108 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3031.pdf with response <Response [403]> 2023-07-05 06:06:21,119 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FI.pdf with response <Response [403]> 2023-07-05 06:06:21,121 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_11.pdf with response <Response [403]> 2023-07-05 06:06:21,124 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/prov33.pdf with response <Response [403]> 2023-07-05 06:06:21,149 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/FS12-3003.pdf with response <Response [403]> 2023-07-05 06:06:21,539 [Dataload ] [ERROR ] /files/uploadURL failed for /app/open-test-data/datasets/documents/69_D_CH_13.pdf with response <Response [403]> 2023-07-05 06:06:21,674 [Dataload ] [INFO ] File location map is saved to output/loaded-documents-datasets.json 2023-07-05 06:06:21,690 [Dataload ] [INFO ] Files that could not be uploaded: 9 2023-07-05 06:06:21,690 [Dataload ] [INFO ] ['/app/open-test-data/datasets/documents/FP.pdf', '/app/open-test-data/datasets/documents/osduka_demo_texts.txt', '/app/open-test-data/datasets/documents/prov22.pdf', '/app/open-test-data/datasets/documents/FS12-3031.pdf', '/app/open-test-data/datasets/documents/FI.pdf',