Arash-Sabet
commented
3 weeks ago @blueww Could you please chime in and let me know your thoughts on this issue? Thanks.
Open Arash-Sabet opened 3 weeks ago
Arash-Sabet
commented
3 weeks ago @blueww Could you please chime in and let me know your thoughts on this issue? Thanks.
blueww
commented
3 weeks ago @Arash-Sabet
The above log you shared is from Storage SDK.
From the error I see: 1. the error happens on API BlobContainerClient.GetPropertiesInternal(); 2. the error is client can't connect to 127.0.0.1:10000 successfully.
Would you please:
Arash-Sabet
commented
3 weeks ago @blueww
The debug log file was not created per the following screenshot, and it seems to be a new problem for you guys to solve:
The connection string is: "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;"
There is nothing blocking the request as the entire application used to work before upgrading the azurite and the nuget packages.
I also used --skipApiVersionCheck --loose in the azurite cli but it was not conclusive.
Arash-Sabet
commented
3 weeks ago @blueww any updates? Thanks.
blueww
commented
3 weeks ago @Arash-Sabet We have never see any customer report azurite debug log not generated. So it’s almost not possible that there’s bug blocks azurite debug log generate.
Does the folder you show above on the docker , or on your client machine host docker? The debug log should be generated on the docker, so normally we will map the a folder on docker to a local folder, then generate debug log in the folder, then you can access debug log from local folder.
Another possible reason is azurite not start at all. There are something on your machine blocks azurite from starting.
Arash-Sabet
commented
3 weeks ago @blueww The following failure message is all I have to share. This exception also happens when we do not use sudo or with --loose or --skipApiVersionCheck switches.
sudo azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 -s -l /usr/local/lib/node_modules/azurite -d /usr/local/lib/node_modules/azurite/debug.log & ========================== Starting Command Output =========================== /usr/bin/bash /azp/_work/_temp/5d5dabf2-5efa-43e2-974a-2ea32571caaf.sh /usr/local/lib/node_modules/azurite/dist/src/common/persistence/MemoryExtentStore.js:53 return this._chunks.get(categoryName)?.chunks.get(id); ^
SyntaxError: Unexpected token '.' at wrapSafe (internal/modules/cjs/loader.js:915:16) at Module._compile (internal/modules/cjs/loader.js:963:27) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.
(/usr/local/lib/node_modules/azurite/dist/src/common/ConfigurationBase.js:7:29) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
blueww
commented
2 weeks ago @Arash-Sabet
From the above error, I really have no idea. And I can't repro this. I will try to contact the code owner of "MemoryExtentStore.js" to see if any idea.
One workaround is: you might can switch back to 3.29.0, if you work doesn't depends on change of 3.30.0.
Arash-Sabet
commented
2 weeks ago @blueww We have already had to downgrade the azurite to 3.29.0 and the NuGet packages too.
blueww
commented
2 weeks ago @Arash-Sabet
I get responds from the code owner of "MemoryExtentStore.js", however he seems also don't have much idea. He mentioned:
I don't know. A syntax error may come from a very old version of node running on newer syntax .js file. In memory persistence was introduced in 3.28.0. 3.29.0 works for the user but that also has in-memory persistence. Very confusing.
Arash-Sabet
commented
2 weeks ago @blueww That sounds like an issue in the implementation of azurite. This is a hinderance to us and other people who use azurite within docker. The issue has to be diligently resolved by Microsoft as the owner of this product. The response from the code owner is not satisfying as he must nail down this issue and address it. With this issue around, upgrading azurite NuGet packages becomes impossible unless you guys come up with another workaround.
blueww
commented
2 weeks ago @Arash-Sabet
The problem is the current error log is not enough to get the root cause of the issue. We can't repro this issue, and we don't get other customer report similar issue.
We will need either a repro environment or more error log/trace to investigation this issue. Could you give the very detail steps to repro this issue? Or share a repro environment?
If we could get or setup a repro environment, it would definitely help the investigation.
BTW, could you check the node version of your repro environment?
Arash-Sabet
commented
2 weeks ago @blueww If other customers are not reporting this issue it does not mean that this problem does not exist. We cannot browse the entire internet to see what all people have to say or they may have not stumbled upon this issue yet. The error I shared is a fact from Azurite's error messages log that I collected.
I am unable to share an environment to reproduce this issue. But I hope the following docker file snippet leads your team to the issue:
FROM ubuntu:22.04
RUN DEBIAN_FRONTEND=noninteractive apt-get update
RUN DEBIAN_FRONTEND=noninteractive apt-get upgrade -y
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
apt-transport-https \
apt-utils \
ca-certificates \
curl \
git \
iputils-ping \
jq \
lsb-release \
software-properties-common \
sudo \
zip \
unzip \
tar \
wget
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
RUN wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN dpkg -i packages-microsoft-prod.deb
RUN rm packages-microsoft-prod.deb
RUN sudo apt-get update
RUN sudo apt-get install -y dotnet-sdk-8.0
RUN sudo apt-get -y install nodejs
RUN sudo apt-get install -y nodejs npm
RUN sudo npm install -g azurite
RUN sudo npm fund
RUN sudo mkdir azuriteThere are other commands that I excluded from this docker file deemed unnecessary in this scope. That's all I can provide.
blueww
commented
1 week ago @Arash-Sabet
For we don't get other customer report similar issue, I don't mean the issue not exist, but mean we don't have repro way from other customer, so can only get the way to repro it from you. And no other customer meet this issue, might will make the issue not so high priority compare with other features/issues requested by more customers.
We might can try to repro this with you above script. However, could you give the detail step by step guild on how to use the script to setup the docker, it will definitely make the repro more efficiently.
Arash-Sabet
commented
1 week ago @blueww I would give it a high priority to have it looked into.
To reproduce the issue: I'm sure there are other folks using k8s for their build agents. Please have your team follow the steps outlined per this blog post and include the script I shared when building the k8s's docker image. I guess that blog post is a re-iteration of a Microsoft's document/practice echoed by the author in the context of making a build agent.
Write a C# Xunit method to create a container on the Azurite instance, run it through a build pipeline, and you will observe the issue. This is how you'd normally launch Azurite inside the docker image from Azure DevOps' YAML file:
- task: Bash@3
displayName: Launching Azurite
inputs:
targetType: 'inline'
script: 'sudo azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 -s -l /usr/local/lib/node_modules/azurite -d /usr/local/lib/node_modules/azurite/debug.log &'@Arash-Sabet
Thanks for the sharing! Will look into this when finish the higher priority work in hand and update later.
Which service(blob, file, queue, table) does this issue concern?
blob
Which version of the Azurite was used?
3.30.0
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
npm
What's the Node.js version?
Presumably the latest LTS version
What problem was encountered?
Steps to reproduce the issue?
We are including
azuritein our docker image by the following script to use the docker image as a build agent where we run our integration tests:To launch the included azurite instance, we issue the following command in our build pipeline's YAML file:
'sudo azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 -s -l /usr/local/lib/node_modules/azurite -d /usr/local/lib/node_modules/azurite/debug.log &'
We built the image and deployed to our k8s cluster a few months ago with the azurite version 3.29.0 and the integration tests have been working fine since then. We recently upgraded the
Azure.Storage.Blobsver12.19.1andAzure.Storage.Queuesver12.17.1NuGet packages that our integration tests use to versions12.20.0and12.18.0, but the tests started failing after producing the following error messages despite we upgraded the azurite's instance to version 3.30.0:Although, this failure does not happen on the development laptop where azurite ver 3.30.0 and same latest nuget packages are present.
Have you found a mitigation/solution?
No