search

Azure / azure-storage-fuse

A virtual file system adapter for Azure Blob storage
Other
659 stars 207 forks source link

blobfuse2 doesn't list all folders in a directory #942

Closed Mmdixon closed 1 year ago

Mmdixon commented 1 year ago

Which version of the blobfuse was used?

blobfuse2 version 2.0.0-preview.3

Which OS (please include version) are you using?

Ubuntu 20.04

What problem was encountered?

Doing ls in a mount directory doesn't list all the folders. But if I ls on a one of those folders I know exists it works. Notice the dedupe folder is the hidden folder.

$ ls /blob/cache/CC
2021-25
$ ls /blob/cache/CC/dedupe/
tokenizer  v0.1.0  v0.2.0  v0.2.1
$ ls /blob/cache/CC
2021-25

Have you found a mitigation/solution?

The files exist but can't list them.

By default, blobfuse logs errors to syslog. If this is relevant, is there anything in the syslog that might be helpful?

Click for syslog ``` # ls /blob/cache/CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (415)]: BlockBlob::GetAttr : name CC Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000 Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_INFO [azstorage.go (284)]: AzStorage::StreamDir : Unblocked List API Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/, marker Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with marker for Path CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 13 21:03:22 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 1 # ls /blob/cache/CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC/dedupe Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC/dedupe Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (415)]: BlockBlob::GetAttr : name CC/dedupe Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/dedupe/, offset 0, count 5000 Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/dedupe/, marker Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 2 objects with marker for Path CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/dedupe/, offset 2, count 5000 Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/dedupe/, marker 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4xLjAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4xLjAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/dedupe/, offset 3, count 5000 Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/dedupe/, marker 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4yLjAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4yLjAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/dedupe/, offset 4, count 5000 Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/dedupe/, marker 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4yLjEwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!84!MDAwMDE3IUNDL2RlZHVwZS92MC4yLjEwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/dedupe/ Oct 13 21:03:57 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/dedupe/, handle: 2 # ls /blob/cache/CC/ Oct 13 21:09:32 turing-ws312 blobfuse2[263529]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC Oct 13 21:09:32 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC Oct 13 21:09:32 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (415)]: BlockBlob::GetAttr : name CC Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/ Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000 Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/, marker Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with marker for Path CC/ Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [block_blob.go (454)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 13 21:09:33 turing-ws312 blobfuse2[263529]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 3 ```

If relevant, please share your mount command.

allow-other: true

logging:
  level: log_debug

components:
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  attribute-expiration-sec: 120
  entry-expiration-sec: 120
  negative-entry-expiration-sec: 240

file_cache:
  path: /tmp/blob/cache
  timeout-sec: 86400
  allow-non-empty-temp: true
  cleanup-on-start: true

attr_cache:
  timeout-sec: 240
vibhansa-msft commented 1 year ago

If your account is a non-HNS account, blobfuse will look for special directory marker files in the container. These are required by blobfuse to identify a directory. If data was uploaded to container through any means which did not create these files, blobfuse will fail to identify them. We have already committed the fix for this as part of #927. If it's possible for you to locally build blobfuse you can take top of the "main" branch and try it out. We are planning to release preview-4 version by end of this month, which shall have the fix.

Mmdixon commented 1 year ago

Ok I might try the main branch then. Previously, when using blobfuse v1.4.5 it was able to list the directories on the same account, without HNS enabled.

Mmdixon commented 1 year ago

@vibhansa-msft I tried on the main branch blobfuse2 version: 2.0.0-preview.4 and added

azstorage:
  virtual-directory: true

to the config. But I am still not seeing the "missing" directories?

gapra-msft commented 1 year ago

Hi @Mmdixon, could you please describe a minimal way to structure the storage account so we can reproduce your issue? I attempted to replicate the issue with no luck so far.

vibhansa-msft commented 1 year ago

Can you also share mount command and config you used for 1.4.5 so that we can compare, in case we are missing some options with blobfuse2.

Mmdixon commented 1 year ago

@gapra-msft it's a Premium LRS BlockBlobStorage account. There are multiple folders each with a couple of subfolders each with ~64K files.

the v1 config file just has the account/container/key in it

$ ./blobfuse2 --version
blobfuse2 version 2.0.0-preview.4
$ ./blobfuse2 mount /blob/cache --config-file=~/code/BlobfuseConfigs/turingitp2Config.yaml
$ ls -l /blob/cache/CC
drwxrwxrwx 2 madixon madixon 4096 Sep 13  2021 2021-25
$ blobfuse --version
blobfuse 1.4.5
$ blobfuse /blob/data1 --config-file=/home/madixon/code/BlobfuseConfigs/turingitp2Config.cfg --tmp-path=/tmp/blob/data1/ --file-cache-timeout-in-seconds=86400 --use-attr-cache=true -o attr_timeout=120 -o entry_timeout=120 -o negative_timeout=240
$ ls -l /blob/data1/CC
drwxrwx--- 2 madixon madixon       4096 Sep 13  2021 2021-25
drwxrwx--- 2 madixon madixon       4096 Sep 11  2021 cache
drwxrwx--- 2 madixon madixon       4096 Jul 30 23:02 cache_conv
drwxrwx--- 2 madixon madixon       4096 Apr 20 01:30 dedupe
...
etc

listing in the blobfuse v1 mount for the first time does take some time compared to blobfuse2 (many blobs in this container).

Mmdixon commented 1 year ago

updated syslog since switching to main (the original one was from the release). Can see BlockBlob::getAttrUsingList gets called.

click for syslog ``` # ls /blob/cache/CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (505)]: BlockBlob::GetAttr : name CC Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (452)]: BlockBlob::getAttrUsingList : name CC Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC, marker Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (478)]: BlockBlob::getAttrUsingList : Item 0 Blob CC Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000 Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_INFO [azstorage.go (284)]: AzStorage::StreamDir : Unblocked List API Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with marker for Path CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 1 # ls /blob/cache/CC/ again for cache hit Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_DEBUG [attr_cache.go (474)]: AttrCache::GetAttr : CC served from cache Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/ Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000 Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with marker for Path CC/ Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 17 20:17:21 turing-ws312 blobfuse2[377835]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 2 ```
vibhansa-msft commented 1 year ago 
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with  marker for Path CC/
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/
Oct 17 20:15:49 turing-ws312 blobfuse2[377835]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 1

As per above logs, in first iteration we tried to get list of blobs on "cc/" path and we got just one object out. In next iteration we tried to get more, and we did not get any response from server. This is kind of server saying there are no elements on this path.

vibhansa-msft commented 1 year ago

Kindly share your v1 mount command and v2 config file.

Mmdixon commented 1 year ago

Kindly share your v1 mount command and v2 config file.

Did I not do that? v2 config file is in the original post + the virtual-directory entry. the v1 mount command is in this comment https://github.com/Azure/azure-storage-fuse/issues/942#issuecomment-1281332953

vibhansa-msft commented 1 year ago 
azstorage:
  sdk-trace: true

Can you add this to your v2 config towards the end and recollect the logs. As I mentioned earlier from storage it appears we are not getting the list of blobs. Just want to validate what request we are sending and what response we are getting.

Mmdixon commented 1 year ago

@vibhansa-msft here is the logs with sdk-trace: true

Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC
Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC
Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (505)]: BlockBlob::GetAttr : name CC
Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (452)]: BlockBlob::getAttrUsingList : name CC
Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC, marker
Oct 18 14:12:42 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> OUTGOING REQUEST (Try=1)#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&maxresults=5000&prefix=CC&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [259fde4b-6324-4515-7d59-6f534ef4fb8c]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:42 GMT]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> REQUEST/RESPONSE (Try=1/63.717915ms, OpTime=236.11542ms) -- RESPONSE SUCCESSFULLY RECEIVED#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&maxresults=5000&prefix=CC&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [259fde4b-6324-4515-7d59-6f534ef4fb8c]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:42 GMT]#012   --------------------------------------------------------------------------------#012   RESPONSE Status: 200 OK#012   Content-Type: [application/xml]#012   Date: [Tue, 18 Oct 2022 18:12:42 GMT]#012   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012   X-Ms-Client-Request-Id: [259fde4b-6324-4515-7d59-6f534ef4fb8c]#012   X-Ms-Request-Id: [3027bc8d-201e-000f-141d-e3b840000000]#012   X-Ms-Version: [2020-08-04]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (478)]: BlockBlob::getAttrUsingList : Item 0 Blob CC
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [azstorage.go (284)]: AzStorage::StreamDir : Unblocked List API
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> OUTGOING REQUEST (Try=1)#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&maxresults=5000&prefix=CC%!F(MISSING)&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [3919332c-f2fb-4c2d-5573-cd0a047c28ed]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:43 GMT]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> REQUEST/RESPONSE (Try=1/51.965242ms, OpTime=52.115184ms) -- RESPONSE SUCCESSFULLY RECEIVED#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&maxresults=5000&prefix=CC%!F(MISSING)&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [3919332c-f2fb-4c2d-5573-cd0a047c28ed]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:43 GMT]#012   --------------------------------------------------------------------------------#012   RESPONSE Status: 200 OK#012   Content-Type: [application/xml]#012   Date: [Tue, 18 Oct 2022 18:12:42 GMT]#012   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012   X-Ms-Client-Request-Id: [3919332c-f2fb-4c2d-5573-cd0a047c28ed]#012   X-Ms-Request-Id: [3027bc9d-201e-000f-211d-e3b840000000]#012   X-Ms-Version: [2020-08-04]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with  marker for Path CC/
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> OUTGOING REQUEST (Try=1)#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&marker=2%21MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%!F(MISSING)&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [af773a45-f7b4-41db-7cb1-854f8aa44681]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:43 GMT]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_INFO [utils.go (231)]: ==> REQUEST/RESPONSE (Try=1/51.303861ms, OpTime=51.483544ms) -- RESPONSE SUCCESSFULLY RECEIVED#012   GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%!F(MISSING)&include=metadata&marker=2%21MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%!F(MISSING)&restype=container&timeout=3601#012   Authorization: REDACTED#012   User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012   X-Ms-Client-Request-Id: [af773a45-f7b4-41db-7cb1-854f8aa44681]#012   X-Ms-Version: [2020-08-04]#012   x-ms-date: [Tue, 18 Oct 2022 18:12:43 GMT]#012   --------------------------------------------------------------------------------#012   RESPONSE Status: 200 OK#012   Content-Type: [application/xml]#012   Date: [Tue, 18 Oct 2022 18:12:42 GMT]#012   Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012   X-Ms-Client-Request-Id: [af773a45-f7b4-41db-7cb1-854f8aa44681]#012   X-Ms-Request-Id: [3027bca2-201e-000f-261d-e3b840000000]#012   X-Ms-Version: [2020-08-04]
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/
Oct 18 14:12:43 turing-ws312 blobfuse2[385334]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 1
vibhansa-msft commented 1 year ago

Can you collect logs for v1 while you are listing and share that too, just to compare on what is going wrong here.

Mmdixon commented 1 year ago

@vibhansa-msft Some v1 logs

click to see v1 logs ``` Oct 19 11:14:14 turing-ws312 blobfuse[393029]: azs_init ran Oct 19 11:14:14 turing-ws312 blobfuse[393029]: ** Post fork authentication for older libcurl version Oct 19 11:14:14 turing-ws312 blobfuse[393029]: Authenticating using account key Oct 19 11:14:14 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=1&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:14 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : e7fbd543-201e-004b-48cd-e3642c000000 Oct 19 11:14:14 turing-ws312 blobfuse[393029]: Successfully Authenticated! Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC is not in the local cache during getattr. Oct 19 11:14:46 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:46 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1f264abe-a01e-005a-27cd-e35337000000 Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC continuation: 2!64!MDAwMDAzIUNDMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/ Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC, representing a directory, found during get_attr. Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 46: azs_readdir called with path = /CC Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 121: Directory /tmp/blob/data1/root/CC/ not found in file cache during readdir operation for /CC. Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = , prefix = CC/ Oct 19 11:14:46 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:46 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : f949eaf6-201e-0026-74cd-e3ce02000000 Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 2 items retreived in 1 iterations. Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:47 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:46 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : b5a612e0-601e-004c-0bcd-e392a9000000 Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-, prefix = CC/ Oct 19 11:14:47 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : ea3047f3-a01e-0051-69cd-e34b43000000 Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh, prefix = CC/ Oct 19 11:14:47 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1393a1cb-a01e-0073-6dcd-e32575000000 Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 4 items retreived in 4 iterations. Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-, prefix = CC/ Oct 19 11:14:47 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 765dfcaf-501e-006d-12cd-e3ff98000000 Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 6 items retreived in 5 iterations. Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-, prefix = CC/ Oct 19 11:14:48 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 3fefe223-d01e-0055-5ccd-e3bec1000000 Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 11 items retreived in 6 iterations. Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:48 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:48 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 5adcfc95-e01e-0019-3ecd-e379de000000 Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 13 items retreived in 7 iterations. Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh, prefix = CC/ Oct 19 11:14:48 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:48 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 7169310f-301e-0076-0bcd-e3d10a000000 Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 14 items retreived in 8 iterations. Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh, prefix = CC/ Oct 19 11:14:48 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:48 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 31eec50e-601e-006e-38cd-e3fc9f000000 Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ--, prefix = CC/ Oct 19 11:14:48 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:48 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 48dd81c0-001e-0075-09cd-e3d20d000000 Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 16 items retreived in 10 iterations. Oct 19 11:14:48 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:48 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : e7fbe5f1-201e-004b-12cd-e3642c000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1f264cf9-a01e-005a-52cd-e35337000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 19 items retreived in 12 iterations. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : f949eb7b-201e-0026-62cd-e3ce02000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 22 items retreived in 13 iterations. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : b5a61352-601e-004c-73cd-e392a9000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 30 items retreived in 14 iterations. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : ea304960-a01e-0051-4fcd-e34b43000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 32 items retreived in 15 iterations. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs. Container = vc-data-blob, delimiter = /, continuation = 2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1393a229-a01e-0073-31cd-e32575000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/2021-25 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/2021-25 is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/2021-25&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 765dfd02-501e-006d-50cd-e3ff98000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/2021-25 continuation: 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/2021-25 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/2021-25 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/2021-25/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/2021-25, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/CC_Langs Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/CC_Langs is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/CC_Langs&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 3fefe2e5-d01e-0055-0acd-e3bec1000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/CC_Langs continuation: 2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/CC_Langs Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/CC_Langs Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/CC_Langs/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/CC_Langs, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/cache Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/cache is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/cache&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 5adcfcff-e01e-0019-20cd-e379de000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/cache continuation: 2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/cache Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/cache Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/cache/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/cache, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/cache_conv Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/cache_conv is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/cache_conv&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 7169313a-301e-0076-32cd-e3d10a000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/cache_conv continuation: Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/cache_conv Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/cache_conv Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/cache_conv/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/cache_conv, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/classifier.bin Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/classifier.bin is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 250: File Prop Cache : size is 9486891125 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/cleaned Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/cleaned is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/cleaned&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 31eec5b3-601e-006e-58cd-e3fc9f000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/cleaned continuation: 2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/cleaned Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/cleaned Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/cleaned/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/cleaned, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/dedupe Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/dedupe is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/dedupe&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 48dd81e8-001e-0075-28cd-e3d20d000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/dedupe continuation: 2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/dedupe Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/dedupe Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/dedupe/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/dedupe, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/download Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/download is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/download&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : e7fbe628-201e-004b-41cd-e3642c000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/download continuation: 2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/download Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/download Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/download, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/extract Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/extract is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/extract&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1f264d77-a01e-005a-4acd-e35337000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/extract continuation: 2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/extract Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/extract Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/extract/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/extract, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/filter Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/filter is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/filter&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : f949eba7-201e-0026-0ccd-e3ce02000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/filter continuation: Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/filter Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/filter Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/filter, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/indexes Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/indexes is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/indexes&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : b5a6137a-601e-004c-0acd-e392a9000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/indexes continuation: 2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/indexes Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/indexes Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/indexes/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/indexes, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/sampled Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/sampled is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/sampled&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : ea3049b3-a01e-0051-1ecd-e34b43000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/sampled continuation: 2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/sampled Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/sampled Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/sampled/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/sampled, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/language_modeling Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/language_modeling is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/language_modeling&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1393a23c-a01e-0073-44cd-e32575000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/language_modeling continuation: Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/language_modeling/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/language_modeling/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/language_modeling, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/sampled2 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/sampled2 is not in the local cache during getattr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/sampled2&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 765dfd17-501e-006d-65cd-e3ff98000000 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/sampled2 continuation: Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/sampled2 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/sampled2 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/sampled2/ Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/sampled2, representing a directory, found during get_attr. Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/sampled3 Oct 19 11:14:49 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/sampled3 is not in the local cache during getattr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/sampled3&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:49 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 3fefe2f5-d01e-0055-18cd-e3bec1000000 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/sampled3 continuation: Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/sampled3 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/sampled3 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/sampled3/ Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/sampled3, representing a directory, found during get_attr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/sharded Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/sharded is not in the local cache during getattr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/sharded&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:50 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 5adcfd2f-e01e-0019-4acd-e379de000000 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/sharded continuation: Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/sharded Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/sharded Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/sharded/ Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/sharded, representing a directory, found during get_attr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/temp Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/temp is not in the local cache during getattr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/temp&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:50 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 71693161-301e-0076-57cd-e3d10a000000 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/temp continuation: 2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/temp Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/temp Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/temp/ Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/temp, representing a directory, found during get_attr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 120: azs_getattr called with path = /CC/trafilatura-extract Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 208: Object /tmp/blob/data1/root/CC/trafilatura-extract is not in the local cache during getattr. Oct 19 11:14:50 turing-ws312 blobfuse[393029]: ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/trafilatura-extract&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:50 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 31eec61b-601e-006e-3ccd-e3fc9f000000 Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 273: In azs_getattr list_segmented_item do loop blob prefix: CC/trafilatura-extract continuation: 2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 0 file CC/trafilatura-extract Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 317: In azs_getattr found blob in list file CC/trafilatura-extract Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 294: In azs_getattr list_segmented_item 1 file CC/trafilatura-extract/ Oct 19 11:14:50 turing-ws312 blobfuse[393029]: Function azs_getattr, in file /home/vsts/work/1/s/blobfuse/utilities.cpp, line 343: Blob /CC/trafilatura-extract, representing a directory, found during get_attr. ```
Mmdixon commented 1 year ago

/ in the URL appear as %!F(MISSING) in v2?

vibhansa-msft commented 1 year ago

Yes I saw that too, let me investigate if that is something playing a role here or just a logging typos.

vibhansa-msft commented 1 year ago

Ok, this is just a logging issue and not the URL. If there is anything in string with '%' logging is not able to escape that properly. So this is not the root-cause for the issue we are trying to trace.

vibhansa-msft commented 1 year ago

I have a theory here which v1 log supports. Need to check if v2 does not handle this situation.

Oct 19 11:14:46 turing-ws312 blobfuse[393029]:  ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:46 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : f949eaf6-201e-0026-74cd-e3ce02000000
Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 2 items retreived in 1 iterations.
Oct 19 11:14:46 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs.  Container = vc-data-blob, delimiter = /, continuation = 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--, prefix = CC/
Oct 19 11:14:47 turing-ws312 blobfuse[393029]:  ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:46 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : b5a612e0-601e-004c-0bcd-e392a9000000
Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs.  Container = vc-data-blob, delimiter = /, continuation = 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-, prefix = CC/
Oct 19 11:14:47 turing-ws312 blobfuse[393029]:  ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : ea3047f3-a01e-0051-69cd-e34b43000000
Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 145: azs_readdir : About to call list_blobs.  Container = vc-data-blob, delimiter = /, continuation = 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh, prefix = CC/
Oct 19 11:14:47 turing-ws312 blobfuse[393029]:  ==> REQUEST/RESPONSE :: GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=/&include=metadata&marker=2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC/&restype=container?&User-Agent=azure-storage-fuse/1.4.5&x-ms-date=Wed, 19 Oct 2022 15:14:47 GMT&x-ms-version=2018-11-09&Authorization=****&Transfer-Encoding=--------------------------------------------------------------------------------RESPONSE Status :: 200 :: REQ ID : 1393a1cb-a01e-0073-6dcd-e32575000000
Oct 19 11:14:47 turing-ws312 blobfuse[393029]: Function azs_readdir, in file /home/vsts/work/1/s/blobfuse/directoryapis.cpp, line 221: #### So far 4 items retreived in 4 iterations.

If we see this snippet from log, we are taking a token and asking backend to return us back with more elements (next block of elements in the listing). Server is returning back an empty list but at the same time it's also saying there are more items left. This means for some reason server did not return back any element and yet asked us to try back. In v2 we might be terminating the listing call if server returns back empty list. Will check the code and update on this.

vibhansa-msft commented 1 year ago

I have a potential fix for the above-described situation in "vibhansa/v2/streamdiremptylist" branch. If you can pull this branch and build locally and give it a try, we can confirm this theory. As of now I am not able to recreate this situation locally so not able to validate my fix. For some reason your account is hitting this corner case where server does not return back any list item for couple of iterations.

Mmdixon commented 1 year ago

thanks @vibhansa-msft the vibhansa/v2/streamdiremptylist branch fixes the issue. Able to see all folders when listing the directory.

vibhansa-msft commented 1 year ago

Thats great, thanks for trying this out quickly. I have raised a PR to merge the fix in main branch. Next release will have this fixed.

vibhansa-msft commented 1 year ago

Can you try listing once with SDK trace on and share the complete log file. From backend side we want to understand in what condition this happens where server does not return a list but does return back a token.

Mmdixon commented 1 year ago

@vibhansa-msft list on your branch? And include the startup mount logs as well?

vibhansa-msft commented 1 year ago

yes, on this working code you can just enable sdk traces and it shall list some REST calls going for listing but not returning back with any item. Just want those ids to trace some logs and follow up with backend team on whether this is expected case or not.

Mmdixon commented 1 year ago

@vibhansa-msft here is the full log

click for full log ``` Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_CRIT [mount.go (359)]: Starting Blobfuse2 Mount : 2.0.0-preview.4 on [Ubuntu 20.04.5 LTS] Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_CRIT [mount.go (360)]: Logging level set to : LOG_DEBUG Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [libfuse.go (217)]: Libfuse::Configure : libfuse Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [libfuse.go (251)]: Libfuse::Configure : read-only false, allow-other true, default-perm 511, entry-timeout 120, attr-time 120, negative-timeout 240, ignore-open-flags: false Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [file_cache.go (197)]: FileCache::Configure : file_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [file_cache.go (272)]: FileCache::Configure : Using default eviction policy Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [file_cache.go (291)]: FileCache::Configure : create-empty false, cache-timeout 86400, tmp-path /tmp/blob/cache, max-size-mb 0, high-mark 80, low-mark 60 Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [attr_cache.go (121)]: AttrCache::Configure : attr_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [attr_cache.go (145)]: AttrCache::Configure : cache-timeout 240, symlink false, cache-on-list true Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [azstorage.go (83)]: AzStorage::Configure : azstorage Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [config.go (270)]: ParseAndValidateConfig : Parsing config Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [config.go (372)]: ParseAndValidateConfig : using the following proxy address from the config file: Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [config.go (376)]: ParseAndValidateConfig : sdk logging from the config file: true Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [config.go (477)]: ParseAndReadDynamicConfig : Reparsing config Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_DEBUG [config.go (383)]: ParseAndValidateConfig : Getting auth type Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [config.go (465)]: ParseAndValidateConfig : Account: turingitp2, Container: vc-data-blob, AccountType: BLOCK, Auth: KEY, Prefix: , Endpoint: https://turingitp2.blob.core.windows.net/, ListBlock: 0, MD5 : false false, Virtual Directory: true Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [config.go (469)]: ParseAndValidateConfig : Retry Config: Retry count 3, Max Timeout 3600, BackOff Time 1, Max Delay 3 Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [block_blob.go (173)]: BlockBlob::SetupPipeline : Setting up Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [block_blob.go (135)]: BlockBlob::getCredential : Getting credential Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_DEBUG [azauth.go (79)]: azAuth::getAzAuth : Account: turingitp2, AccountType: BLOCK, Protocol: https, Endpoint: https://turingitp2.blob.core.windows.net/ Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [block_blob.go (260)]: BlockBlob::SetPrefixPath : path Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_TRACE [block_blob.go (209)]: BlockBlob::TestPipeline : Validating Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&maxresults=2&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [4f9a6b69-3939-4a84-71b0-9571fd3b9f52]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:11 GMT] Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/59.61517ms, OpTime=254.025146ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&maxresults=2&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [4f9a6b69-3939-4a84-71b0-9571fd3b9f52]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:11 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:10 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [4f9a6b69-3939-4a84-71b0-9571fd3b9f52]#012 X-Ms-Request-Id: [edd157d5-301e-0039-7611-ea1512000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_INFO [mount.go (367)]: mount: Mounting blobfuse2 on /blob/cache Oct 27 10:38:11 turing-ws312 blobfuse2[558491]: LOG_DEBUG [mount.go (385)]: mount: foreground disabled, child = false Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_CRIT [mount.go (359)]: Starting Blobfuse2 Mount : 2.0.0-preview.4 on [Ubuntu 20.04.5 LTS] Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_CRIT [mount.go (360)]: Logging level set to : LOG_DEBUG Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse.go (217)]: Libfuse::Configure : libfuse Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse.go (251)]: Libfuse::Configure : read-only false, allow-other true, default-perm 511, entry-timeout 120, attr-time 120, negative-timeout 240, ignore-open-flags: false Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [file_cache.go (197)]: FileCache::Configure : file_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [file_cache.go (272)]: FileCache::Configure : Using default eviction policy Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [file_cache.go (291)]: FileCache::Configure : create-empty false, cache-timeout 86400, tmp-path /tmp/blob/cache, max-size-mb 0, high-mark 80, low-mark 60 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (121)]: AttrCache::Configure : attr_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [attr_cache.go (145)]: AttrCache::Configure : cache-timeout 240, symlink false, cache-on-list true Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (83)]: AzStorage::Configure : azstorage Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [config.go (270)]: ParseAndValidateConfig : Parsing config Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [config.go (372)]: ParseAndValidateConfig : using the following proxy address from the config file: Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [config.go (376)]: ParseAndValidateConfig : sdk logging from the config file: true Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [config.go (477)]: ParseAndReadDynamicConfig : Reparsing config Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [config.go (383)]: ParseAndValidateConfig : Getting auth type Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [config.go (465)]: ParseAndValidateConfig : Account: turingitp2, Container: vc-data-blob, AccountType: BLOCK, Auth: KEY, Prefix: , Endpoint: https://turingitp2.blob.core.windows.net/, ListBlock: 0, MD5 : false false, Virtual Directory: true Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [config.go (469)]: ParseAndValidateConfig : Retry Config: Retry count 3, Max Timeout 3600, BackOff Time 1, Max Delay 3 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (173)]: BlockBlob::SetupPipeline : Setting up Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (135)]: BlockBlob::getCredential : Getting credential Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azauth.go (79)]: azAuth::getAzAuth : Account: turingitp2, AccountType: BLOCK, Protocol: https, Endpoint: https://turingitp2.blob.core.windows.net/ Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (260)]: BlockBlob::SetPrefixPath : path Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [mount.go (367)]: mount: Mounting blobfuse2 on /blob/cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [mount.go (385)]: mount: foreground disabled, child = true Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [mount.go (451)]: Mount::runPipeline : blobfuse2 pid = 558504, transfer pipe = /tmp/transferPipe_558504, polling pipe = /tmp/pollPipe_558504 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (164)]: AzStorage::Start : Starting component azstorage Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (103)]: AttrCache::Start : Starting component attr_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [file_cache.go (140)]: Starting component : file_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [lru_policy.go (108)]: lruPolicy::StartPolicy Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse.go (134)]: Libfuse::Start : Starting component libfuse Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (115)]: Libfuse::initFuse : Initializing FUSE3 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (158)]: Libfuse::initFuse : Registering fuse callbacks Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (162)]: Libfuse::initFuse : Populating fuse arguments Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (186)]: Libfuse::populateFuseArgs Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : blobfuse2 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : /blob/cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : -o Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [lru_policy.go (261)]: lruPolicy::ClearCache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : entry_timeout=120,attr_timeout=120,negative_timeout=240,allow_other Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : -f Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : -ofsname=blobfuse2 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_DEBUG [libfuse_handler.go (219)]: Libfuse::populateFuseArgs : opts : -okernel_cache Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (174)]: Libfuse::initFuse : Mounting with fuse3 library Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (239)]: Libfuse::libfuse_init : init Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (242)]: Libfuse::libfuse_init : Kernel Caps : 52428763 Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (249)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_PARALLEL_DIROPS Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (255)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_AUTO_INVAL_DATA Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (262)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_READDIRPLUS Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (268)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_ASYNC_READ Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (274)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_SPLICE_WRITE Oct 27 10:38:11 turing-ws312 blobfuse2[558504]: LOG_INFO [libfuse_handler.go (286)]: Libfuse::libfuse_init : Enable Capability : FUSE_CAP_WRITEBACK_CACHE Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [file_cache.go (1149)]: FileCache::GetAttr : CC Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (455)]: AttrCache::GetAttr : CC Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (505)]: BlockBlob::GetAttr : name CC Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (452)]: BlockBlob::getAttrUsingList : name CC Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC, marker Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&maxresults=5000&prefix=CC&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [2058f74a-9bf8-45c1-5886-a08e41313513]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/52.973218ms, OpTime=217.36498ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&maxresults=5000&prefix=CC&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [2058f74a-9bf8-45c1-5886-a08e41313513]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [2058f74a-9bf8-45c1-5886-a08e41313513]#012 X-Ms-Request-Id: [fe6b1185-a01e-005a-7411-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (478)]: BlockBlob::getAttrUsingList : Item 0 Blob CC Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (410)]: Libfuse::libfuse_opendir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 0, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [azstorage.go (284)]: AzStorage::StreamDir : Unblocked List API Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [204c7e39-cd21-4c07-4cd4-9e326a11671e]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/49.304078ms, OpTime=49.45408ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [204c7e39-cd21-4c07-4cd4-9e326a11671e]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [204c7e39-cd21-4c07-4cd4-9e326a11671e]#012 X-Ms-Request-Id: [fe6b118c-a01e-005a-7911-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [e9f87c61-3245-4cb9-7821-73bcb66af8b3]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/53.051309ms, OpTime=53.212082ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [e9f87c61-3245-4cb9-7821-73bcb66af8b3]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [e9f87c61-3245-4cb9-7821-73bcb66af8b3]#012 X-Ms-Request-Id: [fe6b1194-a01e-005a-0111-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDLzIwMjEtMjUwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_WARNING [azstorage.go (310)]: AzStorage::StreamDir : next-marker 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- but current list is empty. Need to retry listing Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21156%21MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [601cf9ee-5851-497b-675b-9894580285ac]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/48.498625ms, OpTime=48.655688ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21156%21MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [601cf9ee-5851-497b-675b-9894580285ac]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [601cf9ee-5851-497b-675b-9894580285ac]#012 X-Ms-Request-Id: [fe6b119a-a01e-005a-0711-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!156!MDAwMDcyIUNDL0NDXzIwMjFfTWF5LzIwMjEtMjEvQ0MtTUFJTi0yMDIxMDUwODEyMTQ0Ni0yMDIxMDUwODE1MTQ0Ni0wMDE2NC5qc29ubCEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_WARNING [azstorage.go (310)]: AzStorage::StreamDir : next-marker 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh but current list is empty. Need to retry listing Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 1, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21160%21MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [307e6d65-8694-449b-6675-e8746fc95e55]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/53.462957ms, OpTime=53.63795ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21160%21MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [307e6d65-8694-449b-6675-e8746fc95e55]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [307e6d65-8694-449b-6675-e8746fc95e55]#012 X-Ms-Request-Id: [fe6b11a2-a01e-005a-0f11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!160!MDAwMDc2IUNDL0NDXzIwMjFfTWF5L3Njb3Jlc190ZXN0L0NDLU1BSU4tMjAyMTA1MTgwMzMxNDgtMjAyMTA1MTgwNjMxNDgtMDAwMDkuanNvbmwhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 2, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [9df02cea-b4e1-4c3b-5346-9b54331198af]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/63.063532ms, OpTime=63.198975ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [9df02cea-b4e1-4c3b-5346-9b54331198af]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [9df02cea-b4e1-4c3b-5346-9b54331198af]#012 X-Ms-Request-Id: [fe6b11ac-a01e-005a-1911-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!76!MDAwMDEyIUNDL0NDX0xhbmdzMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 3, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [34315bf0-8872-4727-4760-5490d37bf00b]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/49.867399ms, OpTime=50.016951ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [34315bf0-8872-4727-4760-5490d37bf00b]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [34315bf0-8872-4727-4760-5490d37bf00b]#012 X-Ms-Request-Id: [fe6b11b6-a01e-005a-2311-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 3 objects with 2!72!MDAwMDA5IUNDL2NhY2hlMCEwMDAwMjghOTk5OS0xMi0zMVQyMzo1OTo1OS45OTk5OTk5WiE- marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 6, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [285658d7-7f6f-4252-70fe-2ba45a933101]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/50.446811ms, OpTime=50.617944ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [285658d7-7f6f-4252-70fe-2ba45a933101]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [285658d7-7f6f-4252-70fe-2ba45a933101]#012 X-Ms-Request-Id: [fe6b11bf-a01e-005a-2c11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!76!MDAwMDExIUNDL2NsZWFuZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 7, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [ba1b9149-4c6e-4ac1-6b24-4b7f1249ba4d]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/52.59616ms, OpTime=52.745843ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [ba1b9149-4c6e-4ac1-6b24-4b7f1249ba4d]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [ba1b9149-4c6e-4ac1-6b24-4b7f1249ba4d]#012 X-Ms-Request-Id: [fe6b11c6-a01e-005a-3311-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!72!MDAwMDEwIUNDL2RlZHVwZTAhMDAwMDI4ITk5OTktMTItMzFUMjM6NTk6NTkuOTk5OTk5OVoh marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 8, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21168%21MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [a0caa7c4-9474-4415-48ee-ac3309c84496]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/48.811759ms, OpTime=48.975402ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21168%21MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [a0caa7c4-9474-4415-48ee-ac3309c84496]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [a0caa7c4-9474-4415-48ee-ac3309c84496]#012 X-Ms-Request-Id: [fe6b11d0-a01e-005a-3c11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!168!MDAwMDgyIUNDL2Rvd25sb2FkLzIwMTktNTEvQ0MtTUFJTi0yMDE5MTIwNzEwNTc1NC0yMDE5MTIwNzEzMzc1NC0wMDI4MC53YXJjLmd6LmluY29tcGxldGUhMDAwMDI4ITE2MDEtMDEtMDFUMDA6MDA6MDAuMDAwMDAwMFoh marker for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ-- for Path CC/ Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_WARNING [azstorage.go (310)]: AzStorage::StreamDir : next-marker 2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ-- but current list is empty. Need to retry listing Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 8, count 5000 Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ-- Oct 27 10:38:49 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21156%21MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [02d90d05-6cb0-494c-65e0-293f8e427e66]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/52.234064ms, OpTime=52.385767ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%21156%21MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [02d90d05-6cb0-494c-65e0-293f8e427e66]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [02d90d05-6cb0-494c-65e0-293f8e427e66]#012 X-Ms-Request-Id: [fe6b11d4-a01e-005a-3f11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!156!MDAwMDcxIUNDL2Rvd25sb2FkLzIwMjItMDUvQ0MtTUFJTi0yMDIyMDEyODE1MjUzMC0yMDIyMDEyODE4MjUzMC0wMDQ4NS53YXJjLmd6ITAwMDAyOCExNjAxLTAxLTAxVDAwOjAwOjAwLjAwMDAwMDBaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 9, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [d6ddc56b-eab5-4ac6-70ea-a5c2a80fbea0]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/53.339004ms, OpTime=53.476297ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [d6ddc56b-eab5-4ac6-70ea-a5c2a80fbea0]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [d6ddc56b-eab5-4ac6-70ea-a5c2a80fbea0]#012 X-Ms-Request-Id: [fe6b11e1-a01e-005a-4c11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!76!MDAwMDExIUNDL2V4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_WARNING [azstorage.go (310)]: AzStorage::StreamDir : next-marker 2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- but current list is empty. Need to retry listing Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 9, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [12543dc7-84ca-4ea3-7f13-ab6a4c3d692d]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/48.904192ms, OpTime=49.063354ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE-&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [12543dc7-84ca-4ea3-7f13-ab6a4c3d692d]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [12543dc7-84ca-4ea3-7f13-ab6a4c3d692d]#012 X-Ms-Request-Id: [fe6b11ec-a01e-005a-5711-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 2 objects with 2!72!MDAwMDA5IUNDL2ZpbHRlciEwMDAwMjghMTYwMS0wMS0wMVQwMDowMDowMC4wMDAwMDAwWiE- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 11, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [0cb0d0e5-f2a6-4f27-6b89-7326a78475ad]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/51.675792ms, OpTime=51.814625ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [0cb0d0e5-f2a6-4f27-6b89-7326a78475ad]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [0cb0d0e5-f2a6-4f27-6b89-7326a78475ad]#012 X-Ms-Request-Id: [fe6b11ef-a01e-005a-5a11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 2 objects with 2!76!MDAwMDExIUNDL2luZGV4ZXMwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 13, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [c6725deb-e453-4dd1-4778-1a83ff0eae38]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/58.907166ms, OpTime=59.059379ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2176%21MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [c6725deb-e453-4dd1-4778-1a83ff0eae38]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [c6725deb-e453-4dd1-4778-1a83ff0eae38]#012 X-Ms-Request-Id: [fe6b11f4-a01e-005a-5f11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 4 objects with 2!76!MDAwMDExIUNDL3NhbXBsZWQwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 17, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [bb88c333-e653-404e-4e2c-ae4572887220]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/55.204828ms, OpTime=55.34371ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2172%21MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [bb88c333-e653-404e-4e2c-ae4572887220]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [bb88c333-e653-404e-4e2c-ae4572887220]#012 X-Ms-Request-Id: [fe6b1200-a01e-005a-6b11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 1 objects with 2!72!MDAwMDA4IUNDL3RlbXAwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (302)]: AzStorage::StreamDir : next-marker 2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [attr_cache.go (262)]: AttrCache::ReadDir : CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [azstorage.go (278)]: AzStorage::StreamDir : Path CC/, offset 18, count 5000 Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [block_blob.go (519)]: BlockBlob::List : prefix CC/, marker 2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> OUTGOING REQUEST (Try=1)#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2192%21MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [fdae3e57-77b6-4017-408a-888eaa4bf7de]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_INFO [utils.go (239)]: SDK : ==> REQUEST/RESPONSE (Try=1/48.627565ms, OpTime=48.775508ms) -- RESPONSE SUCCESSFULLY RECEIVED#012 GET https://turingitp2.blob.core.windows.net/vc-data-blob?comp=list&delimiter=%2F&include=metadata&marker=2%2192%21MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ--&maxresults=5000&prefix=CC%2F&restype=container&timeout=3601#012 Authorization: REDACTED#012 User-Agent: [Azure-Storage-Fuse/2.0.0-preview.4 (Ubuntu 20.04.5 LTS) Azure-Storage/0.14 (go1.19.2; linux)]#012 X-Ms-Client-Request-Id: [fdae3e57-77b6-4017-408a-888eaa4bf7de]#012 X-Ms-Version: [2020-08-04]#012 x-ms-date: [Thu, 27 Oct 2022 14:38:50 GMT]#012 --------------------------------------------------------------------------------#012 RESPONSE Status: 200 OK#012 Content-Type: [application/xml]#012 Date: [Thu, 27 Oct 2022 14:38:49 GMT]#012 Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]#012 X-Ms-Client-Request-Id: [fdae3e57-77b6-4017-408a-888eaa4bf7de]#012 X-Ms-Request-Id: [fe6b1204-a01e-005a-6f11-ea5337000000]#012 X-Ms-Version: [2020-08-04] Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_DEBUG [azstorage.go (299)]: AzStorage::StreamDir : Retrieved 0 objects with 2!92!MDAwMDIzIUNDL3RyYWZpbGF0dXJhLWV4dHJhY3QwITAwMDAyOCE5OTk5LTEyLTMxVDIzOjU5OjU5Ljk5OTk5OTlaIQ-- marker for Path CC/ Oct 27 10:38:50 turing-ws312 blobfuse2[558504]: LOG_TRACE [libfuse_handler.go (435)]: Libfuse::libfuse_releasedir : CC/, handle: 1 ```
vibhansa-msft commented 1 year ago

Thanks for sharing the logs. As updated earlier we have the fix ready and merged in main branch, next release shall have it. Closing this issue.