search

s3fs-fuse / s3fs-fuse

FUSE-based file system backed by Amazon S3
GNU General Public License v2.0
8.65k stars 1.02k forks source link

Couldn't upload files on macOS 12 #2105

Open davodm opened 1 year ago

davodm commented 1 year ago

Additional Information

Version of s3fs being used (s3fs --version)

Amazon Simple Storage Service File System V1.91 (commit:unknown) with GnuTLS(gcrypt)

Version of fuse being used (pkg-config --modversion fuse, rpm -qi fuse or dpkg -s fuse)

2.9.9

Kernel information (uname -r)

21.6.0

GNU/Linux Distribution, if applicable (cat /etc/os-release)

macOS Monterey 12.6.3

How to run s3fs, if applicable

s3fs mybucket ${HOME}/Documents/myfolder -d -f -o curldbg -o allow_other -o multipart_size=50 -o endpoint=eu-west-1

s3fs mybucket ${HOME}/Documents/myfolder -d -f -o curldbg

s3fs syslog messages (grep s3fs /var/log/syslog, journalctl | grep s3fs, or s3fs outputs)

2023-02-01T09:47:36.378Z [INF] s3fs.cpp:s3fs_flush(2409): [path=/._ugc.json][pseudo_fd=2]
2023-02-01T09:47:36.378Z [INF]       fdcache_entity.cpp:RowFlush(1347): [tpath=][path=/._ugc.json][pseudo_fd=2][physical_fd=29]
2023-02-01T09:47:36.378Z [INF]       cache.cpp:DelStat(591): delete stat cache entry[path=/._ugc.json]
2023-02-01T09:47:36.378Z [INF] s3fs.cpp:s3fs_release(2467): [path=/._ugc.json][pseudo_fd=2]
2023-02-01T09:47:36.378Z [INF]       fdcache.cpp:GetFdEntity(485): [path=/._ugc.json][pseudo_fd=2]
2023-02-01T09:47:36.379Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=R_OK X_OK ]
2023-02-01T09:47:36.380Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:36.381Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/]
2023-02-01T09:47:36.381Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.363Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=W_OK ]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=R_OK ]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=X_OK ]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.368Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/pro 02.png]
2023-02-01T09:47:39.368Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png]
2023-02-01T09:47:39.368Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.368Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png
2023-02-01T09:47:39.368Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png
2023-02-01T09:47:39.368Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png] [] []
2023-02-01T09:47:39.368Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.368Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.368Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.368Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.368Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.368Z [CURL DBG] > HEAD /pro%2002.png HTTP/1.1
2023-02-01T09:47:39.368Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.368Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.368Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.368Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=62aa659491209951e7becc8bddf7ca072c12a89817f813ca4bdd4310eacf4ea3
2023-02-01T09:47:39.368Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.368Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.368Z [CURL DBG] > 
2023-02-01T09:47:39.392Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.392Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.392Z [CURL DBG] < x-amz-request-id: Q8CPZ4WKP7S8ARVX
2023-02-01T09:47:39.392Z [CURL DBG] < x-amz-id-2: Zd1m9V0DnhzuGNiFXBjgjygkX2tqjEY4SyDYhQ5oFPqMbHI+ccdcjVX75d66JvpeJ9dPqbc1ryg=
2023-02-01T09:47:39.392Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.392Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.392Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.392Z [CURL DBG] < 
2023-02-01T09:47:39.392Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.392Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.392Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png/]
2023-02-01T09:47:39.392Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png/][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.392Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png/
2023-02-01T09:47:39.392Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png/
2023-02-01T09:47:39.392Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png/] [] []
2023-02-01T09:47:39.392Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.392Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.392Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.392Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.392Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.392Z [CURL DBG] > HEAD /pro%2002.png/ HTTP/1.1
2023-02-01T09:47:39.392Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.392Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.392Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.392Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=1e0fbc152ed607e81015e62550e540cbe5f064290778b5b7491ba050ed7354eb
2023-02-01T09:47:39.392Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.392Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.392Z [CURL DBG] > 
2023-02-01T09:47:39.415Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.416Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.416Z [CURL DBG] < x-amz-request-id: Q8CTF056AFQAA86W
2023-02-01T09:47:39.416Z [CURL DBG] < x-amz-id-2: hJlk1PWhcz6XEFSbBsAo5/pTglRIQ2zUWK2BkYPiSJo20hCnP4gqUUc6cztxfIKwDdiUC28Uq6w=
2023-02-01T09:47:39.416Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.416Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.416Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.416Z [CURL DBG] < 
2023-02-01T09:47:39.416Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.416Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.416Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png_$folder$]
2023-02-01T09:47:39.416Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png_$folder$][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.416Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png_%24folder%24
2023-02-01T09:47:39.416Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png_%24folder%24
2023-02-01T09:47:39.416Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png_$folder$] [] []
2023-02-01T09:47:39.416Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.416Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.416Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.416Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.416Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.416Z [CURL DBG] > HEAD /pro%2002.png_%24folder%24 HTTP/1.1
2023-02-01T09:47:39.416Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.416Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.416Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.416Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=f2abd8f21927388e804d7a8d106a856a19f24dc9c48e3e690e7c0ef6c087d656
2023-02-01T09:47:39.416Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.416Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.416Z [CURL DBG] > 
2023-02-01T09:47:39.439Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.439Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.439Z [CURL DBG] < x-amz-request-id: Q8CQCCB2DT8DHVBG
2023-02-01T09:47:39.439Z [CURL DBG] < x-amz-id-2: +dvaU9tKRpk6LAz2sPWu4ps5Guk+BdxSGqtg53obmrIBvkLopqD7M4/grYXX5rMfHSLqyosMd+0=
2023-02-01T09:47:39.439Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.439Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.439Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.439Z [CURL DBG] < 
2023-02-01T09:47:39.439Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.439Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.439Z [INF]   s3fs.cpp:list_bucket(2707): [path=/pro 02.png]
2023-02-01T09:47:39.439Z [INF]       curl.cpp:ListBucketRequest(3522): [tpath=/pro 02.png]
2023-02-01T09:47:39.439Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.439Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.439Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=pro%2002.png/] []
2023-02-01T09:47:39.439Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.439Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.439Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.439Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.439Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.439Z [CURL DBG] > GET /?delimiter=/&max-keys=2&prefix=pro%2002.png/ HTTP/1.1
2023-02-01T09:47:39.439Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.439Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.439Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.439Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=7faf20f11a949b78e16dad377cd2300ca13cc38c97ea2ba29510aa61cf4ce1d8
2023-02-01T09:47:39.439Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.439Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.439Z [CURL DBG] > 
2023-02-01T09:47:39.464Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.464Z [CURL DBG] < HTTP/1.1 200 OK
2023-02-01T09:47:39.464Z [CURL DBG] < x-amz-id-2: 1ph92hgssYhWkXZEXum5gDBwcuE3tZKMkRqFQUfjCrX/vXAJUFoVL6e77chR/oYVJnkdd2kvHco=
2023-02-01T09:47:39.464Z [CURL DBG] < x-amz-request-id: Q8CQMBCPPFZHFVFC
2023-02-01T09:47:39.464Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:40 GMT
2023-02-01T09:47:39.464Z [CURL DBG] < x-amz-bucket-region: eu-west-1
2023-02-01T09:47:39.464Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.464Z [CURL DBG] < Transfer-Encoding: chunked
2023-02-01T09:47:39.464Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.464Z [CURL DBG] < 
2023-02-01T09:47:39.464Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.464Z [INF]       curl.cpp:RequestPerform(2324): HTTP response code 200
2023-02-01T09:47:39.466Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.466Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.467Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.467Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.467Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.467Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=F_OK]
2023-02-01T09:47:39.468Z [INF] s3fs.cpp:s3fs_access(3456): [path=/][mask=X_OK ]
2023-02-01T09:47:39.468Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/pro 02.png]
2023-02-01T09:47:39.468Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png]
2023-02-01T09:47:39.468Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.468Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png
2023-02-01T09:47:39.468Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png
2023-02-01T09:47:39.468Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png] [] []
2023-02-01T09:47:39.468Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.468Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.468Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.468Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.468Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.468Z [CURL DBG] > HEAD /pro%2002.png HTTP/1.1
2023-02-01T09:47:39.468Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.468Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.468Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.468Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=62aa659491209951e7becc8bddf7ca072c12a89817f813ca4bdd4310eacf4ea3
2023-02-01T09:47:39.468Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.468Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.468Z [CURL DBG] > 
2023-02-01T09:47:39.488Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.488Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.488Z [CURL DBG] < x-amz-request-id: Q8CKPXEBQTV37SCJ
2023-02-01T09:47:39.488Z [CURL DBG] < x-amz-id-2: H8hWMZm4TxkSsmJTBmxFmeskFNRULljLpxjuXPf1yToDr/RmbVWldYam/KjR8CYRJf85Am9gtCA=
2023-02-01T09:47:39.488Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.488Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.488Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.488Z [CURL DBG] < 
2023-02-01T09:47:39.488Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.488Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.488Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png/]
2023-02-01T09:47:39.488Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png/][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.488Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png/
2023-02-01T09:47:39.488Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png/
2023-02-01T09:47:39.488Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png/] [] []
2023-02-01T09:47:39.488Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.488Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.488Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.488Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.488Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.488Z [CURL DBG] > HEAD /pro%2002.png/ HTTP/1.1
2023-02-01T09:47:39.488Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.488Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.488Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.488Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=1e0fbc152ed607e81015e62550e540cbe5f064290778b5b7491ba050ed7354eb
2023-02-01T09:47:39.488Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.488Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.488Z [CURL DBG] > 
2023-02-01T09:47:39.508Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.509Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.509Z [CURL DBG] < x-amz-request-id: Q8CVK57GSQWVBPDM
2023-02-01T09:47:39.509Z [CURL DBG] < x-amz-id-2: HL+igLHw29zcnz7YSK6ri87je525b5RVfjiyz0VSowC5ce3R9LKPcsmP0TC7VO7y86fcZZriLC4=
2023-02-01T09:47:39.509Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.509Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.509Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.509Z [CURL DBG] < 
2023-02-01T09:47:39.509Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.509Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.509Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png_$folder$]
2023-02-01T09:47:39.509Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png_$folder$][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.509Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png_%24folder%24
2023-02-01T09:47:39.509Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png_%24folder%24
2023-02-01T09:47:39.509Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png_$folder$] [] []
2023-02-01T09:47:39.509Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.509Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.509Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.509Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.509Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.509Z [CURL DBG] > HEAD /pro%2002.png_%24folder%24 HTTP/1.1
2023-02-01T09:47:39.509Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.509Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.509Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.509Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=f2abd8f21927388e804d7a8d106a856a19f24dc9c48e3e690e7c0ef6c087d656
2023-02-01T09:47:39.509Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.509Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.509Z [CURL DBG] > 
2023-02-01T09:47:39.530Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.530Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.530Z [CURL DBG] < x-amz-request-id: Q8CH2CQJCJ9H9FX4
2023-02-01T09:47:39.530Z [CURL DBG] < x-amz-id-2: G/OwV2alK23nebsYTbzmccr+Sv6oA4kIjeIC0km6ClCr/8Sn4sFxbdM21xQIxx7rxd9J5BTDy9U=
2023-02-01T09:47:39.530Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.530Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.530Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.530Z [CURL DBG] < 
2023-02-01T09:47:39.530Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.530Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.530Z [INF]   s3fs.cpp:list_bucket(2707): [path=/pro 02.png]
2023-02-01T09:47:39.530Z [INF]       curl.cpp:ListBucketRequest(3522): [tpath=/pro 02.png]
2023-02-01T09:47:39.530Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.530Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.530Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=pro%2002.png/] []
2023-02-01T09:47:39.530Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.531Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.531Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.531Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.531Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.531Z [CURL DBG] > GET /?delimiter=/&max-keys=2&prefix=pro%2002.png/ HTTP/1.1
2023-02-01T09:47:39.531Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.531Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.531Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.531Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=7faf20f11a949b78e16dad377cd2300ca13cc38c97ea2ba29510aa61cf4ce1d8
2023-02-01T09:47:39.531Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.531Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.531Z [CURL DBG] > 
2023-02-01T09:47:39.554Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.554Z [CURL DBG] < HTTP/1.1 200 OK
2023-02-01T09:47:39.554Z [CURL DBG] < x-amz-id-2: tpMepURAzaDhjzEx/taeTg6MLmERtIHv3gIV+YO7a00gwOwKBzgiN4kVu44k3nReDuzVDOhhl8Y=
2023-02-01T09:47:39.554Z [CURL DBG] < x-amz-request-id: Q8CK2V8EVPXHH05T
2023-02-01T09:47:39.554Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:40 GMT
2023-02-01T09:47:39.554Z [CURL DBG] < x-amz-bucket-region: eu-west-1
2023-02-01T09:47:39.554Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.554Z [CURL DBG] < Transfer-Encoding: chunked
2023-02-01T09:47:39.554Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.554Z [CURL DBG] < 
2023-02-01T09:47:39.555Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.555Z [INF]       curl.cpp:RequestPerform(2324): HTTP response code 200
2023-02-01T09:47:39.555Z [INF] s3fs.cpp:s3fs_getattr(763): [path=/pro 02.png]
2023-02-01T09:47:39.555Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png]
2023-02-01T09:47:39.555Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.555Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png
2023-02-01T09:47:39.555Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png
2023-02-01T09:47:39.555Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png] [] []
2023-02-01T09:47:39.555Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.555Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.555Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.555Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.555Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.555Z [CURL DBG] > HEAD /pro%2002.png HTTP/1.1
2023-02-01T09:47:39.555Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.555Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.555Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.555Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=62aa659491209951e7becc8bddf7ca072c12a89817f813ca4bdd4310eacf4ea3
2023-02-01T09:47:39.555Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.555Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.555Z [CURL DBG] > 
2023-02-01T09:47:39.575Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.575Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.575Z [CURL DBG] < x-amz-request-id: Q8CKBBPYRB06HQW5
2023-02-01T09:47:39.575Z [CURL DBG] < x-amz-id-2: jP8En0H2BSqW7zBm0PLE/xJWW+3jDLMMBcnSl43zUQA2TMF8VQ+p7/Thda28nucqfNx/jCB43EM=
2023-02-01T09:47:39.575Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.575Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.575Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.575Z [CURL DBG] < 
2023-02-01T09:47:39.575Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.575Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.575Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png/]
2023-02-01T09:47:39.575Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png/][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.575Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png/
2023-02-01T09:47:39.575Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png/
2023-02-01T09:47:39.575Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png/] [] []
2023-02-01T09:47:39.575Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.575Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.575Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.575Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.575Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.575Z [CURL DBG] > HEAD /pro%2002.png/ HTTP/1.1
2023-02-01T09:47:39.576Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.576Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.576Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.576Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=1e0fbc152ed607e81015e62550e540cbe5f064290778b5b7491ba050ed7354eb
2023-02-01T09:47:39.576Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.576Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.576Z [CURL DBG] > 
2023-02-01T09:47:39.595Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.595Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.595Z [CURL DBG] < x-amz-request-id: Q8CG6DD6F8CG730E
2023-02-01T09:47:39.596Z [CURL DBG] < x-amz-id-2: W4LrcjVthCaixHrc30AnK+bt8ujIT/TGCc0R9gRZjw+KpZINLRbHEVrLnWyU69yrFgktO/vvyrI=
2023-02-01T09:47:39.596Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.596Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.596Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.596Z [CURL DBG] < 
2023-02-01T09:47:39.596Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.596Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.596Z [INF]       curl.cpp:HeadRequest(3100): [tpath=/pro 02.png_$folder$]
2023-02-01T09:47:39.596Z [INF]       curl.cpp:PreHeadRequest(3060): [tpath=/pro 02.png_$folder$][bpath=][save=][sseckeypos=18446744073709551615]
2023-02-01T09:47:39.596Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket/pro%2002.png_%24folder%24
2023-02-01T09:47:39.596Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com/pro%2002.png_%24folder%24
2023-02-01T09:47:39.596Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [HEAD] [/pro 02.png_$folder$] [] []
2023-02-01T09:47:39.596Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com
2023-02-01T09:47:39.596Z [CURL DBG] * Found bundle for host mybucket.s3.amazonaws.com: 0x60000085cd80 [serially]
2023-02-01T09:47:39.596Z [CURL DBG] * Can not multiplex, even if we wanted to!
2023-02-01T09:47:39.596Z [CURL DBG] * Re-using existing connection! (#1) with host mybucket.s3.amazonaws.com
2023-02-01T09:47:39.596Z [CURL DBG] * Connected to mybucket.s3.amazonaws.com (52.218.116.57) port 443 (#1)
2023-02-01T09:47:39.596Z [CURL DBG] > HEAD /pro%2002.png_%24folder%24 HTTP/1.1
2023-02-01T09:47:39.596Z [CURL DBG] > Host: mybucket.s3.amazonaws.com
2023-02-01T09:47:39.596Z [CURL DBG] > User-Agent: s3fs/1.91 (commit hash unknown; GnuTLS(gcrypt))
2023-02-01T09:47:39.596Z [CURL DBG] > Accept: */*
2023-02-01T09:47:39.596Z [CURL DBG] > Authorization: AWS4-HMAC-SHA256 Credential=AmazonKEY/20230201/eu-west-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=f2abd8f21927388e804d7a8d106a856a19f24dc9c48e3e690e7c0ef6c087d656
2023-02-01T09:47:39.596Z [CURL DBG] > x-amz-content-sha256: amazon content sha256
2023-02-01T09:47:39.596Z [CURL DBG] > x-amz-date: 20230201T094739Z
2023-02-01T09:47:39.596Z [CURL DBG] > 
2023-02-01T09:47:39.616Z [CURL DBG] * Mark bundle as not supporting multiuse
2023-02-01T09:47:39.616Z [CURL DBG] < HTTP/1.1 404 Not Found
2023-02-01T09:47:39.616Z [CURL DBG] < x-amz-request-id: Q8CXJ4WH3ZP6Y5R2
2023-02-01T09:47:39.616Z [CURL DBG] < x-amz-id-2: pPCiHHZPQ1pdjPMpxA13NRVah+9ZkPEGcybcuc+sYdQ73GQqpddcHzCqd1z27A8R+u9zWIJu878=
2023-02-01T09:47:39.616Z [CURL DBG] < Content-Type: application/xml
2023-02-01T09:47:39.616Z [CURL DBG] < Date: Wed, 01 Feb 2023 09:47:38 GMT
2023-02-01T09:47:39.616Z [CURL DBG] < Server: AmazonS3
2023-02-01T09:47:39.616Z [CURL DBG] < 
2023-02-01T09:47:39.616Z [CURL DBG] * Connection #1 to host mybucket.s3.amazonaws.com left intact
2023-02-01T09:47:39.616Z [INF]       curl.cpp:RequestPerform(2376): HTTP response code 404 was returned, returning ENOENT
2023-02-01T09:47:39.616Z [INF]   s3fs.cpp:list_bucket(2707): [path=/pro 02.png]
2023-02-01T09:47:39.616Z [INF]       curl.cpp:ListBucketRequest(3522): [tpath=/pro 02.png]
2023-02-01T09:47:39.616Z [INF]       curl_util.cpp:prepare_url(255): URL is https://s3.amazonaws.com/mybucket?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.616Z [INF]       curl_util.cpp:prepare_url(288): URL changed is https://mybucket.s3.amazonaws.com?delimiter=/&max-keys=2&prefix=pro%2002.png/
2023-02-01T09:47:39.616Z [INF]       curl.cpp:insertV4Headers(2696): computing signature [GET] [/] [delimiter=/&max-keys=2&prefix=pro%2002.png/] []
2023-02-01T09:47:39.616Z [INF]       curl_util.cpp:url_to_host(332): url is https://s3.amazonaws.com

Details about the issue

Hi, I'm experiencing a weird issue on s3fs which is I'm able to upload small text/JSON files (~50KB) but I'm not able to upload other types of files or even text files more than 1MB. I'm getting this error and I'm not able to upload, I tried many options but it doesn't work.

Screenshot 2023-02-01 at 09 47 58
ggtakec commented 1 year ago

@davodm The your attached information was a log that confirmed the existence of the uploaded (trying to) file. It didn't include any logs when the upload failed and wasn't enough to know the cause.

The way you got the log is correct, so I think it seems possible to know the reason if you get a log about the upload to fail. When the upload fails, isn't there a log with a prefix of [ERR], [WAN], or [CRT]? If you find them and know the logs before and after them, we may be able to identify the cause. Thanks in advance for your assistance.