invalid submodule

[zouyonghao]

Thanks for this great job!

The dataserver and web-library submodule seems invalid.

[foxsen]

Thanks for reporting, will fix it soon

[zouyonghao]

I tried again, but got

fatal: remote error: upload-pack: not our ref 2ac9c23ea56e6a5f9aa914dd36370415f9572915
fatal: The remote end hung up unexpectedly
Fetched in submodule path 'src/server/dataserver', but it did not contain 2ac9c23ea56e6a5f9aa914dd36370415f9572915. Direct fetching of that commit failed.

[foxsen]

You are right, it can be downloaded but lacked some refs.

[foxsen]

fix(unverified) pushed, please help to verify.

[zouyonghao]

Thanks for quick update.

I tried with the latest commit, the server successfully start after I ran git submodule init && git submodule update in server/dataserver.

However when I use the latest Zotero client dev build, it synced failed.

See the logs below:

db-zotero-minio_1          | 
db-zotero-minio_1          | API: PostPolicyBucket(bucket=zotero)
db-zotero-minio_1          | Time: 12:37:39 UTC 08/23/2021
db-zotero-minio_1          | DeploymentID: 3f520734-2e57-446c-ba26-53329881bd63
db-zotero-minio_1          | RequestID: 169DEFC28FE7C131
db-zotero-minio_1          | RemoteHost: 172.19.0.9
db-zotero-minio_1          | Host: localhost:8082
db-zotero-minio_1          | UserAgent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Zotero/5.0.97-beta.38+f27432347
db-zotero-minio_1          | Error: Data size larger than expected (*errors.errorString)
db-zotero-minio_1          |        4: cmd/handler-utils.go:281:cmd.validateFormFieldSize()
db-zotero-minio_1          |        3: cmd/handler-utils.go:302:cmd.extractPostPolicyFormValues()
db-zotero-minio_1          |        2: cmd/bucket-handlers.go:868:cmd.objectAPIHandlers.PostPolicyBucketHandler()
db-zotero-minio_1          |        1: net/http/server.go:2049:http.HandlerFunc.ServeHTTP()
db-zotero-minio_1          | 
db-zotero-minio_1          | API: PostPolicyBucket(bucket=zotero)
db-zotero-minio_1          | Time: 12:37:39 UTC 08/23/2021
db-zotero-minio_1          | DeploymentID: 3f520734-2e57-446c-ba26-53329881bd63
db-zotero-minio_1          | RequestID: 169DEFC28FE7C131
db-zotero-minio_1          | RemoteHost: 172.19.0.9
db-zotero-minio_1          | Host: localhost:8082
db-zotero-minio_1          | UserAgent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Zotero/5.0.97-beta.38+f27432347
db-zotero-minio_1          | Error: Data size larger than expected (*errors.errorString)
db-zotero-minio_1          |        2: cmd/bucket-handlers.go:870:cmd.objectAPIHandlers.PostPolicyBucketHandler()
db-zotero-minio_1          |        1: net/http/server.go:2049:http.HandlerFunc.ServeHTTP()

Errors reported by Zotero client:

[JavaScript Error: "Unexpected file upload status 400 (1/IB3AK78Z)"]

[JavaScript Error: "<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data.</Message><BucketName>zotero</BucketName><Resource>/zotero/</Resource><RequestId>169DF0795F16F625</RequestId><HostId>3f520734-2e57-446c-ba26-53329881bd63</HostId></Error>" {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 726}]
Zotero.Sync.Storage.Mode.ZFS.prototype._uploadFile<@chrome://zotero/content/xpcom/storage/zfs.js:726:9
tryCatcher@resource://zotero/loader.jsm -> resource://zotero/bluebird/util.js:16:16
module.exports/PromiseSpawn.prototype._promiseRejected@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:106:18
module.exports/Promise.prototype._settlePromise@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:611:17
module.exports/Promise.prototype._settlePromise0@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:649:5
module.exports/Promise.prototype._settlePromises@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:725:13
_drainQueueStep@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:93:9
_drainQueue@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:86:9
Async.prototype._drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:102:5
Async/this.drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:15:9

[JavaScript Error: "Upload request 1/IB3AK78Z failed"]

[JavaScript Error: "A file sync error occurred. Please try syncing again.

If you receive this message repeatedly, restart Zotero and/or your computer and try again. If you continue to receive the message, submit an error report and post the Report ID to a new thread in the Zotero Forums." {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 727}]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "HTTP GET http://localhost:8080/retractions/list failed with status code 404:

<h1>Not Found</h1>
<p>The page you requested could not be found.</p>
"]

[JavaScript Error: "Unexpected file upload status 400 (1/IB3AK78Z)"]

[JavaScript Error: "<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data.</Message><BucketName>zotero</BucketName><Resource>/zotero/</Resource><RequestId>169DF07C819216EE</RequestId><HostId>3f520734-2e57-446c-ba26-53329881bd63</HostId></Error>" {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 726}]
Zotero.Sync.Storage.Mode.ZFS.prototype._uploadFile<@chrome://zotero/content/xpcom/storage/zfs.js:726:9
tryCatcher@resource://zotero/loader.jsm -> resource://zotero/bluebird/util.js:16:16
module.exports/PromiseSpawn.prototype._promiseRejected@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:106:18
module.exports/Promise.prototype._settlePromise@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:611:17
module.exports/Promise.prototype._settlePromise0@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:649:5
module.exports/Promise.prototype._settlePromises@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:725:13
_drainQueueStep@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:93:9
_drainQueue@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:86:9
Async.prototype._drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:102:5
Async/this.drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:15:9

[JavaScript Error: "Upload request 1/IB3AK78Z failed"]

[JavaScript Error: "A file sync error occurred. Please try syncing again.

If you receive this message repeatedly, restart Zotero and/or your computer and try again. If you continue to receive the message, submit an error report and post the Report ID to a new thread in the Zotero Forums." {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 727}]

[JavaScript Error: "Unexpected file upload status 400 (1/IB3AK78Z)"]

[JavaScript Error: "<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data.</Message><BucketName>zotero</BucketName><Resource>/zotero/</Resource><RequestId>169DF07CF6C4DF56</RequestId><HostId>3f520734-2e57-446c-ba26-53329881bd63</HostId></Error>" {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 726}]
Zotero.Sync.Storage.Mode.ZFS.prototype._uploadFile<@chrome://zotero/content/xpcom/storage/zfs.js:726:9
tryCatcher@resource://zotero/loader.jsm -> resource://zotero/bluebird/util.js:16:16
module.exports/PromiseSpawn.prototype._promiseRejected@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:106:18
module.exports/Promise.prototype._settlePromise@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:611:17
module.exports/Promise.prototype._settlePromise0@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:649:5
module.exports/Promise.prototype._settlePromises@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:725:13
_drainQueueStep@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:93:9
_drainQueue@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:86:9
Async.prototype._drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:102:5
Async/this.drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:15:9

[JavaScript Error: "Upload request 1/IB3AK78Z failed"]

[JavaScript Error: "A file sync error occurred. Please try syncing again.

If you receive this message repeatedly, restart Zotero and/or your computer and try again. If you continue to receive the message, submit an error report and post the Report ID to a new thread in the Zotero Forums." {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 727}]

[JavaScript Error: "Unexpected file upload status 400 (1/IB3AK78Z)"]

[JavaScript Error: "<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>MalformedPOSTRequest</Code><Message>The body of your POST request is not well-formed multipart/form-data.</Message><BucketName>zotero</BucketName><Resource>/zotero/</Resource><RequestId>169DF07D406E8E72</RequestId><HostId>3f520734-2e57-446c-ba26-53329881bd63</HostId></Error>" {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 726}]
Zotero.Sync.Storage.Mode.ZFS.prototype._uploadFile<@chrome://zotero/content/xpcom/storage/zfs.js:726:9
tryCatcher@resource://zotero/loader.jsm -> resource://zotero/bluebird/util.js:16:16
module.exports/PromiseSpawn.prototype._promiseRejected@resource://zotero/loader.jsm -> resource://zotero/bluebird/generators.js:106:18
module.exports/Promise.prototype._settlePromise@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:611:17
module.exports/Promise.prototype._settlePromise0@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:649:5
module.exports/Promise.prototype._settlePromises@resource://zotero/loader.jsm -> resource://zotero/bluebird/promise.js:725:13
_drainQueueStep@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:93:9
_drainQueue@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:86:9
Async.prototype._drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:102:5
Async/this.drainQueues@resource://zotero/loader.jsm -> resource://zotero/bluebird/async.js:15:9

[JavaScript Error: "Upload request 1/IB3AK78Z failed"]

[JavaScript Error: "A file sync error occurred. Please try syncing again.

If you receive this message repeatedly, restart Zotero and/or your computer and try again. If you continue to receive the message, submit an error report and post the Report ID to a new thread in the Zotero Forums." {file: "chrome://zotero/content/xpcom/storage/zfs.js" line: 727}]

version => 5.0.97-beta.38+f27432347, platform => Linux x86_64, oscpu => Linux x86_64, locale => en-US, appName => Zotero, appVersion => 5.0.97-beta.38+f27432347, extensions => Zotero LibreOffice Integration (5.0.25.SA.5.0.97-beta.38+f27432347, extension)

[foxsen]

I did not meet such issues.

Something you might check/try:

• Have you run ./bin/init.sh to initialize database?
• after start the client, have you setting the sync options? default user/pass admin/admin
• Have you tried to build the client in current repo? Latest client might have incompatible changes
• Maybe the minio image is incompatible with older one? Now we are using the latest one, which might be different from what I was using(around this March). You can try docker image with tag minio/minio:RELEASE.2021-03-04T00-53-13Z

[zouyonghao]

Thanks @foxsen I have ran ./bin/init.sh and use the default user account. I'll try to build the client and use your minio image version.

[zouyonghao]

After tried several times, I think I have figure out the condition that could fail which is adding large pdf files.

The client can successfully sync with small pdf files, but fail to sync large files (5MB in my testing).

[foxsen]

maybe you can tried to visit minio directly first. localhost:8082 or use minio client.

[zouyonghao]

Thanks @foxsen for your patience and great help.

I'll close this issue because it seems I'm dealing with another issue.