search

zotero / translation-server-old

Server-side Zotero translation based on Mozilla xpcshell (deprecated)
35 stars 24 forks source link

CSL JSON import fails #47

Closed zuphilip closed 7 years ago

zuphilip commented 7 years ago

Currently the CSL JSON import fails in the server tests with

TranslatorTester: Running 1 test for CSL JSON 
TranslatorTester: Running CSL JSON Test 1 
TranslatorTester: Translating 
Translation using CSL JSON failed: 
string => TypeError: invalid 'instanceof' operand Zotero.Item 
stack => Zotero.Utilities.itemFromCSLJSON@chrome://translation-server/content/utilities.js:1640:7 
Zotero.Translate.SandboxManager.prototype.importObject/attachTo[localKey]<@chrome://translation-server/content/translation/translate_firefox.js:503:29 
@chrome://translation-server/content/translation/translate_firefox.js:408:66 
doImport@CSL JSON:63:3 
...

This seems to refer to this line: https://github.com/zotero/zotero/blob/e2a3a6edbb8db4ce48d090f9e99bae6dca70251e/chrome/content/zotero/xpcom/utilities.js#L1640 which is not anymore used in the current Zotero code. Especially the commit "Make itemFromCSLJSON independent of Zotero.Item existance" seems related to this issue. Can this be fixed for the translation server as well?

There are some translator based on CSL JSON which I guess are also not working therefore...

CC @adam3smith

AndrewIsh commented 7 years ago

Indeed, I appear to be having the same problem, the Zotero client imports with no problem.

Input file:

{
   "ISBN":"978-0-19-518139-5",
   "categories":[
      "All terrain cycling",
      "LAKE DISTRICT. TRAVEL",
      "Description and travel",
      "LAKE DISTRICT. CYCLING. SPORTS",
      "Guide-books",
      "Lake District (England)"
   ],
   "container-author":[
      {
         "literal":"Elliot, Mark"
      }
   ],
   "container-title":"Mountain biking in Lakeland",
   "container-title-short":"Mountain biking in Lakeland",
   "id":"1-29096",
   "issued":{
      "raw":"1988-01-01T00:00:01Z"
   },
   "medium": "Book",
   "publisher":"Dalesman",
   "publisher-place":"Clapham via Lancaster",
   "shortTitle":"Mountain biking in Lakeland",
   "submitted":{
      "raw":"1988-01-01T00:00:01Z"
   },
   "type":"chapter"
}

Request: POST http://localhost:1969/import

Headers: Content-Type:application/json

Response: An error occurred during translation. Please check translation with Zotero client.

Console:

zotero(5)(+0243697): POST /import HTTP/1.1
cache-control: no-cache
Postman-Token: 551d0c38-8b66-4528-b7ec-0a075c5c005c
Content-Type: application/json
User-Agent: PostmanRuntime/6.1.6
Accept: */*
Host: localhost:1969
accept-encoding: gzip, deflate
content-length: 1129
Connection: keep-alive

zotero(4)(+0000001): Translate: Binding sandbox to http://www.example.com/

zotero(4)(+0000012): Translate: Parsing code for Bibliontology RDF (14763d25-8ba0-45df-8f52-b8d1108e7ac9, 2017-06-03 10:21:42)

zotero(3)(+0000005): Translate: Initializing RDF data store

zotero(2)(+0000001): Translate: Detect using Bibliontology RDF failed: 
thrown exception => DOMParser error: loading data into data store failed
url => undefined
downloadAssociatedFiles => undefined
automaticSnapshots => undefined

zotero(4)(+0000000): Translate: Parsing code for MODS (0e2235e7-babf-413c-9acf-f27cce5f059c, 2015-02-11 01:24:19)

zotero(2)(+0000006): Translate: Detect using MODS failed: 
thrown exception => DOMParser error: loading data into data store failed
url => undefined
downloadAssociatedFiles => undefined
automaticSnapshots => undefined

zotero(4)(+0000000): Translate: Parsing code for MEDLINE/nbib (9ec64cfd-bea7-472a-9557-493c0c26b0fb, 2015-10-16 18:47:13)

zotero(4)(+0000003): Translate: Parsing code for RDF (5e3ad958-ac79-463d-812b-a86a9235c28f, 2017-06-16 18:48:30)

zotero(2)(+0000001): Translate: Detect using RDF failed: 
thrown exception => XML known invalid
url => undefined
downloadAssociatedFiles => undefined
automaticSnapshots => undefined

zotero(4)(+0000000): Translate: Parsing code for CSL JSON (bc03b4fe-436d-4a1f-ba59-de4d2d7a63f7, 2017-06-03 11:41:00)

zotero(3)(+0000001): Translate: All translator detect calls and RPC calls complete:

zotero(3)(+0000000):    CSL JSON: 100

zotero(5)(+0000000): Translate: Running handler 0 for translators

zotero(4)(+0000000): Translate: Parsing code for CSL JSON (bc03b4fe-436d-4a1f-ba59-de4d2d7a63f7, 2017-06-03 11:41:00)

zotero(3)(+0000002): Translate: Beginning translation with CSL JSON

zotero(3)(+0000000): invalid 'instanceof' operand Zotero.Item at chrome://translation-server/content/utilities.js:1640

zotero(2)(+0000000): Translate: Translation using CSL JSON failed: 
string => TypeError: invalid 'instanceof' operand Zotero.Item
stack => Zotero.Utilities.itemFromCSLJSON@chrome://translation-server/content/utilities.js:1640:7
Zotero.Translate.SandboxManager.prototype.importObject/attachTo[localKey]<@chrome://translation-server/content/translation/translate_firefox.js:503:29
@chrome://translation-server/content/translation/translate_firefox.js:408:66
doImport@CSL JSON:63:3
Zotero.Translate.Base.prototype._translateTranslatorLoaded@chrome://translation-server/content/translation/translate.js:1221:4
Zotero.Translate.Base.prototype.translate/<@chrome://translation-server/content/translation/translate.js:1188:58
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO/initCallback@chrome://translation-server/content/translation/translate.js:2052:17
Zotero.Translate.IO.String.prototype.init@chrome://translation-server/content/translation/translate.js:2548:4
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2079:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype.translate@chrome://translation-server/content/translation/translate.js:1188:4
Zotero.Server.Translation.Import.prototype.translators@chrome://translation-server/content/server_translation.js:375:3
Zotero.Translate.Base.prototype._runHandler@chrome://translation-server/content/translation/translate.js:1021:20
Zotero.Translate.Base.prototype._detectTranslatorsCollected@chrome://translation-server/content/translation/translate.js:1538:3
Zotero.Translate.Base.prototype.complete@chrome://translation-server/content/translation/translate.js:1373:30
Zotero.Translate.Import.prototype.complete@chrome://translation-server/content/translation/translate.js:1992:2
Zotero.Translate.Base.prototype.decrementAsyncProcesses@chrome://translation-server/content/translation/translate.js:992:4
Zotero.Translate.Base.prototype._detectTranslatorLoaded@chrome://translation-server/content/translation/translate.js:1522:3
Zotero.Translate.Base.prototype._detect/<@chrome://translation-server/content/translation/translate.js:1503:17
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO/initCallback@chrome://translation-server/content/translation/translate.js:2052:17
Zotero.Translate.IO.String.prototype.init@chrome://translation-server/content/translation/translate.js:2548:4
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2079:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype._detect@chrome://translation-server/content/translation/translate.js:1502:3
Zotero.Translate.Base.prototype.complete@chrome://translation-server/content/translation/translate.js:1370:5
Zotero.Translate.Import.prototype.complete@chrome://translation-server/content/translation/translate.js:1992:2
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2084:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype._detect@chrome://translation-server/content/translation/translate.js:1502:3
Zotero.Translate.Base.prototype.complete@chrome://translation-server/content/translation/translate.js:1370:5
Zotero.Translate.Import.prototype.complete@chrome://translation-server/content/translation/translate.js:1992:2
Zotero.Translate.Base.prototype.decrementAsyncProcesses@chrome://translation-server/content/translation/translate.js:992:4
Zotero.Translate.Base.prototype._detectTranslatorLoaded@chrome://translation-server/content/translation/translate.js:1522:3
Zotero.Translate.Base.prototype._detect/<@chrome://translation-server/content/translation/translate.js:1503:17
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO/initCallback@chrome://translation-server/content/translation/translate.js:2052:17
Zotero.Translate.IO.String.prototype.init@chrome://translation-server/content/translation/translate.js:2548:4
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2079:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype._detect@chrome://translation-server/content/translation/translate.js:1502:3
Zotero.Translate.Base.prototype.complete@chrome://translation-server/content/translation/translate.js:1370:5
Zotero.Translate.Import.prototype.complete@chrome://translation-server/content/translation/translate.js:1992:2
Zotero.Translate.Base.prototype._detectTranslatorLoaded@chrome://translation-server/content/translation/translate.js:1517:4
Zotero.Translate.Base.prototype._detect/<@chrome://translation-server/content/translation/translate.js:1503:17
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO/initCallback@chrome://translation-server/content/translation/translate.js:2052:17
Zotero.Translate.IO.String.prototype.init@chrome://translation-server/content/translation/translate.js:2548:4
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2079:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype._detect@chrome://translation-server/content/translation/translate.js:1502:3
Zotero.Translate.Base.prototype.complete@chrome://translation-server/content/translation/translate.js:1370:5
Zotero.Translate.Import.prototype.complete@chrome://translation-server/content/translation/translate.js:1992:2
Zotero.Translate.Import.prototype._loadTranslatorPrepareIO@chrome://translation-server/content/translation/translate.js:2084:3
Zotero.Translate.Import.prototype._loadTranslator/<@chrome://translation-server/content/translation/translate.js:2035:3
Zotero.Translate.Base.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:1573:16
Zotero.Translate.Import.prototype._loadTranslator@chrome://translation-server/content/translation/translate.js:2034:2
Zotero.Translate.Base.prototype._detect@chrome://translation-server/content/translation/translate.js:1502:3
Zotero.Translate.Base.prototype._getTranslatorsTranslatorsReceived@chrome://translation-server/content/translation/translate.js:1127:3
Zotero.Translate.Base.prototype._getTranslatorsGetPotentialTranslators/<@chrome://translation-server/content/translation/translate.js:1094:28
Zotero.Translators.CodeGetter.prototype.getCodeFor@chrome://translation-server/content/connector/translator.js:351:4
Zotero.Translators.CodeGetter@chrome://translation-server/content/connector/translator.js:343:2
Zotero.Translators</this.getAllForType@chrome://translation-server/content/connector/translator.js:140:3
Zotero.Translate.Base.prototype._getTranslatorsGetPotentialTranslators@chrome://translation-server/content/translation/translate.js:1093:3
Zotero.Translate.Import.prototype._getTranslatorsGetPotentialTranslators@chrome://translation-server/content/translation/translate.js:2004:3
Zotero.Translate.Base.prototype.getTranslators@chrome://translation-server/content/translation/translate.js:1080:4
Zotero.Translate.Import.prototype.getTranslators@chrome://translation-server/content/translation/translate.js:2024:10
Zotero.Server.Translation.Import.prototype.init@chrome://translation-server/content/server_translation.js:362:3
Zotero.Server.DataListener.prototype._processEndpoint@chrome://translation-server/content/server.js:416:4
Zotero.Server.DataListener.prototype._bodyData@chrome://translation-server/content/server.js:318:3
Zotero.Server.DataListener.prototype._headerFinished@chrome://translation-server/content/server.js:290:3
Zotero.Server.DataListener.prototype.onDataAvailable@chrome://translation-server/content/server.js:201:4
@translation-server/init.js:143:14

url => undefined
downloadAssociatedFiles => undefined
automaticSnapshots => undefined

zotero(5)(+0000001): Translate: Running handler 0 for done

zotero(5)(+0000000): HTTP/1.0 500 Internal Server Error
Content-Type: text/plain

An error occurred during translation. Please check translation with Zotero client.

zotero(3)(+0000000): [JavaScript Error: "not well-formed" {file: "moz-nullprincipal:{b66e9bfe-e704-439b-93fe-f7eb25738bfe}" line: 1 column: 1 source: "{"}] at undefined:1

zotero(3)(+0000000): [JavaScript Error: "not well-formed" {file: "moz-nullprincipal:{29e30ec5-9fe2-407a-8ad8-ee601b34e195}" line: 1 column: 1 source: "{"}] at undefined:1
adam3smith commented 7 years ago

@AndrewIsh sorry, I forgot -- what's the context you're using this in?

AndrewIsh commented 7 years ago

Sorry, "context"?

AndrewIsh commented 7 years ago

Interestingly, I monkey patched the change that @zuphilip linked to into the translation server's utilities.js, rebuilt then ran, and now I'm getting a different error when importing CSL-JSON:

zotero(2)(+0000000): Translate: Translation using CSL JSON failed: 
string => TypeError: Zotero.ItemFields.isBaseField is not a function
stack => Zotero.Utilities.itemFromCSLJSON@chrome://translation-server/content/utilities.js:1702:9
Zotero.Translate.SandboxManager.prototype.importObject/attachTo[localKey]<@chrome://translation-server/content/translation/translate_firefox.js:503:29
@chrome://translation-server/content/translation/translate_firefox.js:408:66
doImport@CSL JSON:63:3
AndrewIsh commented 7 years ago

I should add that I'm completely new to the translation-server, I only discovered it today. So I'm just testing stuff with Postman at the moment. So apologies if I'm making stupid observations :)

adam3smith commented 7 years ago

I should add that I'm completely new to the translation-server, I only discovered it today. So I'm just testing stuff with Postman at the moment.

That was my question. Wanted to know if you're working on a specific application.

AndrewIsh commented 7 years ago

No, not currently, just at the "is this what we need stage" ;-)

AndrewIsh commented 7 years ago

Re: That latest error, your itemFields.js seems to be missing the definition of isBaseField which is in the equivalent file on the Zotero repo

AndrewIsh commented 7 years ago

BTW, just spotted this, which seems to be the same error as the original, seems like they found a way to work around it https://github.com/zotero/translation-server/issues/12

AndrewIsh commented 7 years ago

Hmmm, adding that workaround has the same end result at the monkey patch I mentioned earlier, we then get the second error string => TypeError: Zotero.ItemFields.isBaseField is not a function

I'll shut up now ;-)

zuphilip commented 7 years ago

Interestingly, I monkey patched the change that @zuphilip linked to into the translation server's utilities.js, rebuilt then ran, and now I'm getting a different error when importing CSL-JSON:

You shouldn't change individual files there but rather we should update the submodule https://github.com/zotero/translation-server/tree/master/modules.

AndrewIsh commented 7 years ago

Aha, I'd not realised you were using submodules, makes sense! How do you usually handle pulling in updates?

dstillman commented 7 years ago

I'm working on this (along with updating translation-server to use Zotero 5.0).

AndrewIsh commented 7 years ago

Thanks for the super speedy work on this. CSL import and export now working perfectly :)