search

elasticio / xml-component

Component to parse, validate, generate XML data & attachments for elastic.io platform
Other
0 stars 5 forks source link

attachmentToJson: File's metadata is not found #16

Closed khanzadyan closed 5 years ago

khanzadyan commented 5 years ago

Background

The task in hand is to read XML files from an SFTP server, create an attachment and then read the attached XML file to convert it to a JSON file for further processing.

ER: it reads File, and it creates the JSON on output from the attachmentToJson action.

AR: when reading the attachment an error is happening:

[2019-03-07 14:45:32.601]: Attachment to string started, message={"id":"cd4fa57c-cd53-4575-a0ab-2caab6cf6548","attachments":{"cd_catalog.xml":{"url":"http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d","size":5077}},"body":{"filename":"cd_catalog.xml","size":5077},"headers":{},"metadata":{},"passthrough":{"step_1":{"id":"cd4fa57c-cd53-4575-a0ab-2caab6cf6548","attachments":{"cd_catalog.xml":{"url":"http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d","size":5077}},"body":{"filename":"cd_catalog.xml","size":5077},"headers":{},"metadata":{}}}} cfg={"_selectedDataSamples":["5c80da713ce2e00011558d64"],"_outDataSamples":["5c80da713ce2e00011558d64"]}
[2019-03-07 14:45:32.601]: Found 1 attachments
[2019-03-07 14:45:32.601]: Processing attachment=cd_catalog.xml content={"url":"http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d","size":5077}
[2019-03-07 14:45:32.676]: at process.internalTickCallback (internal/process/next_tick.js:77:7)
[2019-03-07 14:45:32.676]: Status code: 404,
[2019-03-07 14:45:32.676]: Error: Error in making request to http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d
[2019-03-07 14:45:32.676]: Body: "File's metadata is not found"
[2019-03-07 14:45:32.676]: at TaskExec.processAction (/home/apprunner/lib/actions/attachmentToJson.js:46:19)
[2019-03-07 14:45:32.677]: {"name":"sailor","hostname":"eio.f5ea0e5e8304f9c9f2c14aa2.5bd3187b14101500108f6589.stepgl8vp","pid":7,"ELASTICIO_TASK_ID":"5bd3187b14101500108f6589","ELASTICIO_EXEC_ID":"cc0df33cbf194838ad173e493179f503","ELASTICIO_STEP_ID":"step_2","ELASTICIO_COMP_ID":"5bd8512ce1f6170010803b7f","ELASTICIO_FUNCTION":"attachmentToJson","tag":"5bd3187b14101500108f6589","level":30,"deliveryTag":1,"messageId":"cd4fa57c-cd53-4575-a0ab-2caab6cf6548","parentMessageId":"ca9ea82c-93fa-41eb-b9ad-2b60e950d4f2","traceId":"f5d5b188-c4ec-48fc-bc45-999119c9c941","messagesCount":1,"err":{"message":"Error in making request to http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d \n              Status code: 404, \n              Body: \"File's metadata is not found\"","name":"Error","stack":"Error: Error in making request to http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d \n              Status code: 404, \n              Body: \"File's metadata is not found\"\n    at TaskExec.processAction (/home/apprunner/lib/actions/attachmentToJson.js:46:19)\n    at process.internalTickCallback (internal/process/next_tick.js:77:7)"},"messageProcessingTime":79,"msg":"processMessage emit error","time":"2019-03-07T13:45:32.676Z","v":0}
[2019-03-07 14:45:32.681]: {"name":"sailor","hostname":"eio.f5ea0e5e8304f9c9f2c14aa2.5bd3187b14101500108f6589.stepgl8vp","pid":7,"ELASTICIO_TASK_ID":"5bd3187b14101500108f6589","ELASTICIO_EXEC_ID":"cc0df33cbf194838ad173e493179f503","ELASTICIO_STEP_ID":"step_2","ELASTICIO_COMP_ID":"5bd8512ce1f6170010803b7f","ELASTICIO_FUNCTION":"attachmentToJson","tag":"5bd3187b14101500108f6589","level":30,"deliveryTag":1,"messageId":"cd4fa57c-cd53-4575-a0ab-2caab6cf6548","parentMessageId":"ca9ea82c-93fa-41eb-b9ad-2b60e950d4f2","traceId":"f5d5b188-c4ec-48fc-bc45-999119c9c941","messagesCount":0,"errorCount":1,"promises":1,"messageProcessingTime":83,"msg":"processMessage emit end","time":"2019-03-07T13:45:32.680Z","v":0}
[2019-03-07 14:46:53.851]: Disconnecting
[2019-03-07 14:46:53.851]: Received SIGTERM
[2019-03-07 14:46:53.852]: Successfully disconnected
[2019-03-07 14:46:53.862]: [INFO  tini (1)] Main child exited normally (with status '0')

In the experiment, a standard XML file was used. The actual file exists in on the server storage and confirmed to be identical to the original file from the SFTP server.

zubairov commented 5 years ago

Problematic part here:

[2019-03-07 14:45:32.676]: Status code: 404,
[2019-03-07 14:45:32.676]: Error: Error in making request to http://steward-service.platform.svc.cluster.local:8200/files/fe44b776-d927-4efb-83e9-256cbf2b3f1d
[2019-03-07 14:45:32.676]: Body: "File's metadata is not found"

@khanzadyan seen this one before?

khanzadyan commented 5 years ago

@zubairov yes, I known about this. However, can't understand why it is happening. The file is correctly uploaded and I can confirm it exists.

khanzadyan commented 5 years ago

Stewart is receiving 404 with the following header:

<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

In reality, the file was in the steward and it is identical to the file from the SFTP server.

khanzadyan commented 5 years ago

The meta.json of the attachment:

{"content-length":"14263","host":"steward-service.platform.svc.cluster.local:8200","connection":"close"}