yawik / SimpleImport

Simple Job Import Module. Imports job openings into YAWIK
MIT License
0 stars 1 forks source link

Failed plainTextFetch invalidates complete offer #26

Closed mbo-s closed 5 years ago

mbo-s commented 5 years ago

If the templateValues are not set the import tries to fetch the remote page as plain text. If this remote fetching fails, the job gets the status invalid and will not be imported.

There can be many reasons for such a failure, e.g.

2019-01-17T17:06:03+01:00 ERR (3): Cannot fetch HTML digest for a job, import ID: "32", link: "example.com/3", reason: "Unable to fetch remote data, reason: "Read timed out after 5 seconds""
2019-01-17T17:10:08+01:00 ERR (3): Cannot fetch HTML digest for a job, import ID: "9", link: "example.com/1", reason: "Invalid HTTP status: "404""
2019-01-18T13:17:27+01:00 ERR (3): Cannot fetch HTML digest for a job, import ID: "7763de2bd66926f8fc8b49d384628896", link: "example.com/2", reason: "Unable to fetch remote data, reason: "Unable to enable crypto on TCP connection example.com: make sure the "sslcafile" or "sslcapath" option are properly set for the environment.""

All these jobs were not imported and for at least two of them the error is not the remote site.

Since the plaintext is not critical, the job should regardless of a plaintextexception always be imported.