search

Norconex / crawlers

Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.
https://opensource.norconex.com/crawlers
Apache License 2.0
183 stars 67 forks source link

Add Support Base 64 encoding of password #1068

Open stejacob opened 1 month ago

stejacob commented 1 month ago

Hi,

This is likely an enhancement request.

Could you support for Base-64 encoded passwords be added to the Norconex HTTP crawler in the near future, similar to the functionality available in Autonomy IDOL? Below is an example for reference.

`

yourUsername XY12AbcDefGHIJ==

`

Additionally, would it be possible to provide an option to encode passwords in Base-64 using your encryption tool?

Thank you again!

essiembre commented 1 month ago

Because encoding (as opposed to encrypting) is not a secure way to protect passwords, we are not planning to add it. I am still marking it as a feature request, as we may reconsider if there is enough demand for it.

What issues are you experiencing with using encryption?

stejacob commented 1 month ago

Thank you Pascal.

No problem at all. I was simply looking for an easier way to encrypt passwords in Base-64 without relying on an external file. I understand that it will be less secure without the file but it will be used within our secure network. Thank you again.

Regards,

Stephen Jacob

essiembre commented 1 month ago

You can avoid using an external key file by putting the key value right with the config, like this:

<credentials>
  <username>joe</username>
  <password>3ncryp73d</password>
  <passwordKey>
    <value>paste_the_key_kere</value>
    <source>key</source>
  </passwordKey>
</credentials>

Alternatively, you can also store the decryption key as an environment variable or system property, as described by the Credentials class.