search

dynamicdan / sn-filesync

2-way sync ServiceNow field values to local files
MIT License
66 stars 37 forks source link

Proxy Support #19

Closed TheoSqua closed 8 years ago

TheoSqua commented 8 years ago

Any suggestion on getting filesync to work through a proxy? Our company uses websense endpoint which isn't playing nice with the SSL.

dynamicdan commented 8 years ago

There are some others interested in proxy support but I haven't really had the need to implement it yet. As I also don't need this I have prioritised some other issues. Your best option is a pull request.

The main components are app/src/snc-client.js and the restify node module. According to restify there is proxy support built in (latest version): https://github.com/restify/node-restify/blob/c36c02add12214d201b18fdd073a2d8e1819689a/docs/index.restdown#L2056

You would need to update the module and test the change. I'm happy to provide support.

BTW, have you tried the acceptBadSSL FileSync config option (documented in the README)?

dynamicdan commented 8 years ago

This will be addressed after the experimental branch and restructure is resolved.

dynamicdan commented 8 years ago

The experimental branch now uses restler. Perhaps a different fork is needed for proxy support like: https://github.com/nayanbhana/restler proxy support commits: https://github.com/nayanbhana/restler/commit/072d910740475c6a4bb0321b306e8b3cf346795b https://github.com/nayanbhana/restler/commit/773be5a665a0d6fcff3bd77c479f12202b8419de

Other proxy info: https://github.com/danwrong/restler/issues/9 suggests using proxychains which seems to be a wrapper for running node to use proxies (http://proxychains.sourceforge.net/) https://github.com/danwrong/restler/issues/176

Any ideas how to setup a fake proxy to do testing?

mak42 commented 8 years ago

First of all, so glad I found your Repo. This is exactly what I was looking for. Thought about implementing it myself, but didn't have the time for that. Anyway at my workplace we are also behind a corporate-proxy, so this is a must have for us. Especially because not all of my colleagues are using OSX and it seems proxychains has poor windows support. Is there any specific reason why you use restler? I think a rewrite of snc-client.js with the request library wouldn't be that painful.

UPDATE: Based on WaTech/servicenowrest I've created a SN Rest-Client which uses the request library instead of restler. Currently not so solid, but should work and it respects the http/https proxy environment variables. Maybe you have a look at it. mak42/sn-rest

dynamicdan commented 8 years ago

Hi mak42,

Glad there is someone else who wants this who is also active. The only reason this issue is open is because I haven't found someone to test the proxy setup. The experimental branch is prepared and just requires a change to bin/snc-client.js on lines 48 to 54 to test the proxy setup. I'd appreciate if you could test that first. I'm not too keen at this point to change the connector but if you can sell me on the advantages of the request lib (or your class) then I'll look into it.

Once done just run the default "--search" on the command line to test. You should get some results if the connection details are correct.

mak42 commented 8 years ago

Hi dynamicdan,

Also glad I've found an active developer, all other repos I've looked at seem to be pretty dead. I have tested the experimental branch on my office windows machine, apparently it doesn't seem to work.

01:59:09 - debug:  Error: getaddrinfo ENOTFOUND undefined undefined:80
    at errnoException (dns.js:28:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:79:26)
01:59:09 - debug: -------------------------------------------------------
01:59:09 - warn: No connection available (do we have internet?)
details: {"code":"ENOTFOUND","errno":"ENOTFOUND","syscall":"getaddrinfo","hostname":"undefined","host":"undefined","port":80}
01:59:09 - error:  Error: getaddrinfo ENOTFOUND undefined undefined:80
    at errnoException (dns.js:28:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:79:26)
01:59:09 - info: ERROR in query or response.
01:59:09 - info:  Error: No connection available (do we have internet?)
details: {"code":"ENOTFOUND","errno":"ENOTFOUND","syscall":"getaddrinfo","hostname":"undefined","host":"undefined","port":80}
    at validateResponse (C:\Users\mkirchweger\Development\sn-filesync\lib\snc-client.js:82:24)
    at Request.handleResponse (C:\Users\mkirchweger\Development\sn-filesync\lib\snc-client.js:148:27)
    at emitTwo (events.js:106:13)
    at Request.emit (events.js:191:7)
    at Request.mixin._fireError (C:\Users\mkirchweger\Development\sn-filesync\node_modules\@mishguru\restler\lib\restler.js:225:10)
    at ClientRequest.<anonymous> (C:\Users\mkirchweger\Development\sn-filesync\node_modules\@mishguru\restler\lib\restler.js:264:14)
    at emitOne (events.js:96:13)
    at ClientRequest.emit (events.js:188:7)
    at emitErr (C:\Users\mkirchweger\Development\sn-filesync\node_modules\agent-base\agent.js:83:11)
    at C:\Users\mkirchweger\Development\sn-filesync\node_modules\agent-base\agent.js:94:9
01:59:09 - info: Total records found: 0
01:59:09 - info: (max records returned per search set to 5)

Any Idea? I will have a detailed look at it tomorrow.

Anyway, I think it would be really awesome if we would build modules/libraries for service-now: So you and other developers can easily re-use it. Currently I have 5 modules in mind:

based on these 5 Libraries I would like to build "sn-cli" a command-line tool to:

I definitely will build this, and thanks to your work I don't have to start at zero for "sn-sync" and "sn-search" 👍 :-) Maybe you like my Idea, any help would be greatly appreciated.

UPDATE

I had a look at the restler source-code and the part of "snc-client" where you set the proxy. You had wrong property names, that was the reason why it didn't work.

//you have to change hostname to host and proxy to port
this.defaults.proxy = {
  'hostname':'bla',
  'proxy': '33232'
};
//with this configuration it work like a charm on windows and *nix
this.defaults.proxy = {
  'host':'192.168.1.1',
  'port': '3128'
};
dynamicdan commented 8 years ago

Really glad you spotted that and it works!

As a first step I would then merge that proxy solution into FileSync. As a second step is like to explore the modularisation of FileSync to simple tools/apis.

I have thought about this a lot and am already looking to enhance the API/command line offering by adding push and pull commands. This work is in progress but life has been busy the last few months (was on holiday too).

I've also always considered a record browser GUI that could be used to download records. Potentially also for comparing instances. This is a common request in the field (instance reconciliation). Helps with monitoring deployment and update set issues.

Happy I can join you for the ride! Perhaps it's time the roadmap got an upgrade!

mak42 commented 8 years ago

I see you are from munich, so I really would like to discuss the whole Topic in German. Any Mail Address I can reach out to you? Mine is kirchweger.mak@gmail.com.

dynamicdan commented 8 years ago

Grüß dich!

I am in Germany but my written/email German is not that top-notch. I would also prefer to keep conversation in the open source realm as this is an open project and maybe it will encourage others to contribute and suggest some more ideas/features/solutions etc. I also like to keep my inbox empty if possible (foolish dream I know).

Otherwise we can maybe meet. In Munich any time?

mak42 commented 8 years ago

Servus :) OK I see so you are living in munich but you are not german. My english is not that top-notch, but I will try it, I should get better anyway. Sure I would love to, I will let you know when I'm in munich the next time.

So to the project, I'v created all the repos but hadn't much time the last days to really do something. When I have something to show, I will post here :)