More details for API service setup

[cjnaz]

This is a combination support request and documentation issue. After I get all of this sorted out, I'd be happy to add to the readme.md.

I'm running Centos 7. The setup instructions in the README.md could use a few more details. Some of these may be noob questions…

• Where is it recommended to install the source files to? To a subdir in my user home dir? Step 1 could include the git clone command. I created ~me/xBrowserSync and from this dir I did the git clone, which created the API directory with the files.
• The /var/log dir generally isn't writeable by general users. I set the log file to /home/me/xBrowserSync/xBrowserSync_api.log
• If I run api.js manually from the command line as user me and point a browser on the local machine to 127.0.0.1:8080/info I get {"status":1,"message":"","version":"1.0.3","maxSyncSize":204800}. This config is just for testing and bring-up of the API server.
• Using the Chrome xBrowserSync applet (v1.3.1), I cannot set the Change Service box to http://127.0.0.1:8080/. The server text remains red and the Update button remains grayed out. If I add a .com in the entry ( http://127.0.0.1.com:8080/) the text turns black and the Update tries but fails since that's not the correct address. Seems this may be a bug/enhancement needed for the app to allow an IP address. My current plan is to run the API on my server and not use DDNS since my WAN IP address seldom changes, but I cannot set the app server field to an IP address.
• I need to set up a systemd service file to start the API on system boot. Is there a command line way to kill a currently running api.js?
• I need to get this talking on an ssl / https connection, and perhaps with a virtual web host.
• I suspect that I need to open up the remote port on server firewall, and set up the router port forward.

[nero120]

Hi @cjnaz, first of all I recommend you try running v1.1.0 as this has been updated for latest dependency versions, supports HTTPS and the documentation is a bit better. I'm currently testing before upgrading https://api.xbrowsersync.org and merging to master, but if you do find any issues please raise them here.

Where is it recommended to install the source files to? To a subdir in my user home dir? Step 1 could include the git clone command. I created ~me/xBrowserSync and from this dir I did the git clone, which created the API directory with the files.

Depends on your OS, but really you should be able to run anywhere as long as the account the service will run as has permissions to access that location.

The /var/log dir generally isn't writeable by general users. I set the log file to /home/me/xBrowserSync/xBrowserSync_api.log

In v1.1.0 the file (and directories) will be created as long as the service has permission. If not simply create the file and chown to the account the service will run as.

If I run api.js manually from the command line as user me and point a browser on the local machine to 127.0.0.1:8080/info I get {"status":1,"message":"","version":"1.0.3","maxSyncSize":204800}. This config is just for testing and bring-up of the API server.

Yep that looks like it works correctly when run as you. As I said before, I recommend you use v1.1.0 rather than v1.0.3.

Using the Chrome xBrowserSync applet (v1.3.1), I cannot set the Change Service box to http://127.0.0.1:8080/.

That's correct, the current version of the app does not allow IP addresses. This is simple to overcome though, just create a local hostname and point it to 127.0.0.1, then use that host name when setting the xBrowserSync service with the correct port, e.g. http://xbrowsersync.local:8080.

OS specific stuff is beyond the scope of this repo, but following a guide similar to this helps in getting a node service up and running. Perhaps they have one specific for centos?

For SSL/HTTPS, use v1.1.0 and refer to the README for setting the config values correctly to point to your SSL certs.

[cjnaz]

I installed v1.1.0, and also upgraded node.js from v6.something to v8.11.1. On the local machine I'm able to connect to the API service at http://xbrowsersync.localhost:8080. I'm currently starting the api from the command line as me (not root). from the Chrome app on the local machine I try to Create New Sync I get "Exception occurred in NewSyncLogsService.newSyncsLimitHit: Unable to determine client's IP address". Version dumps, screen shots, and log file output follow.

Edited dumps below for "me" as my non-root login, and the machine name to "xxx"

[me@xxx xBrowserSync]$ mongo --version MongoDB shell version v3.6.3 git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5 OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013 allocator: tcmalloc modules: none build environment: distmod: rhel70 distarch: x86_64 target_arch: x86_64 [me@xxx xBrowserSync]$ node --version v8.11.1

[root@xxx etc]$ systemctl status mongod \u25cf mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2018-04-09 22:47:34 MST; 5min ago Docs: https://docs.mongodb.org/manual Process: 5005 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=0/SUCCESS) Process: 5002 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS) Process: 4999 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS) Process: 4996 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS) Main PID: 5007 (mongod) CGroup: /system.slice/mongod.service \u2514\u25005007 /usr/bin/mongod -f /etc/mongod.conf

Apr 09 22:47:34 xxx systemd[1]: Starting High-performance, schema-free document-oriented database... Apr 09 22:47:34 xxx mongod[5005]: about to fork child process, waiting until server is ready for connections. Apr 09 22:47:34 xxx mongod[5005]: forked process: 5007 Apr 09 22:47:34 xxx mongod[5005]: child process started successfully, parent exiting Apr 09 22:47:34 xxx systemd[1]: Started High-performance, schema-free document-oriented database.

The only edit I've made to the config.json is "path": "/home/xxx/xBrowserSync/log.txt"

[me@xxx xBrowserSync]$ node API/dist/api.js & [2] 5524 [me@xxx xBrowserSync]$ Service started on 127.0.0.1:8080 Exception occurred in NewSyncLogsService.newSyncsLimitHit: Unable to determine client's IP address

[me@xxx xBrowserSync]$ cat log.txt {"name":"xBrowserSync_api","hostname":"xxx","pid":5524,"level":30,"msg":"Service started on 127.0.0.1:8080","time":"2018-04-10T05:54:50.362Z","v":0} {"name":"xBrowserSync_api","hostname":"xxx","pid":5524,"level":50,"req":{"method":"POST","url":"/","headers":{"host":"xbrowsersync.localhost:8080","connection":"keep-alive","content-length":"95996","accept":"application/json, text/plain, */*","origin":"chrome-extension://lcbjdhceifofjlpecfpeimnnphbcjgnc","accept-version":"1.0.x","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36","content-type":"application/json;charset=UTF-8","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9"},"remoteAddress":"::1","remotePort":33044},"err":{"message":"Unable to determine client's IP address","name":"ClientIpAddressEmptyException","stack":"ClientIpAddressEmptyException: Unable to determine client's IP address\n at NewSyncLogsService. (/home/me/xBrowserSync/API/dist/newSyncLogsService.js:55:29)\n at Generator.next ()\n at fulfilled (/home/me/xBrowserSync/API/dist/newSyncLogsService.js:4:58)\n at \n at process._tickCallback (internal/process/next_tick.js:188:7)"},"msg":"Exception occurred in NewSyncLogsService.newSyncsLimitHit","time":"2018-04-10T05:55:09.939Z","v":0} Tried SELinux permissive (setenforce 0) with same result. In setting up v1.1.0, I ran into one snag when setting up the database. Since you changed the case of the name of the database from xBrowserSync to xbrowsersync the db.newsynclogs.createIndex complained about creating the same indexes with a database name that varied only by case. I futz'd around with mongo and eventually got it to not complain. Here's a mongo shell query… > use xbrowsersync switched to db xbrowsersync > db.newsynclogs.getIndexes() [ { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "xbrowsersync.newsynclogs" }, { "v" : 2, "key" : { "ipAddress" : 1, "syncCreated" : 1 }, "name" : "ipAddress_1_syncCreated_1", "ns" : "xbrowsersync.newsynclogs" } ] 

[nero120]

Glad you've got v1.1.0 up and running @cjnaz!

from the Chrome app on the local machine I try to Create New Sync I get "Exception occurred in NewSyncLogsService.newSyncsLimitHit: Unable to determine client's IP address"

This means that an ip address could not be determined from req.ip which should be populated by express. If you have logging enabled, check the log file and search for ClientIpAddressEmptyException. Check the value of remoteAddress in that log entry and let me know what you find.

Optionally, if you use vscode, you can debug the API using the "Debug API" launch configuration which will run the API service and allow you to set a break point in the NewSyncLogsService.newSyncsLimitHit function to determine the value returned by NewSyncLogsService.getClientIpAddress.

In setting up v1.1.0, I ran into one snag when setting up the database. Since you changed the case of the name of the database from xBrowserSync to xbrowsersync the db.newsynclogs.createIndex complained about creating the same indexes with a database name that varied only by case.

Ah yes sorry about that, I should provide some upgrade documentation for anyone upgrading from v1.0.3. The easiest way to do it is to delete the xBrowserSync db first before following the steps in the readme. If you need to migrate existing syncs you can export them using the command:

mongoexport --db xBrowserSync -c bookmarks --out /path/to/export/file

and import them (once the readme steps have been followed) with:

mongoimport --db xbrowsersync -c bookmarks --file /path/to/export/file

[cjnaz]

In the log.txt in my above post: "remoteAddress":"::1","remotePort":33044},"err":{"message":"Unable to determine client's IP address","name":"ClientIpAddressEmptyException","stack":"ClientIpAddressEmptyException: Unable to determine client's IP address\n at NewSyncLogsService.

I'm just starting with debugging in VS code. In the launch.json I changed the '\\' in this line to '/' to get debug to run, fyi: "program": "${workspaceFolder}/dist/api.js", You are working on Windows?

Stepping into the function:

Not a lot of clues here. express isn't returning the ip? Is this somehow an artifact of how I'm referencing the server?

[nero120]

In the log.txt in my above post: "remoteAddress":"::1","remotePort":33044},"err":{"message":"Unable to determine client's IP address","name":"ClientIpAddressEmptyException","stack":"ClientIpAddressEmptyException: Unable to determine client's IP address\n at NewSyncLogsService.

Ah I know what's causing the problem. It's the way I'm extracting the ipv4 value from req.ip, and that's the ipv6 localhost value that is in your log. I'll commit a fix shortly, then you can get the latest v1.1.0 and hopefully that will resolve your issues.

I'm just starting with debugging in VS code. In the launch.json I changed the '\' in this line to '/' to get debug to run, fyi: "program": "${workspaceFolder}/dist/api.js", You are working on Windows?

Not sure how those slashes got in there, I'll sort that out as well. Thanks for your help @cjnaz!

[nero120]

Fixed in e8b4f3d and 04e817b.

[cjnaz]

Looks like e8b4f3d fixed the problem. The service now runs and creates a sync. As I touch bookmarks in Chrome I see the /var/lib/mongo/collection* file get updated. Looks good. I suggest closing this issue, and I may open another with my platform specific setup suggestions. Next for me... create a systemd service file so that the api starts with the machine. Set up a virtual web host with ssl, and get it exported through my router.

[nero120]

Great stuff @cjnaz!

[nero120]

Hi @cjnaz, I've just commit my final changes for v1.1.0 which include the remaining tests and some important stability fixes. I now consider this release fully tested and feature complete so I recommend pulling the latest code and re-installing so you can benefit from these changes.

I'm planning to release v1.1.0 to https://api.xbrowsersync.org/ over the weekend so if you do find any more problems, please do raise another issue! 😄

[cjnaz]

Turns out that I'm not git literate. I git branch -D v1.1.0 my local copy and tried to re-download v1.1.0 with git checkout -b v1.1.0 origin/v1.1.0. At this point I ended up with the earlier v1.1.0 with the IP address regular expression issue. I then tried to pull in commits with git fetch origin refs/heads/v1.1.0:refs/v1.1.0 which did a few updates but apparently not the commit with the regex fix. So... I have no news for you. If you'll give me the proper git cmd sequence I'll try again. :(

[cjnaz]

Ok, I got it. Did a git pull and found the new location on the config file and redid my log file edit. Built and ran locally. Did a new sync without error. Minimal testing.

[nero120]

Wonderful stuff! I'm going to merge v1.1.0 into master tonight as from my testing it's looking nice and stable.