Linking together with Sinopia-apm

[shirgho]

Hi Corevo,

Thank you for uploading these repositories. Could you please explain how to link together your siniopia-apm and apm-server packages? From what I understand, I should be able to access the packages from within Atom -> Edit -> Preferences -> Install, and not see the regular atom.io packages, right?

I have set up the sinopia-apm by following your instructions, and could successfully npm publish a package to the localhost sinopia-apm.

I have tried following the apm-server instructions. (instead of npm build, i had to do npm run-script build, and instead of npm serve, i used npm start). I have set my apm-server the same as the npm server, i.e localhost:4873. Perhaps this is incorrect? (In the ATOM_NODE_URL e.g).

I get the following output on npm start.

> nodemon lib/index.js --exec babel-node

[nodemon] 1.17.5
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `babel-node lib/index.js`
(node:15742) UnhandledPromiseRejectionWarning: Error: [object Object]
    at /home/sahar193/Tasks/apm-server/lib/index.js:26:9
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)
(node:15742) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:15742) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[nodemon] clean exit - waiting for changes before restart

My Sinopia server recognises an attempt was made to access it by

warn  --- config file  - /home/sahar193/.config/sinopia/config.yaml
 warn  --- http address - http://localhost:4873/
 http  <-- 404, user: undefined, req: 'GET //-/all', error: file not found

Outside Atom, when I do npm install mypackage, it does download the package, but inside a nodemodules folder, with a package-lock.json on the same level as the nodemodules folder.

Thank you for reading through! I am sure I am making some mistakes along the way.

[corevo]

It's trying to access //-/all, but it should try /-/all, try removing the last / from NPM_REGISTRY

[shirgho]

Thanks for the hint, the requests works now but I am still a little confused.

Perhaps just sinopia-apm first. When I do npm install, I get a node_modules folder alongside a package-lock.json file. Inside this node_modules folder is the actual package, alongside the other installed dependancies. Is this also the result you get?

npm install classify-ebtb
npm WARN saveError ENOENT: no such file or directory, open '/home/sahar193/Tasks/serverpacks/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/home/sahar193/Tasks/serverpacks/package.json'
npm WARN serverpacks No description
npm WARN serverpacks No repository field.
npm WARN serverpacks No README data
npm WARN serverpacks No license field.

+ classify-ebtb@1.0.1
added 1 package in 1.333s

Sinopia's output:

$ sinopia
 warn  --- config file  - /home/sahar193/.config/sinopia/config.yaml
 warn  --- http address - http://localhost:4873/
 http  --> 404, req: 'GET https://registry.npmjs.org/classify-ebtb', bytes: 0/21
 http  <-- 304, user: sameer, req: 'GET /classify-ebtb', bytes: 0/0

[corevo]

Yes, but that's just using npm, try installing the extension using apm.

[shirgho]

Right, so three scenarios using apm.

If I use it without actually starting the apm server, but with sinopia running and the apm configs and variables set(again, I just set the apm-server to the same place, localhost:4873):

http  <-- 404, user: undefined, req: 'GET /api/packages/classify-ebtb', error: file not found

Secondly, if I npm start the apm-server, that uncaught error message appears as in my first post, and Sinopia crashes. I think this occurs when it has searched for the package locally, and not finding it goes to registery.npmjs.org. apm-server output:

[nodemon] starting `babel-node lib/index.js`
(node:15529) UnhandledPromiseRejectionWarning: Error: FetchError: invalid json response body at http://localhost:4873/-/all reason: Unexpected end of JSON input
    at /home/sahar193/Tasks/apm-server/lib/index.js:26:9
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)
(node:15529) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:15529) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
[nodemon] clean exit - waiting for changes before restart

Sinopia output:

http  --> 200, req: 'GET https://registry.npmjs.org/-/all' (streaming)
fatal --- uncaught exception, please report this
Error: Invalid JSON (Unexpected "\u001f" at position 0 in state STOP)
    at Parser.proto.charError (/home/sahar193/.nvm/versions/node/v8.11.3/lib/node_modules/sinopia-apm/node_modules/jsonparse/jsonparse.js:90:16)
    at Parser.proto.write (/home/sahar193/.nvm/versions/node/v8.11.3/lib/node_modules/sinopia-apm/node_modules/jsonparse/jsonparse.js:154:23)
    at Stream.<anonymous> (/home/sahar193/.nvm/versions/node/v8.11.3/lib/node_modules/sinopia-apm/node_modules/JSONStream/index.js:21:12)
    at Stream.stream.write (/home/sahar193/.nvm/versions/node/v8.11.3/lib/node_modules/sinopia-apm/node_modules/through/index.js:26:11)
    at IncomingMessage.ondata (_stream_readable.js:639:20)
    at emitOne (events.js:121:20)
    at IncomingMessage.emit (events.js:211:7)
    at IncomingMessage.Readable.read (_stream_readable.js:475:10)
    at flow (_stream_readable.js:846:34)
    at resume_ (_stream_readable.js:828:3)

3) If I turn off the internet and repeat the commands(starting sinopia and then npm start the apm-server), Sinopia does not crash. Instead, apm-server gives out an error message:

npm start

> apm-server@0.0.0 start /home/sahar193/Tasks/apm-server
> nodemon lib/index.js --exec babel-node

[nodemon] 1.17.5
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `babel-node lib/index.js`
events.js:183
      throw er; // Unhandled 'error' event
      ^

Error: listen EADDRINUSE :::4873
    at Object._errnoException (util.js:992:11)
    at _exceptionWithHostPort (util.js:1014:20)
    at Server.setupListenHandle [as _listen2] (net.js:1355:14)
    at listenInCluster (net.js:1396:12)
    at Server.listen (net.js:1480:7)
    at Function.listen (/home/sahar193/Tasks/apm-server/node_modules/express/lib/application.js:618:24)
    at /home/sahar193/Tasks/apm-server/lib/index.js:22:7
    at <anonymous>
    at process._tickDomainCallback (internal/process/next_tick.js:228:7)
[nodemon] app crashed - waiting for file changes before starting...

and then apm-search/install give the following output:

error --- uplink error: getaddrinfo EAI_AGAIN registry.npmjs.org:443 http <-- 200, user: undefined, req: 'GET /-/all', bytes: 0/33826666 http <-- 404, user: undefined, req: 'GET /api/packages/search?q=classify-ebtb', error: file not found ht

[corevo]

What is your configuration again?
Post all the environments variables (reduct the server names of course), It seems to me you have pointed apm-server to the wrong npm upstream.

[shirgho]

I am not actually using any servers yet. I set the atom domain etc to localhost:4873 as well. Will this not work?

The .env file: PORT=4873 NPM_REGISTRY=http://localhost:4873 DOMAIN=http://localhost:4873 FEATURED=one,two,three DEBUG=application*

[shirgho]

echo $ATOM_API_URL http://localhost:4873/api echo $ATOM_NODE_URL http://localhost:4873/download/atom-shell

[corevo]

So you can't set DOMAIN and NPM_REGISTRY to the same port, DOMAIN is what apm-server will listen to, and where it will receive requests.
While NPM_REGISTRY is where your npm server will listen, pointing them to the same ports, means that one of them will crash because that port is taken.

[shirgho]

OK, sorry, I have changed the port for DOMAIN to 8000, as well as well for the env variables. Should I also do something else, or is just giving it another port is enough?

Now apm-server outputs:

> apm-server@0.0.0 start /home/sahar193/Tasks/apm-server
> nodemon lib/index.js --exec babel-node

[nodemon] 1.17.5
[nodemon] to restart at any time, enter `rs`
[nodemon] watching: *.*
[nodemon] starting `babel-node lib/index.js`
  application listening on 6000 +0ms

Not sure about the 6000 though, when I set the port for DOMAIN to 8000

Now when I do apm install from another terminal:

Installing classify-ebtb to /home/sahar193/.atom/packages ✗
Request for package information failed: connect ECONNREFUSED 127.0.0.1:8000 (ECONNREFUSED)

ps. I still need to close my internet connection. Otherwise the older errors still occur and Sinopia crashes.

[shirgho]

OK, so I set the port to 6000, and now it works! I could successfully apm install :D Is the port hardcoded somewhere?

Now, the next step, can I install/update this package from within Atom, the install packages page? So far, atom still tries to search for the package from atom.io:443

[corevo]

No, the port is something default, unless the env var PORT is defined.
Atom might have new environment variables controlling where he searches from the ones I've found were ATOM_API_URL, ATOM_NODE_URL, with these changed everything worked.
It's been a year and a half, perhaps things changed, iirc Atom has all the configuration in one place, so search for one of these, and maybe you'll find a new variable.

[shirgho]

OK, thank you, I will keep looking, and post again if I can get to updating packages from within Atom from sinopia-apm.

[shirgho]

Another question, my ATOM_NODE_URL is currently pointing to http://localhost:6000/download/atom-shell as per the instructions. But does it not need access to a real Atom shell?

btw, the ATOM_NODE_URL var is now ATOM_ELECTRON_URL.

[corevo]

It definitely does need access, to download the headers.
If you can access atom.io then you don't need to set it, if you're behind a firewall, you're gonna need to set it and download the shell, then put it in the correct directory.

[shirgho]

OK, thanks.

The only problem remaining is that Setting->Install from inside the Atom GUI does not route to localhost:6000. It still looks for packages through atom.io.

When I start atom with: env "HTTP_PROXY=http://localhost:6000" atom, it does not search through atom.io. Instead it gives the error: tunneling socket could not be established, cause=socket hang up

Without the env variable set, just setting the proxy variables in atom's .apmrc does not seem to have any effect.

[corevo]

apm-server is not a proxy, it is a mirror.

[shirgho]

OK, thanks.

An update: I can update private packages through Atom now! From Settings -> Updates. So even though I still can't search for these locally hosted packages, I can update them.

[corevo]

Maybe they changed the search url to hard code look on atom.io.
If you can find the place where it's defined we can open an issue on atom/atom to change it back (or send a pr).

[shirgho]

Here it looks like it is fixed: https://github.com/atom/settings-view/blob/8729f2b114aca74b4e80b1b356d75717c99e65da/lib/atom-io-client.coffee#L11

Here there is an option to workaround with a proxy, I think: https://github.com/atom/settings-view/blob/master/lib/package-manager.coffee