Meteor-Community-Packages / stratosphere

Meteor private package server
MIT License
28 stars 7 forks source link

Question #2

Closed rwatts3 closed 8 years ago

rwatts3 commented 8 years ago

Hello,

First thank you for building this project. I am curious is this simply a packaging system similar to Atmosphere, where I deploy stratosphere on my server, then I can make pull my own packages? Also is this approach dual compatible meaning can I use Atmospherejs packages as well as my own private packages ?

And finally I am using meteor 1.2.0.2 , and meteor tool s at 1.1.9, will this have any adverse affects by modifying that file with the changes in your instructions?

sebakerckhof commented 8 years ago

Hello,

This might indeed need some clarification in the readme. This is indeed something like atmosphere, but which you can deploy on your own servers. The way meteor works is that you can have only 1 package server set and it needs to contain all packages you use. Therefore stratosphere will contain the information about all upstream (atmosphere) packages + your own packages. So yes, you will have access to all atmosphere packages + your own ones.

The only adverse affect of the file change is that you can not publish to atmosphere with the file change. It disables login to the package server, which atmosphere requires but stratosphere currently does not support. You can however always undo the file change and everything is back to normal.

rwatts3 commented 8 years ago

Interesting, Do you have a solution to the last response?

The only adverse affect of the file change is that you can not publish to atmosphere with the file change. It disables login to the package server, which atmosphere requires but stratosphere currently does not support. You can however always undo the file change and everything is back to normal.

I'd be interested in possibly doing a PR if you can point me in the right direction.

sebakerckhof commented 8 years ago

Well, Meteor-tool tries to do an oAuth login with a meteor developer account. So in theory, setting up oAuth for meteor developer accounts (using default package: meteor-developer-accounts) should be sufficient. But this gave me errors. I describe my findings here: https://github.com/meteor/meteor/issues/4497 . This could be a bug in Meteor, but I'm not familiar enough with oAuth to say. However, if this would be solved, then no file modifications would be necessary and installation of stratosphere would be much less cumbersome.

hellogerard commented 8 years ago

@sebakerckhof I agree that this app is sorely needed. I am also interested in hosting my own packages. Maybe we can figure out the OAuth issue together :-)

I cloned the repo, created my OAuth app on meteor.com, and went to start it. But it's not clear what needs to be done in step 5. I found my packages database file. But when I look in the app, there are no instructions. Help?

rwatts3 commented 8 years ago

Have you tried typing meteor login prior to publishing the package. ? On Tue, Oct 13, 2015 at 8:40 AM Gerard Sychay notifications@github.com wrote:

@sebakerckhof https://github.com/sebakerckhof I agree that this app is sorely needed. I am also interested in hosting my own packages. Maybe we can figure out the OAuth issue together :-)

I cloned the repo, created my OAuth app on meteor.com, and went to start it. But it's not clear what needs to be done in step 5. I found my packages database file. But when I look in the app, there are no instructions. Help?

— Reply to this email directly or view it on GitHub https://github.com/sebakerckhof/stratosphere/issues/2#issuecomment-147754318 .

hellogerard commented 8 years ago

Not yet. I thought that before I started publishing my own packages, I would see all the Atmosphere packages on my Stratosphere app. Right now, the main screen says 'No published packages', since I haven't done the syncing step and the directions aren't clear.

More info:

I went ahead and created a toy Meteor app with meteor --create. Then, started it with METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor --port 9000 (after starting Stratosphere on 3000). I am getting a Match fail:

Exception while invoking method 'syncNewPackageData' Error: Match error: Last deletion is required
I20151013-11:43:47.498(-4)?     at [object Object].SimpleSchema.condition (packages/aldeed_simple-schema/packages/aldeed_simple-schema.js:1465:1)
I20151013-11:43:47.498(-4)?     at checkSubtree (packages/check/match.js:220:1)
I20151013-11:43:47.498(-4)?     at check (packages/check/match.js:32:1)
I20151013-11:43:47.498(-4)?     at [object Object].Meteor.methods.syncNewPackageData (server/rpc/syncNewPackageData.js:54:9)
I20151013-11:43:47.498(-4)?     at maybeAuditArgumentChecks (livedata_server.js:1692:12)
I20151013-11:43:47.498(-4)?     at livedata_server.js:708:19
I20151013-11:43:47.498(-4)?     at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
I20151013-11:43:47.499(-4)?     at livedata_server.js:706:40
I20151013-11:43:47.499(-4)?     at [object Object]._.extend.withValue (packages/meteor/dynamics_nodejs.js:56:1)
I20151013-11:43:47.499(-4)?     at livedata_server.js:704:46
I20151013-11:43:47.499(-4)? Sanitized and reported to the client as: Last deletion is required [400]
sebakerckhof commented 8 years ago

Note that, although the database will contain all packages (your own + from atmosphere) the GUI will only show private packages. Mainly because the GUI is written pretty fast and not ready to handle huge amounts of data (no good search, no good paging, ...). + Atmosphere does a pretty good job for public packages.

sebakerckhof commented 8 years ago

@hellogerard the issue could have 2 causes. 1) I forgot something, I just pushed a fix. 2) You made an error in step 5 with the database copy.

Basically if you have your packages.data.db you should copy it and rename it to a sanitized version of your package server url, in your case this would be: localhostX9000.data.db. Normally this should be visible in the instructions in the frontend somewhere. But I rewrote the frontend for beta1 (it used to be based on Telescope) and I might have missed this part.

This copying is also something I hope we can avoid in the future, therefore I opened this issue: https://github.com/meteor/meteor/issues/5391

rwatts3 commented 8 years ago

Feel free to assign something to me if you'd like help

hellogerard commented 8 years ago

@sebakerckhof That last commit fixed the error. Thanks!

Interestingly, after that was fixed, I didn't need to copy the database file. I just started Stratosphere, and started my app with the environment variable, and everything seems to be working. I can start the app, run meteor list, and add packages no problem.

I tried publishing my first private package and got stopped. I will create a new issue for that.

sebakerckhof commented 8 years ago

Yeah, copying of the database is not always necessary. It depends on what command you execute, so the step is there to make sure it always works. However, once it has synchronized, you're good to go for every meteor-tool command.

sebakerckhof commented 8 years ago

@rwatts3 All help is welcome. However for some things you'll need a good understanding of how meteor deals with packages and the structure of the package database. This isn't documented somewhere, I'll try to do this once I've got some time.

The easiest thing to help with would be improvements to the frontend (based on angular-material), but this is also the least important for now.

The most annoying thing is the oAuth flow, since it means there is no login to the package server and the file modifications are necessary. Maybe you can help there. I'll make a new branch where I'm going to enable meteor-developer-accounts and then maybe we'll find what goes wrong.

rwatts3 commented 8 years ago

Sounds good, I'll look for the new branch and pull once it's available.

So from what I understand the goal is to allow the oAuth to function without modifying the meteor-tools file.

sebakerckhof commented 8 years ago

Exactly.

I've tried this before but I ran into errors described here: https://github.com/meteor/meteor/issues/4497

Thing is that there are 3 parties involved (meteor-tool, stratosphere and meteor.com servers) and all communication is asynchronous, which makes it very hard to debug.

hellogerard commented 8 years ago

@sebakerckhof Thanks for the clarification.

Basically if you have your packages.data.db you should copy it and rename it to a sanitized version of your package server url, in your case this would be: localhostX9000.data.db. Normally this should be visible in the instructions in the frontend somewhere. But I rewrote the frontend for beta1 (it used to be based on Telescope) and I might have missed this part.

and

Yeah, copying of the database is not always necessary. It depends on what command you execute, so the step is there to make sure it always works. However, once it has synchronized, you're good to go for every meteor-tool command.

These remarks might make good additions to the README. (I know, I know, pull request)

sebakerckhof commented 8 years ago

@rwatts3 This branch has meteor-developer-accounts enabled and a minimal front-end to test manual login https://github.com/sebakerckhof/stratosphere/tree/oauth I can login if I set the correct credentials in my settings.json file and I have to set my redirect url to: http://localhost:3000/_oauth/meteor-developer?close for manual login to work.

That should be enough to get you started (of course, You'll have to undo the changes to auth-client.js ).

@hellogerard Yeah, the readme can use a lot of improvements, but I'd rather focus on trying to make sure some steps are not necessary anymore.

sebakerckhof commented 8 years ago

@rwatts3 Actually, I just found out that I can login with meteor-tool 1.1.9 and the oauth branch. Can you verify?

You will however need to set the redirect url in your meteor app to: http://[your url:port]/_oauth/meteor-developer?close

and make sure the settings in settings.json are correct.

hellogerard commented 8 years ago

Everything is working for me now :+1:

rwatts3 commented 8 years ago

Yeah I verified the same thing. So there is no need to change the file anymore ? And we have full leverage of both tools On Wed, Oct 14, 2015 at 12:32 PM Gerard Sychay notifications@github.com wrote:

Everything is working for me now [image: :+1:]

— Reply to this email directly or view it on GitHub https://github.com/sebakerckhof/stratosphere/issues/2#issuecomment-148169208 .

rwatts3 commented 8 years ago

Ok so i'm having trouble setting the environment variable . Meteor still publishes to atmosphere.

sebakerckhof commented 8 years ago

That's... very strange. Really almost impossible if you see how simple the code is: https://github.com/meteor/meteor/blob/devel/tools/meteor-services/config.js#L139

Could you maybe do a console log there of your environment variables. Are you sure you've set them permanently on your system and not just temporarily (what the commands in the readme will do)?

rwatts3 commented 8 years ago

I'll try to set it again. I'm using windows right now mac is at home but it didn't quite work that well last time It published 2 packages to the atmosphere

rwatts3 commented 8 years ago

Actually , I am going to push my stratosphere to Digital Ocean so that I can use it across multiple projects. Do you know if the environment settings is for my own local computer or if it should be pushed with the stratosphere app as well.

rwatts3 commented 8 years ago

Ok ,

I think we should consider writing scripts to handle the editing of the environment variables and the db for the user. This will make it a more seamless project.

hellogerard commented 8 years ago

I usually prepend the environment variable on the command line, like this:

$ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor
$ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor add my:package
$ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor list
sebakerckhof commented 8 years ago

I've got them saved in my bash profile on linux and using the gui editor on windows.

I'm currently finishing a (quick and dirty) user management in the GUI and some smaller modifications for beta2.

2015-10-16 21:13 GMT+02:00 Gerard Sychay notifications@github.com:

I usually prepend the environment variable on the command line, like this:

$ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor $ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor add my:package $ METEOR_PACKAGE_SERVER_URL=http://localhost:3000 meteor list

— Reply to this email directly or view it on GitHub https://github.com/sebakerckhof/stratosphere/issues/2#issuecomment-148810112 .

rwatts3 commented 8 years ago

The oauth branch is gone ? I've got everything setup however I can't seem to login upon creating a new package.

sebakerckhof commented 8 years ago

The branch is merged into master.

rwatts3 commented 8 years ago

Receiving this error when trying to publish Windows 7

Error: access-denied
    at sendAuthorizeRequest (C:\tools\meteor-services\auth.js:351:11)
    at oauthFlow (C:\tools\meteor-services\auth.js:382:25)
    at Object.exports.loginWithTokenOrOAuth (C:\tools\meteor-services\auth.js:956:17)
    at Object.exports.loggedInConnection (C:\tools\meteor-services\auth-client.js:87:12)
    at Object.exports.loggedInPackagesConnection (C:\tools\packaging\package-client.js:232:21)
    at Command.main.registerCommand.name [as func] (C:\tools\cli\commands-packages.js:305:32)
    at C:\tools\cli\main.js:1378:23
sebakerckhof commented 8 years ago

I just released beta 2 which has a lot of changes regarding user authentication and management.

Note that the settings.json file template has changed (allowedUsers is renamed to superUsers).

Some remarks: -I cannot secure synchronization of packages, since meteor-tool does not try to authenticate for this. So if you want your packages to be truly private protect stratosphere in some other way. So authorization only works for publishing and unpublishing of packages.

-If loginRequired is set to true, you need to add at least one superUser (meteor developer account name). Super users can add and manage other users in a new part of the GUI.

rwatts3 commented 8 years ago

Thank you I will pull the latest update down now and give it a spin

rwatts3 commented 8 years ago

By the way I got it working. I think we can make some improvements to the UI. When I have some time, I'll create a branch on my repo and explore a little maybe, you'd be interested in some of the ideas.