search

rlidwka / sinopia

Private npm repository server
5.51k stars 656 forks source link

Project no longer maintained? #376

Open mattfysh opened 8 years ago

mattfysh commented 8 years ago

It seems this project is no longer being maintained, with a high number of unanswered pull requests and issues raised. The last version v1.4.0 was published 8 months ago. I've reached out to the author via email a week ago and have not yet received a response

Does anyone know the process here is, anyone interested in forking this project and publishing under a new name?

rbecheras commented 8 years ago

Yes I planned to support a fork of the project.

I think about a complete re-factored project to provide

The aim will be to take advantage of its advanced personalization features while continuing to benefit of the project's progress and updates when it evolves.

mattfysh commented 8 years ago

@vernak2539 ?

czardoz commented 8 years ago

A lot of forks seem to be present (presumably because of PRs not being merged) https://github.com/rlidwka/sinopia/network

RnbWd commented 8 years ago

I tried a year ago - it's the most complex library written in express I've ever seen

scic commented 8 years ago

I would love if this project would be actively maintained again. Currently the missing/broken pull-through of scoped packages is beginning to become a blocker for me. Sadly this code is above my understanding so I can't help.

RnbWd commented 8 years ago

I've started using scoped packages in sinopia before it was officially implemented in npm - there's a configuration in yaml that enables pulling @name/package's. I haven't used sinopia since the scopes were implemented officially and npm was upgraded to 3.0 - so I'm not sure if the scopes pull from npm or just act locally - but effectively it can be used instead of npm as a private repo - I used it w/scoped variables - but not via npm.

jdx commented 8 years ago

I've got a npm server that's a bit different than sinopia, but a lot simpler that might be worth a look: https://github.com/dickeyxxx/elephant

We're using it in production at Heroku with great success! It's very fast!

mima0815 commented 8 years ago

@rlidwka can you give us a short statement on the status of sinopia?

RnbWd commented 8 years ago

I just turned on my docker sinopia container for the first time in 6 months and it seems to work fine. Many of the problems posted here might be solvable by changing a few lines of code or simply making the config.yml work properly - which is honestly esoteric. But if it works.. it works. I consistently had a 10x speed improvement the entire time I used it hosted on a tiny docker server in the cloud - but I still haven't tried scoped variables since they were officially implemented. They worked fine last summer, but it requires the proper setup in the config

edit: scoped variables are working fine with me - but they're local to the sinopia. check out the configuration for my docker repo https://github.com/RnbWd/sinopia-docker

double edit: I'm using node 5.7.1 and npm 3.8.1 - but I can only get sinopia to run on 0.10 (it briefly worked with iojs) so my docker container - running sinopia in 0.10 - is probably more stable to use than if one were to just download sinopia 'vanilla' in a modern node context

ChadKillingsworth commented 8 years ago

Proxying scoped packages work just fine - you just have to set the config correctly (the default is wrong).

You need 2 asterisks:

packages:
  '**':
    # allow all users (including non-authenticated users) to read and
    # publish all packages
    #
    # you can specify usernames/groupnames (depending on your auth plugin)
    # and three keywords: "$all", "$anonymous", "$authenticated"
    access: $all

    # allow all known users to publish packages
    # (anyone can register by default, remember?)
    publish: $authenticated

    # if package is not available locally, proxy requests to 'npmjs' registry
    proxy: npmjs
RnbWd commented 8 years ago

I use this setting for scoped packages:

@/

scoped

allow_publish: $authenticated

I haven't used the "*" setting, I'm jw if there's an issue the "@/*" setting when proxying npm?

ChadKillingsworth commented 8 years ago

The matching algorithm uses the node glob library so it's the same matching rules. A single * won't match a /, but the double ** does.

*/* works just as well if you want to have a rule for just scoped packages.

ChadKillingsworth commented 8 years ago

I also emailed @rlidwka about a month ago and never heard back. He has near-continuous public activity on github for the past several months which leads me to believe he is purposefully ignoring this project.

A fork, name change and license change would be in order it seems. I just don't have the time right now to maintain a package like this appropriately.

RnbWd commented 8 years ago

I'd volunteer to give it shot - I'm maintaining the docker container that seems to have the most downloads for this project - and I had to forked it for a minor feature so I could proxy it via ssl with minimal config. I've read every line of the code base, so I have a mental model of its structure. The foundation uses libraries that are not compatible with modern nodejs. Those sections can be rewritten, the library can be modernized, i know where it needs to change - but it's ambitious. I attempted to modernize it a year ago, but none of that's relevant now.

I need to think this over for a few days - it's not a trivial library to maintain. On Sun, Mar 13, 2016 at 7:25 PM Chad Killingsworth notifications@github.com wrote:

I also emailed @rlidwka https://github.com/rlidwka about a month ago and never heard back. He has near-continuous public activity on github for

cuzzinz commented 8 years ago

It would be amazing for this project to be forked and modernized. It is by far the fastest solution even compared to local-npm and lazy_npm.

I have compared a few different solutions and sinopia is faster by 15-30%. It is very impressive.

dgaya commented 8 years ago

First, thanks @rlidwka for this great software.

I already maintain a fork, https://github.com/fl4re/sinopia It is not intended to become the 'official' fork. In fact there are other forks (for example @mathieumg or @ngash) quite active too.

Our fork is internally used at my company, therefore we are going to maintain it in the future.

In this fork I have already merged some relevant PRs:

And we have a branch (OAuth_login) that replaces completely the registration/login workflow by Github/Google Oauth instead of plain and open user password. Beside this branch, I plan to maintain the master branch as close as possible to the original sinopia.

If anyone in the community considers our fork a good fork, I will be glad to accept PR. PRs will be accepted if:

PRs will not be accepted if:

ChadKillingsworth commented 8 years ago

@dgaya Thanks! I'll send a PR or 2 your way. Also, would you consider switching the license on your fork? I'm not sure of the implications of that, but the current license has always made me nervous.

RnbWd commented 8 years ago

I think I'm going to create an org - what should we call it? I don't want to use sinpopia unless @rlidwka is okay with it (love your work btw). I have some urls - like.. 'karmasync' 'event-loop' 'synctea' all .coms I'm not using at the moment (well karmasync links to a github clone and synctea is my sinopia) not that it matters but they're cheap and i don't think anyone is using them.

I want to collaborate with a group - it's not a one man project. there are other projects that are similar and it might be nice to merge with them, but none are as good as sinopia. Considering it seems to work perfectly fine for me at the moment

Feel free to use https://synctea.com for some http2 sinopia awesomeness.

I think this project deserves a working group.

edit: npm config set strict-ssl false if you decide to use synctea.com

edit: also i like what @dgaya did - but my original attempt was far more ambitious and changed lots of dependencies and code styles -- too bad I lost it - it's not worth looking for now

dsuckau commented 8 years ago

I really hope this project will be maintained again. We use sinopia in our company for private packages. We are kind of dependent on it now, but don't trust this repository anymore. I would contribute too....

donny-dont commented 8 years ago

@RnbWd maybe https://en.wikipedia.org/wiki/Sinopia would provide some inspiration.

Some things that pop out from it are

RnbWd commented 8 years ago

Thanks - I haven't started working on anything yet but I agree with everyone that this project should be maintained. Sinopia seems to work for most things - i did notice some weirdness with @* names being pulled from npm - I just want a general consensus about what everyone is looking for, I'll need to go though the issues.

Syncing with external store (s3) is something I see mentioned constantly. I host using Docker - I can sync all the repos with an external volume that I can send to s3 or something, and I proxy with nginx. But if you're just relying on sinopia, adding hooks in the repo itself seems to be one of the most requested features, so that's one major feature that we need to decide if we want to expand it - or use external libraries for the management (like an external file watcher syncing repos)

But the critical necessity is to make sure that it will remain compatible with npm itself. It currently uses an older version of express to fake npm's backend - so if npm's api changes, then those will need to be reflected manually as express routes. But if modules from npm's repo could be used instead, then it might be possible simplify the express portion of the app without worrying about future compatibility. -- I need to create multiple test branches to see what's possible

RnbWd commented 8 years ago

I found a ton of resources here - awesome-npm

Here are a few modules that I think sinopia would benefit from:

food for thought :satisfied:

cuzzinz commented 8 years ago

This line stood out on the wiki for me:

To make the faces in the final fresco, Cennini recommended that the artist first paint them with an undercoat of brownish green, called a verdaccio [...] then used a fine brush and sinopia to paint the lines under the eyes, around the nose, the eyebrows, the mouth and the shading under the upper lip.

verdaccio - a green color popular in late medieval Italy for fresco painting

Since it will be a fork, follow the subject the orig project used but a new "color." .... verdaccio

Just my 2 cents

P.S. I will definitely be testing out whatever fork is created. I am using it to help eliminate the need to locally cache any modules on a jenkins slave that is building via a docker container. Smaller footprint.

jdx commented 8 years ago

@RnbWd if you look at my project it does S3 syncing much better.

RnbWd commented 8 years ago

I'll definitely check it out - I haven't started working on anything yet. If the startup I was working at last year didn't dissolve I'd have started a fork 6 months ago - but I was able to brainstorm quite a bit - and the iojs fork / es6 features made me hesitant to start any ambitious projects On Tue, Apr 5, 2016 at 10:49 AM Jeff Dickey notifications@github.com wrote:

@RnbWd https://github.com/RnbWd if you look at my project https://github.com/dickeyxxx/elephant it does S3 syncing much better. We use it in production at Heroku and it handles 1000 RPM beautifully.

— You are receiving this because you were mentioned.

Reply to this email directly or view it on GitHub https://github.com/rlidwka/sinopia/issues/376#issuecomment-205917410

jmwilkinson commented 8 years ago

What's the status on this? Is someone going to create an org called "verdaccio" and fork sinopia?

I will do it if I have to. I have just about 0 time to maintain it, though, so everyone here who is interested would have to help contribute, and hopefully shift the community over. To that end, a primary objective would be no api breaks.

If no one has taken the initiative by Monday, I'll go ahead and make it then.

RnbWd commented 8 years ago

I can't get mine to work anymore..

trentearl commented 8 years ago

This project is important to me and my company, I have created the organization: verdaccio

And forked the repo https://github.com/verdaccio/sinopia

I've added @RnbWd and happy to add others well.

I look forward to seeing this project continue to grow.

trentearl commented 8 years ago

Renamed to: https://github.com/verdaccio/verdaccio

cuzzinz commented 8 years ago

Not a real issue per se but definitely should start going through the code and replace sinopia with verdaccio. I will not be using this project for a while as things have changed gears from me but I will definitely be following it and will contribute any way I can.

:-D glad someone used the name I posted.

RnbWd commented 8 years ago

thx :) I hadn't made any changes - I just noticed inconsistent behavior with every single version of node - it needs to be fixed

ghost commented 8 years ago

So I'm relieved to see there's some movement on jump starting this project again, as the original author has abandoned it. We rely on it, and I recently found a bug which I urgently need to fix.

I note that @dgaya kindly agreed to maintain one fork, but does not intend it to become official. I feel we should all get behind a single new fork, and this new one which @trentearl has created for us gets my vote. Is verdaccio going to be the official one going forward? I like that name, and I've already logged an issue there which I intend to fix in the near future.

cuzzinz commented 8 years ago

I am glad this was forked. I have a few small default configuration changes I was going to try and submit. Like changing the default logging to file instead of stdout. Also I need to find the Ansible script I used for sinopia and see if they can participate in the verdaccio group.

I think verdaccio should have some small adjustments to function more like legitimate server rather than a one off that you run from the command line when you need it. Something that can be run in the background as a daemon and started on boot in case if server failure. Most people can figure this out on their own but it would be nice for it to have all these functions build in and instructions for the less system savy node developer.

cuzzinz commented 8 years ago

Also a docker version might make sense for someone that is running their application in an unsupported node version for sinopia. While everyone works together to figure out the versions of node that work with sinopia and fix/validate a go forward for node version support.

RnbWd commented 8 years ago

Docker provides a parallel way to decouple the content of sinopia (repos, login info) - and sync it with an s3 store by using volumes - and then any program running on the cpu can watch the folder for changes (gulp or some equivalent if using nodejs) - or the volume can be decoupled and shared with another docker container that's syncing files from s3 (or anything) - and the changes will be synced with the original sinopia container. It could be added into sinopia's core - but the larger it gets the harder it'll be to maintain, and I like the idea of a 'universal' storage sync completely independent of how sinopia just uses the fs to store everything. What @rlidwka wrote simple, brilliant and effective - it just needs another process running to sync everything between different machines

edit; to clarify - a dockerfile should be included in the main repo - also extending the config.yaml plugin system for docker or different services without docker would be useful - or proxying with nginx, etc.

olalonde commented 8 years ago

Whoever is going to maintain this I have a feature request: make it a 12-factor app and use https://github.com/maxogden/abstract-blob-store for the storage layer so I can run it on Heroku/Deis/other PaaS :)

RnbWd commented 8 years ago

@olalonde I've wanted to use abstract-blog-store since I started using sinopia... I haven't made much progress, but I love the idea.

I've thought lot about this project and future directions it could go (hammock driven development 😝 ). I feel like there's 2 potential directions it could go:

1) Maintain the express api designed to behave like the npm registry

2) Leverage the libraries used by npm itself to build their registry and cli app

Either way, (the second option might be better in the long run, but it would be a complete rewrite), it should still function as independent layer in-between npm.org and the user, and it should work with modern nodejs and the npm registry as it functions today. I have other ideas, but I need to actually hash some code because I still don't know what's up with this project. But if I start working on it, a lot will happen very quickly and I'll people here know. Can't promise anything though.

Delagen commented 8 years ago

moved to www.sonatype.com/nexus-repository-oss and glad.

todkap commented 8 years ago

My team has forked Sinopia internally and added persistence support w/ CouchDB/Cloudant. I would be more than willing to donate this code out to the community in any shape makes sense. The code is actually quite clean in terms of adding in a persistence layer (basically replacing FS with CouchDB and changing some require statements) but does require some extra work to make it configurable to allow the deployer to select between FS and CouchDB.

qballer commented 8 years ago

So we have two projects: https://github.com/verdaccio/verdaccio https://github.com/fl4re/sinopia

Any ideas which fork to choose ?

todkap commented 8 years ago

@qballer To me, it would be great if we could get the two forked projects to determine if makes sense to collaborate and consolidate down to one fork (I cannot easily determine how much the two code bases overlap at this point but one has about 2x the number of commits to it). From what I can tell, this current project appears to be stagnant and if we could get another one to drive forward with, IBM would be willing to donate our persistence update to support CouchDB.

jmwilkinson commented 8 years ago

My vote is to merge fl4re/sinopia into verdaccio. I think this project is important enough to have a community-driven group/repository. Furthermore, we can't republish "sinopia" to npm.

As part of that, though, I think verdaccio needs to have public members, so the community can see who maintains it. Furthermore, it needs to have a relatively significant number of constant contributors, so issues get resolved and pull requests get merged.

aeweidne commented 8 years ago

@qballer @jmwilkinson I've been approved to externalize the Cloudant based fork and continue contributions. The only problem I could see is that we've based our fork on top of this original project and not the fl4re one, so merging may be less trivial. As @todkap said, there is also some additional work around making FS or Cloudant an option to the end user.

There are additional packaging aspects that could be improved as well, but those are definitely issues to be sorted out when we settle in on a new home :).

Any ideas re: name? Not being able to publish to sinopia is certainly an issue.

trentearl commented 8 years ago

I currently maintain and own the verdaccio (https://github.com/verdaccio/verdaccio) fork. It doesn't have a huge amount of momentum and no version has been publish to npm. My main goal is to have a maintained npm server for small companies to use.

From these discussions I see two options:

  1. Merge the fl4re fork into verdaccio and adding public more members as @jmwilkinson has suggested.
  2. Scrapping verdaccio and trying to push the community to the cloudant fork. I dont know much about the fork but I would assume a fork from an established company would be better for the community. I would be happy to donate the name and repo to cloudant if the this option gets support and the name is liked.

Opinions?

@aeweidne and @todkap can you tell us more about this fork, do you plan on supporting fs packages, it seems like we would need that. Are you using this internally so that the company needs it? or is it more of a pet project?

aeweidne commented 8 years ago

There would have to be some analysis done by the community stakeholders. I could get our fork out as an external fork of this repository to let @trentearl and co. get eyes on it.

We were in need of a quick and dirty private npm solution that could be used for external deployments, and after our quick changes to get Cloudant functionality in, haven't made any updates because its been surprisingly stable. Since there is no internal roadmap at the moment (including plans re: fs), you could call it a pet project. Externalizing it would be breathing some new life into the work.

It may be that people who have been with the project longer may wish to extract out our Cloudant functionality and rebase it on top of another fork, either way I am willing to contribute. The tests weren't in great condition when we forked it originally, so I can't say much about how much we may have broken the fs capabilities. The cloudant functionality (correct me if I'm wrong @todkap) was written such that couch could be plugged in instead without any code changes. That could be an option for local runtimes that prefer not to depend on Cloudant.

todkap commented 8 years ago

Thanks @aeweidne for the response. Let me add a little more context. To Alex's point, we have been hosting an internal NPM registry for multiple projects for a select number of IBM cloud deployments. The code has been surprisingly stable as noted and there have been a few bug fixes that were done by various team members after the initial cloudant/couchdb support was added.

The libraries we use today are vanilla couchdb ones (nano) so it should work with couchdb but only has been tested with cloudant.

The code itself simply comments out the "requires" for the various "fs" related libraries. The work that would be needed would be to allow a way to denote "fs" or "database" persistence in some config file or startup arguments. We have not had to touch much of the core code to get this working to be honest (it is very minimal). I tried to be smart in naming them to make them easy to find. I think I delivered about 4-5 js files in this entire fork :)

We would love to have this be community driven and having the best of all of these forks combined to deliver an exceptional experience would be great. I am sure that there are some gaps (security could be improved for instance) that having this be actively developed and leveraged by developers would certainly bring attention to these gaps and hopefully closed :)

aeweidne commented 8 years ago

I have externalized our fork at https://github.com/aeweidne/sinopia. I figure @trentearl @jmwilkinson and co may want to look it over and decide if its wanted in verdaccio and if so, the best way to get it in.

qballer commented 8 years ago

Diversity is good. I believe that with some abstraction one project can accommodate different storage needs. I personally favor keeping it open (org?) and have different stack holders given privileges to the project.

todkap commented 8 years ago

i think our cloudant fork can easily be integrated with one of these other forks since it is very unobtrusive. The part that needs some cleanup is the "accommodate different storage". However, I think what we showed in our fork is that the code sort of lends itself to this type of extensions. The part that needs work is to have some form of factory that instantiates the different plugins based upon config. There are three files with the name "couchdb" in the lib directory. Take a look at them and also look at something like storage where we changed the required for local to something like var Local = require('./local-storage-couchdb') from ./local-storage

Can I make a suggestion for the name :) I sort of like the name irrawaddy as in irrawaddy dolphin but this may be too hard to type npm install on :)

todkap commented 8 years ago

We wrote a recipe on how to deploy Sinopia to IBM's Bluemix and the benefits we received. As we continue to evaluate next steps, it would be great for the community to see how we benefited from Sinopia and why we really are interested in helping drive this community going forward.

https://developer.ibm.com/recipes/tutorials/deploying-your-own-private-node-package-manager-to-bluemix/