Not listing packages

[bradenpowers]

If I navigate to my local sinopia repository it doesn't show a listing of the packages that I have on the server even though I have installed numerous packages and I see them in the file system.

[lambert123]

I'm having the exact same issue and would be interested in the resolution.

[dsuckau]

For me the same now suddenly. I updated two weeks ago to 1.0.0-beta2 and restarted sinopia, everything was fine. My packages were still listed in the web UI. But today, I restarted it again and the web UI says "No packages". But they are still in my local_storage folder and via npm install I can install them.

[lambert123]

I would love a lead on how to fix this. Since the packages exist, and users can use sinopia in general, it feels like it's not looking in the right place to me. I'm just not sure how to track it down.

[rlidwka]

1. can you try to explicitly provide a path to the config file (i.e. sinopia -c /path/to/config.yaml)?
2. is there a file named .sinopia-db.json in the storage folder? (that's where local package list gets stored... a quirk that needs to be reconsidered though)

[dsuckau]

1.) I already did provide a path to the config file:

command=sinopia -c /opt/sinopia/config.yaml

(from my supervisord config)

2.) I have two storage folders: /opt/sinopia/storage and /opt/sinopia/local_storage. In local_storage are all my private packages, but no .sinopia-db.json. In storage are the cached public packages from npmjs and a .sinopia-db.json, that only contains the titles of my private packages in a "list" array and a hashed "secret".

Is that how it should be?

[rlidwka]

@dsuckau , yeah, it looks how it should. I don't know where the issue is coming from. :(

[lambert123]

Mine starts and states the path to the config file it uses, and that file has:

storage: /npm/sinopia/storage

In that directory I have 389 package directories:

09:19:51 2 0> ls -l .sinopia-db.json -rw-r--r-- 1 foobar development 87 Dec 22 09:08 .sinopia-db.json

09:19:59 2 0> file .sinopia-db.json .sinopia-db.json: ASCII text, with no line terminators

09:20:05 2 0> more .sinopia-db.json {"list":[],"secret":"giant_string_of_numbers"}

[lambert123]

Still interested in resolution if possible.

[rellaguo]

I also have the same issue. I cannot see any package on sinopia web GUI. In .sinopia-db.json, I can see: {"list":[],"secret":"c649864c562c335aa53ce9cd8f64681d05a2f25bbab4c7399bd0c5a9858d0d40"}

But under /home/dev/.local/share/sinopia/storage, there are several packages. [dev@devhost storage]$ ls ansi-styles chalk diskdb has-color merge node-uuid strip-ansi

[rlidwka]

@rellaguo , it only shows packages that were manually uploaded (with npm publish) previously. It won't display packages fetched from npm (otherwise it'll have to show thousands of dependencies, which the interface wasn't designed for).

[rellaguo]

@rlidwka , thank you. How to know which packages have been cached by sinopia?

[rlidwka]

Checking the storage folder as you did above is the only way right now. Why do you need to know this?

[rellaguo]

as a repository, I suppose user would want to know which packages have been cached. My case is many projects share one repository. We only cache what we need. Once all packages are ready, we will turn off public registry.

[dsuckau]

I'am a little wondered about that a 1.0.0 can be released without this bug being fixed.. :/

Still no packages listing in 1.0.0...

[noamokman]

:+1: I'm using a sinopia server on a closed network and the cached packages is all we use, It would be nice to see them with the web interface.

Is there any progress on the matter?

[ghost]

:+1: Would love to see this bug fixed please?

[dsuckau]

I did put some console.log's in my index-web.js

storage.get_local(function(err, packages) {

  console.log('get packages')
  console.log('packages count', packages.length)
  console.log('allowed packages count', packages.filter(allow).length)
  console.log(req.remote_user)
  console.log(config.packages)

  if (err) throw err // that function shouldn't produce any
  next(template({
    name:       config.web && config.web.title ? config.web.title : 'Sinopia',
    packages:   packages.filter(allow),
    baseUrl:    base,
    username:   req.remote_user.name,
  }))

})

The result:

get packages
packages count 8
allowed packages count 0
{ name: undefined,
  groups: 
   [ '$all',
     '$anonymous',
     '@all',
     '@anonymous',
     'all',
     'undefined',
     'anonymous' ],
  real_groups: [] }
{ 'cat-*': 
   { allow_access: [ 'cat' ],
     allow_publish: [ 'cat' ],
     storage: 'local_storage',
     proxy_access: [],
     proxy_publish: [],
     access: [],
     proxy: [],
     publish: [] },
  '*': 
   { allow_access: [ 'cat' ],
     allow_publish: [ 'cat' ],
     proxy: [ 'npmjs' ],
     proxy_access: [],
     proxy_publish: [],
     access: [],
     publish: [] } }

So I see no packages in my web interface, because they get filtered by user rights in index-web.js:

function allow(package) {
  return config.allow_access(package.name, req.remote_user)
}

Function allow_access() in config.js:

Config.prototype.allow_access = function(package, user) {
  return allow_action.call(this, package, user.groups, 'allow_access')
      || allow_action.call(this, package, user.groups, 'access')
}

function allow_action(package, who, action) {
  return (this.get_package_setting(package, action) || []).reduce(function(prev, curr) {
    if (typeof(who) === 'string' && curr === who) return true
    if (Array.isArray(who) && who.indexOf(curr) !== -1) return true
    return prev
  }, false)
}

Config.prototype.get_package_setting = function(package, setting) {
  for (var i in this.packages) {
    if (minimatch.makeRe(i).exec(package)) {
      return this.packages[i][setting]
    }
  }
  return undefined
}

So the user groups of the web interface request are getting matched against the allow_access property of each packages. In my case it always returns false because the web interface user has the groups ['$all', '$anonymous', '@all', '@anonymous', 'all', 'undefined', 'anonymous'], but my packages configuration says:

packages:
  'cat-*':
    allow_access: cat
    allow_publish: cat
    storage: 'local_storage'
  '*':
    allow_access: cat
    allow_publish: cat
    proxy: npmjs

So my packages have allow_access: [ 'cat' ], allow_publish: [ 'cat' ].

I configured it like this, because I only want the npm user cat to publish and install private and public packages as well. How should the web interface user ever get the group cat?

Maybe I misconfigured something?

[rlidwka]

@dsuckau , no, it's configured correctly. Names like all, anonymous are ones we used in 0.x versions, they are deprecated now, but still supported because of backward compatibility reasons.

[dsuckau]

Yes, but could you say something more about why the web interface expects the username of the packages configuration and where it should get it from?

[rlidwka]

How should the web interface user ever get the group cat?

Technically, when user cat logs in, it'll automatically get assigned a group named cat. So user and group are kinda the same thing (except when you're using advanced auth plugins like LDAP).

Did I understand the question correctly?

[dsuckau]

Yes, you did unterstand it correctly. But in my case the web interface is public, there is no auth plugin. The cat user is only for the npm client.

[MagIciaNGTAO]

:+1: for this feature

[dsuckau]

This is not a feature request, it is a bug report.....

[frvge]

An overview of the packages that are proxied from upstream is also appreciated. Either as a list (per first-letter?) or a count.

[dsuckau]

@frvge Yes, of course, would be a nice feature. But I think there should be a new issue for it, phrased as feature request. Because this issue was originally opened for the bug, that the web UI does not list any local packages anymore.

@rlidwka I updated to 1.2.1 today and now have a Login-form on the upper right (default web UI). There I can use my npm user and password and see again my packages.

But when I am not logged in, it still says "no packages".

Is this correct now? Is there no way anymore to see the packages in the web UI without being logged in?

@bradenpowers @lambert123 @khushil Does it work for you now, too, with the login form?

[frvge]

@dsuckau Thanks. I opened #235 .

[KeithWoods]

Just a note on this. I had a similar issues after migrating sinopia from one box to another. I too had different folders for public and private packages, in the public directory there was a .sinopia-db.json.

The web gui wasn't listing the packages however they were getting served by sinopia ok. I copied .sinopia-db.json from the old box to the new box and the web gui then correctly displayed the packages. i.e.:

{"list":["private-package1","private-package2"],"secret":"longAlphaNumericTextGoeshere"}

[Tinha4Sale]

I am having a similar issue and I am able to fixed it by copying the ".sinopia-db.json" file to the Sinopia local_storage folder (this is the folder for our private repositories). My .sinopia-db.json file do not contains the line ""secret":"longAlphaNumericTextGoeshere"" thou, but the web browser is now able to display our private repository list.
NPM version is 2.1.8 Node version is 0.10.33
Sinopia version is 0.13.2

[weijyewang]

@Tinha4Sale I did that same thing. It works on last Friday, but today is no longer working. The web gui shows "No Packages" and the .sinopia-db.json list is empty.

[Tinha4Sale]

@weijye91 Any changes on your environment, or did someone deleted the ".siopia-db.json" file? If you have a back up copy of that file, perhaps you can can restore it from back up. Unfortunately I am not an expert on Sinopia, most of my knowledge is learned through Google.

[weijyewang]

@Tinha4Sale I tried to copy the json file again and it is working this time. Had no idea why.

[atticoos]

I think there might be some misconceptions in this thread -- isn't the list of packages on the Web GUI simply to show packages uploaded to the private registry, not public packages that have been cached from npmjs.org?

If anything, I could see a feature being added to show the cached packages. But I don't think this was the original intention of the web GUI

[noamokman]

Some of us use sinopia as an npm server on a closed network, the cached modules are all there is to the server, this makes the Web ui irrelevant as we can't access npmjs.

[mafshin]

My storage is full of packages and the private repo works as expected while the Web GUI doesn't list anything.

My .sinopia-db.json file was like this:

{"list":[],"secret":"1ae ...."}

Once I manually updated the list array, I could see the added package in Web GUI listing:

{"list":["typescript", "yaml"],"secret":"1ae ...."}

The problem is that list array is not updated automatically upon install requests.

[storjarn]

I'm using verdaccio, the actively maintained fork of sinopia, and experienced the same thing this morning. The error in the logs when I couldn't restart the service successfully was:

`Starting verdaccio... warn --- config file - /etc/verdaccio/config.yaml

fs.js:439 return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode); ^ Error: EACCES, permission denied '/var/lib/verdaccio/storage/.sinopia-db.json'`

I checked, and my /var/lib/verdaccio folder, which should be owned by the verdaccio user, was chown'd to root. Someone on my team or our puppet probably did this. I chown'd it back to verdaccio and it worked again. Maybe hoping this helps someone.