如何在本地(内部局域网)实现下载 公共仓库的包

[johnsonGong]

背景:

1. 公司的开发环境为: 独立的 内部局域网, 即：不能连接互联网;
2. 想搭建内部 cnpm源, 实现 npm 的安装;
3. cnpmjs.org 的服务器环境是 linux CentOS; 现状:
4. 已实现 同步公共仓库比如: lodash 到本地 /root/.cnpmjs.org/nfs
5. registryHost: '192.x.x.x' // 无效；
6. 修改服务器上， lodash 其中一个包(lodash-4.17.11.tgz) 中的文件(package.json 增加量 描述，一次判断下载的版本 是 外网-淘宝，还是内网)；
7. 本地 cnpm修改 registry 指向 cnpm 服务器； 结果： 下载的 lodash 还是 外网的包!!

[johnsonGong]

希望是， 公共仓库同步到本地服务器后，执行 cnpm install 能从本地服务器下载，而不是 外网。

[johnsonGong]

'use strict';

var mkdirp = require('mkdirp'); var copy = require('copy-to'); var path = require('path'); var fs = require('fs'); var os = require('os'); var utility = require('utility');

var version = require('../package.json').version;

var root = path.dirname(__dirname); var dataDir = path.join(process.env.HOME || root, '.cnpmjs.org');

var config = { version: version, dataDir: dataDir,

/**

• Cluster mode */ enableCluster: false, numCPUs: os.cpus().length,

  /*

• server configure */

  registryPort: 7001, webPort: 7002, // bindingHost: '127.0.0.1', // only binding on 127.0.0.1 for local access

  // debug mode // if in debug mode, some middleware like limit wont load // logger module will print to stdout debug: process.env.NODE_ENV === 'development', // page mode, enable on development env pagemock: process.env.NODE_ENV === 'development', // session secret sessionSecret: 'cnpmjs.org test session secret', // max request json body size jsonLimit: '10mb', // log dir name logdir: path.join(dataDir, 'logs'), // update file template dir uploadDir: path.join(dataDir, 'downloads'), // web page viewCache viewCache: false,

  // view files directory viewDir: path.join(root, 'view', 'web'),

  // config for koa-limit middleware // for limit download rates limit: { enable: false, token: 'koa-limit:download', limit: 1000, interval: 1000 60 60 * 24, whiteList: [], blackList: [], message: 'request frequency limited, any question, please contact fengmk2@gmail.com', },

  enableCompress: false, // enable gzip response or not

  // default system admins admins: { // name: email fengmk2: 'fengmk2@gmail.com', admin: 'admin@cnpmjs.org', dead_horse: 'dead_horse@qq.com', },

  // email notification for errors // check https://github.com/andris9/Nodemailer for more informations mail: { enable: false, appname: 'cnpmjs.org', from: 'cnpmjs.org mail sender adderss@gmail.com', service: 'gmail', auth: { user: 'address@gmail.com', pass: 'your password' } },

  logoURL: 'https://os.alipayobjects.com/rmsportal/oygxuIUkkrRccUz.jpg', // cnpm logo image url adBanner: '', customReadmeFile: '', // you can use your custom readme file instead the cnpm one customFooter: '', // you can add copyright and site total script html here npmClientName: 'cnpm', // use ${name} install package packagePageContributorSearch: true, // package page contributor link to search, default is true

  // max handle number of package.json dependencies property maxDependencies: 200, // backup filepath prefix backupFilePrefix: '/cnpm/backup/',

  /**

• database config */

  database: { db: 'cnpmjs', username: 'tgbsxf', password: '123456',

  // the sql dialect of the database // - currently supported: 'mysql', 'sqlite', 'postgres', 'mariadb' dialect: 'mysql',

  // custom host; default: 127.0.0.1 host: '127.0.0.1',

  // custom port; default: 3306 port: 3306,

  // use pooling in order to reduce db connection overload and to increase speed // currently only for mysql and postgresql (since v1.5.0) pool: { maxConnections: 10, minConnections: 0, maxIdleTime: 30000 },

  dialectOptions: { // if your server run on full cpu load, please set trace to false trace: true, },

  // the storage engine for 'sqlite' // default store into ~/.cnpmjs.org/data.sqlite storage: path.join(dataDir, 'data.sqlite'),

  logging: !!process.env.SQL_DEBUG, },

  // package tarball store in local filesystem by default nfs: require('fs-cnpm')({ dir: path.join(dataDir, 'nfs') }), // if set true, will 302 redirect to nfs.url(dist.key) downloadRedirectToNFS: false,

  // registry url name, r.cnpmjs.org registryHost: '106.12.77.25:7001',

  /**

• registry mode config */

  // enable private mode or not // private mode: only admins can publish, other users just can sync package from source npm // public mode: all users can publish enablePrivate: false,

  // registry scopes, if don't set, means do not support scopes scopes: [ '@cnpm', '@cnpmtest', '@cnpm-test' ],

  // some registry already have some private packages in global scope // but we want to treat them as scoped private packages, // so you can use this white list. privatePackages: [],

  /**

• sync configs */

  // the official npm registry // cnpm wont directly sync from this one // but sometimes will request it for some package infomations // please don't change it if not necessary officialNpmRegistry: 'https://registry.npmjs.com', officialNpmReplicate: 'https://replicate.npmjs.com',

  // sync source, upstream registry // If you want to directly sync from official npm's registry // please drop them an email first sourceNpmRegistry: 'https://registry.npm.taobao.org', sourceNpmWeb: 'https://npm.taobao.org',

  // upstream registry is base on cnpm/cnpmjs.org or not // if your upstream is official npm registry, please turn it off sourceNpmRegistryIsCNpm: true,

  // if install return 404, try to sync from source registry syncByInstall: true,

  // sync mode select // none: do not sync any module, proxy all public modules from sourceNpmRegistry // exist: only sync exist modules // all: sync all modules syncModel: 'exist', // 'none', 'all', 'exist'

  syncConcurrency: 1, // sync interval, default is 10 minutes syncInterval: '10m',

  // sync polular modules, default to false // because cnpm can't auto sync tag change for now // so we want to sync popular modules to ensure their tags syncPopular: false, syncPopularInterval: '1h', // top 100 topPopular: 100,

  // sync devDependencies or not, default is false syncDevDependencies: false, // try to remove all deleted versions from original registry syncDeletedVersions: true,

  // changes streaming sync syncChangesStream: false, handleSyncRegistry: 'http://127.0.0.1:7001',

  // default badge subject badgeSubject: 'cnpm', // defautl use https://badgen.net/ badgeService: { url: function(subject, status, options) { options = options || {}; let url = https://badgen.net/badge/${utility.encodeURIComponent(subject)}/${utility.encodeURIComponent(status)}; if (options.color) { url += /${utility.encodeURIComponent(options.color)}; } if (options.icon) { url += ?icon=${utility.encodeURIComponent(options.icon)}; } return url; }, },

  packagephobiaURL: 'https://packagephobia.now.sh', packagephobiaSupportPrivatePackage: false,

  // custom user service, @see https://github.com/cnpm/cnpmjs.org/wiki/Use-Your-Own-User-Authorization // when you not intend to ingegrate with your company's user system, then use null, it would // use the default cnpm user system userService: null,

  // always-auth https://docs.npmjs.com/misc/config#always-auth // Force npm to always require authentication when accessing the registry, even for GET requests. alwaysAuth: false,

  // if you're behind firewall, need to request through http proxy, please set this // e.g.: httpProxy: 'http://proxy.mycompany.com:8080' httpProxy: null,

  // snyk.io root url snykUrl: 'https://snyk.io',

  // https://github.com/cnpm/cnpmjs.org/issues/1149 // if enable this option, must create module_abbreviated and package_readme table in database enableAbbreviatedMetadata: false,

  // global hook function: function* (envelope) {} // envelope format please see https://github.com/npm/registry/blob/master/docs/hooks/hooks-payload.md#payload globalHook: null,

  opensearch: { host: '', }, };

if (process.env.NODE_ENV === 'test') { config.enableAbbreviatedMetadata = true; }

if (process.env.NODE_ENV !== 'test') { var customConfig; if (process.env.NODE_ENV === 'development') { customConfig = path.join(root, 'config', 'config.js'); } else { // 1. try to load $dataDir/config.json first, not exists then goto 2. // 2. load config/config.js, everything in config.js will cover the same key in index.js customConfig = path.join(dataDir, 'config.json'); if (!fs.existsSync(customConfig)) { customConfig = path.join(root, 'config', 'config.js'); } } if (fs.existsSync(customConfig)) { copy(require(customConfig)).override(config); } }

mkdirp.sync(config.logdir); mkdirp.sync(config.uploadDir);

module.exports = config;

config.loadConfig = function (customConfig) { if (!customConfig) { return; } copy(customConfig).override(config); };

[rhinoceros]

背景:

1. 公司的开发环境为: 独立的 内部局域网, 即：不能连接互联网;
2. 想搭建内部 cnpm源, 实现 npm 的安装;
3. cnpmjs.org 的服务器环境是 linux CentOS; 现状:
4. 已实现 同步公共仓库比如: lodash 到本地 /root/.cnpmjs.org/nfs
5. registryHost: '192.x.x.x' // 无效；
6. 修改服务器上， lodash 其中一个包(lodash-4.17.11.tgz) 中的文件(package.json 增加量 描述，一次判断下载的版本 是 外网-淘宝，还是内网)；
7. 本地 cnpm修改 registry 指向 cnpm 服务器； 结果： 下载的 lodash 还是 外网的包!!

建议在公司内部新做一个lodash，命名成私有包：@your_company/lodash ,重新发布。 不建议你自己直接改nfs下的文件。（时间久了，改的多了，自己都记不清楚有哪些是改过的；而且自己改这类公共的包，可能造成其他公共包也直接应用了你改过的版本，引起混乱）

你可以考虑用这种模式： https://github.com/cnpm/cnpmjs.org/wiki/Different-Modes Private + Scope Mode config.enablePrivate = true && config.scopes = ['@cnpm', '@cnpmtest'] Everyone can install from this registry Only admins can publish to this registry Can publish with scope, and only in @cnpm and @cnpmtest

[johnsonGong]

背景:

1. 公司的开发环境为: 独立的 内部局域网, 即：不能连接互联网;
2. 想搭建内部 cnpm源, 实现 npm 的安装;
3. cnpmjs.org 的服务器环境是 linux CentOS; 现状:
4. 已实现 同步公共仓库比如: lodash 到本地 /root/.cnpmjs.org/nfs
5. registryHost: '192.x.x.x' // 无效；
6. 修改服务器上， lodash 其中一个包(lodash-4.17.11.tgz) 中的文件(package.json 增加量 描述，一次判断下载的版本 是 外网-淘宝，还是内网)；
7. 本地 cnpm修改 registry 指向 cnpm 服务器； 结果： 下载的 lodash 还是 外网的包!!

建议在公司内部新做一个lodash，命名成私有包：@your_company/lodash ,重新发布。 不建议你自己直接改nfs下的文件。（时间久了，改的多了，自己都记不清楚有哪些是改过的；而且自己改这类公共的包，可能造成其他公共包也直接应用了你改过的版本，引起混乱）

你可以考虑用这种模式： https://github.com/cnpm/cnpmjs.org/wiki/Different-Modes Private + Scope Mode config.enablePrivate = true && config.scopes = ['@cnpm', '@cnpmtest'] Everyone can install from this registry Only admins can publish to this registry Can publish with scope, and only in @cnpm and @cnpmtest

谢谢，把公共资源"包装"成私有资源， 想过这个方案，可是 太多了。。。后边再试试。

---

跟领导汇报过： 使用CNPM 搭建内网私有NPM源平台，特别是有 防火墙 (我们是 物理隔断。。。)的情况下， 暂不能实现。 原因: CNPM会检查 包的版本，查看是否有更新 --- 必须链接外网；

方案:

1. 提供 代理域名 proxy -- 安全要求太高pass;
2. 自行研究CNPM源代码,修改器逻辑(不让检查版本)，直接从 nfs下载 -- 有点技术要求，后边会尝试下；
3. 笨方法 -- 直接上传 node_modules;

[johnsonGong]

算了，暂时不用了。 3Q, cnpmjs.org.