dcsan
commented
8 years ago Closed dcsan closed 7 years ago
dcsan
commented
8 years ago 
engelgabriel
commented
8 years ago $ METEOR_PROFILE=1 meteor
[[[[[ /Volumes/gabrielengel/Sites/Rocket.Chat ]]]]]
=> Started proxy.
| Selecting package versions -
| Select Package Versions: 15011.0
| new CS.Input: 31.4
| Input#loadFromCatalog (sqlite): 11263.8
| check for previous versions in catalog: 1.2
| new CS.Solver: 137.6
| Solver#analyze: 137.2
| analyze allowed versions: 23.2
| analyze root dependencies: 1.3
| analyze reachability: 24.3
| analyze constraints: 84.1
| analyze pre-releases: 2.9
| other Solver#analyze: 1.3
| Solver#getAnswer: 3575.7
| new Logic.Solver (MiniSat start-up): 71.5
| require root dependencies: 6.6
| generate package variables: 420.7
| generate dependency requirements: 367.4
| generate constraints: 732.7
| pre-solve: 960.1
| minimize unknown_packages: 2.8
| minimize conflicts: 388.7
| minimize unanticipated_prereleases: 12.4
| calculate previous_root distance costs: 23.4
| minimize previous_root_incompat: 111.0
| minimize previous_root_major: 3.0
| minimize previous_root_minor: 11.7
| minimize previous_root_patch: 20.3
| calculate previous_indirect distance costs: 13.8
| minimize previous_indirect_incompat: 46.2
| minimize previous_indirect_major: 5.2
| minimize previous_indirect_minor: 7.3
| minimize previous_indirect_patch: 6.3
| lock down important versions: 20.4
| calculate new_indirect version costs: 1.5
| minimize new_indirect_major: 4.7
| minimize new_indirect_patch: 7.6
| minimize new_indirect_rest: 8.8
| minimize total_packages: 307.1
| generate version map: 2.0
| other Solver#getAnswer: 9.9
| other Select Package Versions: 1.3
|
| Input#loadFromCatalog (sqlite): 11263.8
| pre-solve: 960.1
| generate constraints: 732.7
| generate package variables: 420.7
| minimize conflicts: 388.7
| generate dependency requirements: 367.4
| minimize total_packages: 307.1
| minimize previous_root_incompat: 111.0
| analyze constraints: 84.1
| new Logic.Solver (MiniSat start-up): 71.5
| minimize previous_indirect_incompat: 46.2
| new CS.Input: 31.4
| analyze reachability: 24.3
| calculate previous_root distance costs: 23.4
| analyze allowed versions: 23.2
| lock down important versions: 20.4
| minimize previous_root_patch: 20.3
| calculate previous_indirect distance costs: 13.8
| minimize unanticipated_prereleases: 12.4
| minimize previous_root_minor: 11.7
| other Solver#getAnswer: 9.9
| minimize new_indirect_rest: 8.8
| minimize new_indirect_patch: 7.6
| minimize previous_indirect_minor: 7.3
| require root dependencies: 6.6
| minimize previous_indirect_patch: 6.3
| minimize previous_indirect_major: 5.2
| minimize new_indirect_major: 4.7
| minimize previous_root_major: 3.0
| analyze pre-releases: 2.9
| minimize unknown_packages: 2.8
| generate version map: 2.0
| calculate new_indirect version costs: 1.5
| other Solver#analyze: 1.3
| analyze root dependencies: 1.3
| other Select Package Versions: 1.3
| check for previous versions in catalog: 1.2
| Total: 15008.1
=> Started MongoDB.
| Building plugin `builLivechat` in pac... -
| Select Package Versions: 27265.7
| new CS.Input: 73.8
| Input#loadFromCatalog (sqlite): 23590.3
| new CS.Solver: 89.0
| Solver#analyze: 88.3
| analyze allowed versions: 30.1
| analyze root dependencies: 1.3
| analyze reachability: 12.0
| analyze constraints: 39.8
| analyze pre-releases: 2.3
| other Solver#analyze: 2.9
| Solver#getAnswer: 3508.7
| new Logic.Solver (MiniSat start-up): 57.7
| require root dependencies: 7.0
| generate package variables: 479.3
| generate dependency requirements: 177.5
| generate constraints: 316.9
| pre-solve: 926.7
| minimize unknown_packages: 7.7
| minimize conflicts: 530.0
| minimize unanticipated_prereleases: 4.8
| calculate previous_root distance costs: 16.7
| minimize previous_root_incompat: 139.5
| minimize previous_root_major: 1.8
| minimize previous_root_minor: 9.4
| minimize previous_root_patch: 101.6
| calculate previous_indirect distance costs: 12.3
| minimize previous_indirect_incompat: 9.5
| minimize previous_indirect_major: 12.1
| minimize previous_indirect_minor: 54.0
| minimize previous_indirect_patch: 16.5
| minimize previous_indirect_rest: 1.3
| lock down important versions: 15.9
| minimize new_indirect_minor: 6.9
| minimize total_packages: 584.8
| generate version map: 5.3
| other Solver#getAnswer: 11.2
| other Select Package Versions: 3.4
|
| Input#loadFromCatalog (sqlite): 23590.3
| pre-solve: 926.7
| minimize total_packages: 584.8
| minimize conflicts: 530.0
| generate package variables: 479.3
| generate constraints: 316.9
| generate dependency requirements: 177.5
| minimize previous_root_incompat: 139.5
| minimize previous_root_patch: 101.6
| new CS.Input: 73.8
| new Logic.Solver (MiniSat start-up): 57.7
| minimize previous_indirect_minor: 54.0
| analyze constraints: 39.8
| analyze allowed versions: 30.1
| calculate previous_root distance costs: 16.7
| minimize previous_indirect_patch: 16.5
| lock down important versions: 15.9
| calculate previous_indirect distance costs: 12.3
| minimize previous_indirect_major: 12.1
| analyze reachability: 12.0
| other Solver#getAnswer: 11.2
| minimize previous_indirect_incompat: 9.5
| minimize previous_root_minor: 9.4
| minimize unknown_packages: 7.7
| require root dependencies: 7.0
| minimize new_indirect_minor: 6.9
| generate version map: 5.3
| minimize unanticipated_prereleases: 4.8
| other Select Package Versions: 3.4
| other Solver#analyze: 2.9
| analyze pre-releases: 2.3
| minimize previous_root_major: 1.8
| minimize previous_indirect_rest: 1.3
| analyze root dependencies: 1.3
| Total: 27262.1the first build is always awful, but this version is meant to improve subsequent builds (code edits) i'm running on a decent macbook pro and get
I removed livechat and some others
initial build
| Total: 18792.5
touch client .coffee file
| Total: 13843.5
touch server .coffee file
| Total: 13547.4feels a lot longer tho :)
with the sorry state of testing in meteor , edit/build is the only way to work :-1: hopefully this will also be fixed in 1.3
As I said I pulled out some other packages (eg push-notifications), but then hit errors with the site, so not sure what dependencies are for a minimal core
timkinnane
commented
8 years ago @dcsan my team has been thinking about this also. There's A LOT of non essential code running and contributions that might only be useful for a handful of people. I've been advised to disable my contributed packages by default, but I think this has been inconsistently enforced.
I have a solution in mind, let me know if it works for you and I'd be really happy for some help on it.
There might also need to be some refactoring of non-essential packages that are not able to be disabled because they've been woven into the core fabric without conditional checks. Can't think of exact examples, but I've noticed some before.
@garychapman has been working on our draft for the step 1 docs. We're looking for help to complete this...
rocketchat:cors # Cross Origin Resource Sharing
# all non-qualified packages can be found at https://atmospherejs.com/meteor/<packagename>
accounts-facebook # Login service for Facebook accounts
accounts-github # Login service for Github accounts
accounts-google # Login service for Google accounts
accounts-meteor-developer # accounts-meteor-developer
accounts-password # Password support for accounts
accounts-twitter # Login service for Twitter accounts
blaze-html-templates # Compile HTML templates into reactive UI with Meteor Blaze
check # Check whether a value matches a pattern
coffeescript # Javascript dialect with fewer braces and semicolons
ddp-rate-limiter # The DDPRateLimiter allows users to add rate limits to DDP methods and subscriptions. DDP = Distributed Data Protocol
ecmascript # ecmascript
ejson # Extended and Extensible JSON library
email # Send email messages
fastclick # Faster touch events on mobile
http # Make HTTP calls to remote servers
jquery # Manipulate the DOM using CSS selectors
less # Leaner CSS language
logging # Logging facility.
meteor-base # Packages that every Meteor app needs
mobile-experience # Packages for a great mobile user experience - A set of Cordova/PhoneGap-specific packages that set some good defaults when building for mobile
mongo # Adaptor for using MongoDB and Minimongo over DDP
random # Random number generator and utilities
rate-limit # An algorithm for rate limiting anything
reactive-dict # This package provide ReactiveDict, a general-purpose reactive datatype for use with tracker.
reactive-var # This package provide ReactiveVar, a general-purpose reactive datatype for use with tracker
reload # Reload the page while preserving application state.
service-configuration # Manage the configuration for third-party services
session # Session variable
spacebars # Handlebars-like template language for Meteor
standard-minifiers # Standard minifiers used with Meteor apps by default.
tracker # Dependency tracker to allow reactive callbacks
rocketchat:action-links # Action Links are a way to add custom javascript functions to RocketChat messages
rocketchat:api
rocketchat:assets
rocketchat:authorization # Supports role or permission based authorization, and defines the association between them.
rocketchat:autolinker # detects phone numbers, URLs etc in messages and converts them to hyperlinks
rocketchat:cas # ?? Central Authentication Service
rocketchat:channel-settings # ?? room name, topic, type etc
rocketchat:channel-settings-mail-messages # ?? maybe related to sending incoming message emails when offline
rocketchat:colors # ?? message colouring
rocketchat:custom-oauth # oauth (there's specific mention of gitlab but this could be for generic oauth)
rocketchat:emojione # emoji support
rocketchat:favico # favico
rocketchat:file # basic file handling
rocketchat:file-upload # file uploading support - incl. S3, progress reporting etc
#rocketchat:github-enterprise # github enterprise Oauth
rocketchat:gitlab # An implementation of the GitLab OAuth2 flow? DO NOT REMOVE or RocketChat.Info becomes undefined
rocketchat:highlight-words # highlighing of individual words
rocketchat:iframe-login # UI for facebook/twitter/google/oauth login
#rocketchat:importer # basic import functionality
#rocketchat:importer-hipchat # hipchat importer
#rocketchat:importer-slack # slack importer
rocketchat:integrations # ?? presumably this is base code for integrations with other packages
rocketchat:katex # KaTeX is a fast, easy-to-use JavaScript library for TeX math rendering on the web
rocketchat:ldap # Lightweight Directory Access Protocol
rocketchat:lib # This package contains the main libraries of Rocket.Chat.
rocketchat:livechat
rocketchat:logger # basic logging
rocketchat:mailer # send subscription/unsubscribe email messages
rocketchat:markdown # self-explanatory?
rocketchat:mentions # self-explanatory?
rocketchat:mentions-flextab # show user mentions in other channels
rocketchat:message-attachments # self-explanatory?
rocketchat:message-pin # self-explanatory?
rocketchat:message-star # self-explanatory?
rocketchat:migrations # ?? migrates a RC database
rocketchat:oauth2-server-config
rocketchat:oembed # embedded media support
rocketchat:otr # Off The Record (encrypted) message support
rocketchat:piwik # Rocket.Chat supports adding Piwik url and site id to track the analytics of your server.
rocketchat:push-notifications
rocketchat:reactions # self-explanatory?
rocketchat:slackbridge
rocketchat:slashcommands-asciiarts # self-explanatory - IRC-style /join /kick etc
rocketchat:slashcommands-invite # self-explanatory?
rocketchat:slashcommands-join # self-explanatory?
rocketchat:slashcommands-kick # self-explanatory?
rocketchat:slashcommands-leave # self-explanatory?
rocketchat:slashcommands-me # self-explanatory?
rocketchat:slashcommands-mute # self-explanatory?
rocketchat:slashcommands-topic # self-explanatory?
#rocketchat:spotify # spotify oauth?
rocketchat:statistics # needed at startup
rocketchat:streamer # DB-less 2 way communication over DDP with better performance.
rocketchat:theme # change colours!
rocketchat:tooltip #
rocketchat:tutum # docker-style container platform?
rocketchat:ui # self-explanatory?
rocketchat:ui-account # self-explanatory?
rocketchat:ui-admin # self-explanatory?
rocketchat:ui-flextab # self-explanatory?
rocketchat:ui-login # self-explanatory?
rocketchat:ui-master # self-explanatory?
rocketchat:ui-message # self-explanatory?
rocketchat:ui-sidenav # self-explanatory?
rocketchat:version # extracts version info for the Administration/Info panel
rocketchat:webrtc # Web Real-Time Communication support - could be optional as it can be turned on/off via Admin panel
#rocketchat:wordpress # wordpress oauth?
rocketchat:videobridge
#rocketchat:chatops
#rocketchat:internal-hubot # Meteor doesn't interact really well with NPM and the NPM module loading mechanism which hubot uses for its scripts. So we've split out most of hubot's scripts into this separate module.
#rocketchat:irc # https://github.com/haosdent/Rocket.Chat/tree/irc/packages/rocketchat-irc
konecty:change-case # convert strings between camelCase, etc https://atmospherejs.com/konecty/change-case
konecty:delayed-task # buffer/delay tasks https://atmospherejs.com/konecty/delayed-task
konecty:mongo-counter # atomic counters for mongo https://atmospherejs.com/konecty/mongo-counter
konecty:multiple-instances-status # Keep a collection with active servers/instances https://atmospherejs.com/konecty/multiple-instances-status
konecty:nrr # Non-Reactive Render https://atmospherejs.com/konecty/nrr
konecty:user-presence # track user status https://atmospherejs.com/konecty/user-presence
arunoda:streams # says not compatible with meteor > 0.9 https://atmospherejs.com/arunoda/streams
chrismbeckett:toastr # non-blocking notifications (slide-in from the side a-la the bot responses) https://atmospherejs.com/chrismbeckett/toastr
dispatch:run-as-user # Allow server and client to run as user in restricted mode https://atmospherejs.com/dispatch/run-as-user
francocatena:status # displays server↔ browser connection status (e.g. that 'waiting for server connection' bar) https://atmospherejs.com/francocatena/status
jalik:ufs # Upload FileSystem https://atmospherejs.com/jalik/ufs
jalik:ufs-gridfs # Use Mongo GridFS with UploadFS https://atmospherejs.com/jalik/ufs-gridfs
jparker:gravatar # use Gravatar images https://atmospherejs.com/jparker/gravatar
kadira:blaze-layout # layout manager for Blaze https://atmospherejs.com/kadira/blaze-layout
kadira:flow-router # client-side routing https://atmospherejs.com/kadira/flow-router
kenton:accounts-sandstorm # login service for sandstorm.io apps https://atmospherejs.com/kenton/accounts-sandstorm
mizzao:autocomplete # autocompletion (presumably used when looking up users to DM, for example) https://atmospherejs.com/mizzao/autocomplete
mizzao:timesync # NTP-style time synchronization between server and client, and facilities to use server time reactively in Meteor applications
momentjs:moment # date wrangling https://atmospherejs.com/momentjs/moment
monbro:mongodb-mapreduce-aggregation # https://atmospherejs.com/monbro/mongodb-mapreduce-aggregation
mrt:reactive-store # https://atmospherejs.com/mrt/reactive-store
mystor:device-detection # client-side device detection https://atmospherejs.com/mystor/device-detection
nimble:restivus # https://atmospherejs.com/nimble/restivus
nooitaf:colors # add colours to strings https://atmospherejs.com/nooitaf/colors
ostrio:cookies # cookie wrangling https://atmospherejs.com/ostrio/cookies
#pauli:accounts-linkedin # linked login package https://atmospherejs.com/pauli/accounts-linkedin
perak:codemirror # JS text browser-based editor https://github.com/perak/codemirror
percolate:synced-cron # schedule jobs across multiple servers https://atmospherejs.com/percolate/synced-cron
raix:handlebar-helpers # self-explanatory https://atmospherejs.com/raix/handlebar-helpers
raix:push # mobile push notifications https://atmospherejs.com/raix/push
raix:ui-dropped-event # ?? https://atmospherejs.com/raix/ui-dropped-event
smoral:sweetalert # popup (modal) alerts https://atmospherejs.com/smoral/sweetalert
steffo:meteor-accounts-saml # SAML v2 login support for existing password based accounts https://atmospherejs.com/steffo/meteor-accounts-saml
tmeasday:crypto-md5 # MD-5 hashing, possibly superseded by https://atmospherejs.com/jparker/crypto-md5
todda00:friendly-slugs # generate URL-friendly slugs https://atmospherejs.com/todda00/friendly-slugs
underscorestring:underscore.string # string-manipulation package https://atmospherejs.com/underscorestring/underscore.string
yasaricli:slugify # basic slugification https://atmospherejs.com/yasaricli/slugify
yasinuslu:blaze-meta # Blaze-meta makes it super simple to manage SEO data https://atmospherejs.com/yasinuslu/blaze-metaHave you guy tried the develop branch? We are now using Meteor 1.4.1.1 an it seems considerably faster.
dcsan
commented
8 years ago yeah i basically gave up on meteor 6 months ago because of rebuild times. it's nice for hackathons but as soon as you really have an app with a bunch of code you're in deep shit.
engelgabriel
commented
8 years ago Related to https://github.com/meteor/meteor/issues/4284
engelgabriel
commented
8 years ago @dcsan have you guys still working with Rocket.Chat? Can we have a talk to get your feedback on your reasons either way?
engelgabriel
commented
8 years ago @timkinnane I like the idea about the commented file, this would be a great start. Ideally, ONLY the rocketchat:lib should be required and it should have all its dependencies, and all the other packages should be optional. I thought about renaming rocketchat:lib to rocketchat:core
dcsan
commented
8 years ago @engelgabriel happy to have a chat. we're not using Rocket.Chat. although it's fine as an end-user it's too painful and tedious to work on as a developer. waiting for meteor to reload all the time is just a complete killer for productivity and developer happiness, so a project of this scope isn't really a fit for meteor. we built an internal system that's similar just using plain node/meteor/react/redux/webpack etc. Rebuilding everything entirely from scratch was more appealing than fighting on down a dead-end with meteor, at some point you just have to cut your losses.
timkinnane
commented
8 years ago @dcsan I started a package here, testing removing some (imo) extraneous packages. I took some fairly big ones out, like Livechat, Sandstorm. The result is a much faster and less complicated build, but I had to also chase down some errors where packages were being referenced in core without checking if the package is enabled.
That part was pretty simple, e.g. SandstormEnabled = (typeof RocketChat.Sandstorm !== "undefined") then wrap the relevant calls in if SandstormEnabled condition.
Still, it would be nice to have an easy consistent method in rocketchat lib instead, like ' if RocketChat.packageEnabled('Sandstorm')
NB: I started from v0.38 for this branch, because that's where my production apps are. I'd be happy for any help testing and merging with current develop before I submit the PR.
timkinnane
commented
8 years ago Ideally another part of that PR would be the docs to detail what each package is/does. I don't know what half of them do, but it would be an important reference for people who are impacted by us turning off features they may rely on.
asthomasdk
commented
8 years ago Were you able to get Rocket to remove the packages so that they do not show up in the UI?
I have been trying to remove the OAuth settings we don't want to have available - and even though I had removed the packages, they still show up in the UI - even in a new database.
timkinnane
commented
8 years ago I've found that certain settings will exist in the app long after being removed, even after directly deleting them through mongo. Try restarting the server from Administration menu > General > Restart. That does some kind of magic cleanup that doesn't seem to occur any other way.
asthomasdk
commented
8 years ago Afraid that didn't work.
engelgabriel
commented
8 years ago @asthomasdk there is an environment variable to LOCK and HIDE admin settings that you don't want to expose, even to the Rocket.Chat admin. Please as @sampaiodiego for this.
You can chat with him at https://demo.rocket.chat/direct/diego.sampaio
asthomasdk
commented
8 years ago thanks @engelgabriel
I have been trying to use these settings and have also been looking through the code to try and understand how this is meant to work. I will get in touch with @sampaiodiego and discuss in more detail.
timkinnane
commented
8 years ago Hey @asthomasdk, @engelgabriel made this new branch for testing and running minimal package builds: https://github.com/RocketChat/Rocket.Chat/tree/minimal-packages - if you discover any tricks it would be great if you could do a PR to that branch.
asthomasdk
commented
8 years ago Sure thing. I will try and find some time soon again to look at this.
engelgabriel
commented
7 years ago I think this is not really an issue since meteor 1.4, as it compiles ONLY the changes packages and the rebuild is much faster. Can we close this?
timkinnane
commented
7 years ago Happy to close for those reasons, but I'm still disabling many of the packages we don't need just for simplicity - which may reduce initial build time, but also avoids exposing some of the less polished components in the UI and admin pages (probably also side-steps some minor bugs).
@engelgabriel, I remember you mentioned changing rocketchat-lib to rocketchat-core and possibly moving some code from packages into that so its more obvious which code is absolutely core vs optional add-ons. Is there another issue to follow that approach or should we re-title this issue?
Also, I just noticed your comment re the "environment variable to LOCK and HIDE admin settings" - is that documented? I appreciate anything that can be done to streamline the basic experience.
timkinnane
commented
7 years ago Found another one. rocketchat-ui-master calls a package method of rocketchat-iframe-login which may be disabled. Have submitted PR to resolve with package enabled conditional. PR #5634
engelgabriel
commented
7 years ago @timkinnane let's create a new issue for it: https://github.com/RocketChat/Rocket.Chat/issues/5679
Making any code edit on this project gives a 20~30 second delay for rebuilding. this is pretty damn frustrating to work on.
does anyone have tips to get faster rebuilds?
I tried to remove some packages, but hit various errors, so wondering if there is a core set that are absolutely required, it would be nice to separate out the .meteor/packages file a bit.