Website requirements

[chrisdickinson]

Replacing #27 in order to pin this. I want give more context on the pending software design decisions we need to make in order to build out a website (this affects #102 and #103, which are much appreciated!)

the problem

Entropic needs a website, because:

1. In order to allow people to sign up for accounts, we use GitHub OAuth login for signup and token creation
2. In order to allow people to manage their account, we display a token management UI.
3. In order to aid discovery & vetting of packages, we want to display information about packages (dependencies, dependents, etc.)
4. In order to aid vetting of packages, we want to display information about package contents (styled, rendered package contents as html)

The current website doesn't have any styles, is likely inaccessible, and is hard for developers to extend. Importantly, it's also embedded into the registry service, which could present security & caching concerns down the line.

constraints, rakes in the yard, & open questions

splitting into services

At present, Entropic runs in a single process. Because certain tasks (like syncing external packages) are too time-consuming to run in the duration of a request/response cycle, we need to split this out into multiple services. Eventually, I believe we'll need 4 services:

1. The existing external registry API service we expose today, talking to...
2. A :sparkles:new:sparkles: internal api encapsulating our data stores that the website, worker, and registry service can leverage.
3. A :sparkles:new:sparkles: external website service talking to that same internal api
4. A :sparkles:new:sparkles: worker service for processing syncing requests, talking to the internal api

For the production entropic instance, @ceejbot took a clever approach with nginx proxy_pass's so that we can simulate privileged APIs for the website to use by mounting them on the registry at non-exposed routes (for example, /private/.) This lets us start working on the website without having completed the other two :sparkles:new:sparkles: apis. However, until we split out the website process from the registry process, web handlers will use HTTP clients to talk to internal handlers in the same process. We'll start this process for the registry as well.

This is quite a mouthful: this is all to say that when the website starts needing to get at registry data, please pull in @chrisdickinson and/or @ceejbot.

Single page app or Server Side Template Language?

We'll want to consider whether we use a server-side templating language like Nunjucks to render pages in a traditional fashion, or go a single page app route using something like React + Next.js. Right now I am leaning towards server side templating using Nunjucks (or similar):

• Our present Oauth & CLI signup needs lean on url routing. (They could, yanno, not. But.)
• Our form needs are pretty basic.
• SPA's introduce a certain level of maintenance complexity (bundling FE assets, etc.) Next solves this pretty nicely, which is one of the (many) things I like about it, but it's still a cost.

I could be wrong here, so please poke holes in this. What are we buying with a single page app?

CSS

Let us do the simplest thing that works, then iterate. I propose we use Tachyons for a base coat, then specialize with an added stylesheet. (When this gets unwieldy, that's when we should start looking into css-modules or css-in-js as an additional layer on top of tachyons.)

[superhawk610]

In my experience, any project that requires more than the most simple frontend will see large DX benefits when using an SPA over simpler template rendering. If the frontend were only required for signup/login/auth, I think an argument could be made in favor of a templating language. However, since it also aims to create package search/detail/statistics interfaces, I believe the SPA approach becomes more appealing.

I foresee the following elements requiring a non-negligible amount of client-side scripting:

• search autocompletion
• package description markdown rendering
• interactive UI (tabs, form autocomplete/validation, routing, etc)

If those features aren't necessary and can be avoided, then an SPA may be overkill. However, as soon as you want to work on one of them, you'll have to write client-side JS, which will lead to some sort of client-side dependency bundling/processing to optimize in production, so I believe that maintenance complexity will naturally arise one way or another - may as well take advantage of the DX boosts associated with an SPA:

• better separation of concerns (you can't tightly couple your server logic and frontend when they're split into two separate projects)
• hot code reloading for faster design iteration
• better end-user experience (only a single initial page load, no page flashes when changing routes)
• easy access to more complex build tools (static typing, CSS-in-JS, minification, asset optimization, etc)
• the web client can make use of the same REST routes that the CLI client uses, so more streamlined experience between the two
• assets can be cached client-side and don't need to be regenerated on every request like with a templating engine

The biggest downside of an SPA is definitely the added complexity, but I personally believe that cost is worthwhile and will lead to a better product long-term. I'm interested to see what others have to say - I'll press the pause button on #102 until this discussion progresses further.

[tunnckoCore]

@superhawk610 absolutely agree. SPAs may sound complex, but for me, they aren't and definitely worth all the features, pluses, minuses, and optimizations that they give - not only DX is bumped to the skies but also the UX.

Just to throw it here: Bet all-in on Gatsby, except maybe for the "internal" one which makes sense to be Next.js

[jakeburden]

Don't know if you'd want to lean on or trust Google to keep the service up, but firebase offers a nice way to add Github OAuth to your site without needing to create url routes.

This is my typical setup for it:

// helpers/firebase.js
import * as firebase from 'firebase/app'

import 'firebase/auth'
import 'firebase/firestore'

import config from '../config/firebase'

const provider = new firebase.auth.GoogleAuthProvider()

const app = !firebase.apps.length
  ? firebase.initializeApp(config)
  : firebase.app()

const db = firebase.firestore()

export { app, provider, db }

// pages/index.js

import { app } from '../helpers/firebase'
import { useAuthState } from 'react-firebase-hooks/auth'

const Page = () => {
  const [user, initialising, error] = useAuthState(app.auth())

  return (
    <Layout>
      <NavWithGithubAuthButton user={user} />
      {user ? <p>Signed In!</p> : <p>Not!</p>}
    </Layout>
  )
}

export default Page

[superhawk610]

Firebase auth is probably overkill since entropic isn't using any other firebase features like firestore (everything's already set up to use self-hosted Redis/Postgres instances). The server-side logic for handling registration/login is already there, we just need a way to:

• receive and store an OAuth token client-side (replacing /www/login/providers/:provider/callback with a client-side route), or
• generate a corresponding entropic frontend token for the client to store that doesn't reveal the GitHub (or whatever provider) actual OAuth token and use that to authenticate any elevated request from the frontend, or
• make use of the existing session middleware and keep the OAuth implementation completely server-side (using cookies instead of Authentication: Bearer XXXX headers for elevated requests)

[Wizyma]

Hi @chrisdickinson i would love to work on this part and contribute to the project ! And i most definetly agree with @superhawk610 !

[trodrigues]

I was kind of anticipating this whole discussion about what technologies to use and not knowing what was necessary right now that’s why my PR (#103) was really just about the most simple HTML/CSS site possible right now so that at least some information about the project could exist in a website.

Being a frontend developer myself and having worked with all the popular frameworks, in large and small apps, client side only SPAs and server rendered SPAs and having managed all sorts of complexity, I’ve also went with a super simple approach for my own personal website (using eleventy+nunjucks) because I really didn’t need more, and I think a similar approach could work here, similar to what Chris described:

• Start small, build just some static pages
• Using a templating system like nunjucks (which is mostly some templating to display data and some conditionals and loops) doesn’t lock you in too much
• First bits of javascript based on the needs described above are probably small enough that they don’t even need to be bundled
• If you do need to start bundling, you can do so with the most basic webpack/parcel build, just to put some files together
• If there’s a need to go the SPA (or SSR SPA) route after that there shouldn’t be too much lock in and it should be straightforward to migrate

TL;DR: start with something simple, add tooling as necessary.

[superhawk610]

First bits of javascript based on the needs described above are probably small enough that they don’t even need to be bundled

I don't think that's true since we'll need a client-side markdown rendering library (and whatever dependencies it brings with it). Granted, we could render/cache project READMEs server-side, but then that introduces a non-negligible overhead at scale and I feel like one of the benefits of this project is that it's usable by anyone, no need to drive up the hosting costs. Same for server-side rendering, at scale it will require higher container compute time and will more quickly require higher hosting tiers, while an SPA can be thrown behind a Cloudflare proxy and forgotten about.

If there’s a need to go the SPA (or SSR SPA) route after that there shouldn’t be too much lock in and it should be straightforward to migrate

I think this project is straightforward enough to know now whether or not this will ever be required; the project's scope is pretty well defined, and there are existing alternatives like NPM to use as a case study (NPM does use an SPA). While I appreciate the desire to start small and work your way up, why not go ahead and plan for the future now? If we go with templated SSR now and then 6 months or a year later want to switch, all of the template code will have been wasted since it won't be able to translate to SPA land.

Additionally, SPAs will allow for a more useful PR diffing experience since changes to the frontend UI will never touch the backend files, and vice versa. It will be clear from filename alone whether changes were made to the visual presentation or to the business logic, and contributors that only want to participate in one won't have to slog through the other.

From the points already provided I think it's fair to say that:

• SSR templating => easier maintainability, worse user experience
• SPA => tougher maintainability, better user experience

In my mind, better UX should always win out.

[jefflembeck]

NPM does use an SPA

npm uses a SPA but it’s barely necessary and was more of a way for us to dog food common ways for our users to build a web application. (Also the entire site is SSR + rehydrated front end)

[zacanger]

FWIW (coming from a recovering FED), SSR with only minimal client-side JS would make things a lot easier for folks who like to browse with JS off or use alternative browsers (including text-mode browsers). And you could get a lot of the DX benefits with SSR by using something like Next.

[wardpeet]

I'm biased in favor of gatsby but gatsby will be great for this. You don't have to run a server because everything is static hosted so it's going to be pretty cheap. Api calls can be cloud functions as they can scale pretty heavily.

For the packages pages itself it's probably to best to go for full client side for now as gatsby doesn't have incremental build support yet which means we have to rebuild all pages.

I don't mind guiding/writing a bunch of the website in gatsby if that's the direction you want to go.

[iainjreid]

From the points already provided I think it's fair to say that:

• SSR templating => easier maintainability, worse user experience • SPA => tougher maintainability, better user experience

Just my two-penny worth, but I would also lean towards a more user-centric approach. This project will likely appeal to a wide range of frontend developers and designers alike, and the appearance and feel of tools can impact heavily on how or even if they are adopted.

[venikx]

I'd like to raise a few questions before jumping into the SPA vs SSR debate:

• How do SPA's deal with search engine indexing? Is it a goal to optimize it to "compete" with npm?
• How do SPA's deal with minimal browsers?
• What is defined "good UX"? What is considered the main use-case of the registry? It's important to ask, because depending on the answer SSR or SPA can be the better user experience (imho). I don't mind that an mail application client takes a while to load in the browser, because I might "live" in it for a while, once it's loaded. I do mind slow loading speeds when browsing content (let's say Medium). So it's important to understand where entropic would fall onto this spectrum.
• Is DX valued more or less, compared to UX? And how bad/good are both sides?
• What is more valuable when thinking about server-side (cost) vs client-side time (perf)?

[nilsnh]

You raise good questions @venikx.

I'd like to raise a few questions before jumping into the SPA vs SSR debate:

* How do SPA's deal with search engine indexing? Is it a goal to optimize it to "compete" with npm?

According to Google they seem to be able to index SPAs decently since 2015. But Facebook doesn't parse JS (Could that be the same case for Twitter?).

* How do SPA's deal with minimal browsers?

SPA won't work in text mode browsers (I think?). If we're transpiling javascript we'll be able to support old browsers.

* What is defined "good UX"? What is considered the main use-case of the registry?

One way to define good user experience is that the site is as accessible as possible to the diverse Javascript ecosystem. Like, it should definitely be accessible by screen readers. It should be easy for newcomers to learn what it is and isn't. Likewise, it should be an effective site for more experienced users.

  It's important to ask, because depending on the answer SSR or SPA can be the better user experience (imho). I don't mind that an mail application client takes a while to load in the browser, because I might "live" in it for a while, once it's loaded. I do mind slow loading speeds when browsing content (let's say Medium). So it's important to understand where entropic would fall onto this spectrum.

* Is DX valued more or less, compared to UX? And how bad/good are both sides?

* What is more valuable when thinking about server-side (cost) vs client-side time (perf)?

All of this is just my five cents. :smile_cat: Whatever we build should be maintainable for the future and as accessible as possible for screen readers, for newcomers and for experienced users. And we also need to consider localization. :thought_balloon:

[alexgarces]

I agree with @trodrigues' approach. I think using SPA at this point is overkill. SPA is not a good solution for all use cases.

If we barely know the features we will need, why should the project be locked to SPA? This is anticipating the future and we are in a very early state.

SPA's are a great solution for high interactive web apps (like dashboards, for example). But SPA's are terrible for some very basic web features. For example, forget about decent indexing and SEO on SPA, compared to plain HTML or SSR.

Also, as @nilsnh suggests, do we want the website to be accesible for everyone or only for last-generation browsers? SPA's rely totally on JS which can easily end up with performance issues (so more complexity to deal with).

@chrisdickinson +1 to Tachyons!

[venikx]

@alexgarces I agree. I'm a fan of SPA's, but that's because I mostly work on dashboards and I think that's an amazing case for SPA's, since there is a lot of user interaction.

If we really don't know the amount of features yet, I'd also opt for @trodrigues' approach. Start small, introduce more complexity when the benefits outweighs the trade-offs.

[superhawk610]

If we barely know the features we will need...we are in a very early state.

Yes and no - the project is young, but it is aiming to directly replace an existing product with years' of trial and error backing it. Entropic isn't aiming to reinvent the wheel, just to take an existing wheel (NPM) and change the way it is hosted and managed on a high level (at least, that's my understanding, please correct me if I'm wrong). As I listed earlier, I believe the featureset is already fairly well-defined:

• client-side markdown renderer
• client side form interactions (regex validation, checking for duplicate username before submissions, etc)
• client side UI interactions (tabbed panes, route transitions, loading states, etc)
• client-side asset caching

forget about decent indexing and SEO on SPA, compared to plain HTML or SSR.

SPA and SSR are not mutually exclusive. The only functionality exclusively available to an SSR is page-specific <meta> tags, and this can still be very easily accomplished by spinning up a very minimal SSR renderer (either using an existing solution like Gatsby/Next or even a simple Express server) to inject the relevant SEO info into the index.html before serving - this approach has the advantage of minimizing server-side work while still allowing for the richer UX of an SPA.

EDIT: I've added a PoC to my PR branch, you can check it out here.

Also, as @nilsnh suggests, do we want the website to be accesible for everyone or only for last-generation browsers? SPA's rely totally on JS which can easily end up with performance issues (so more complexity to deal with).

While they do rely on JS, SPAs also make quickly implementing and iterating on things like i18n and a11y much easier.

If we really don't know the amount of features yet

I've seen a few comments mention this line of logic, and I'm curious to hear more about your thought process - what aspects do you think are still left to decide on?

[chrisdickinson]

The feature set is not defined (that's on me, sorry!) This is not an exhaustive list & I will do a more complete writeup ~elsewhere~ as soon as I get some time!

• The hierarchy of values for the website are: we want the server to be easy to set up and maintain, easy to use, and easy to develop, in that order.
  • By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors.
• Entropic instances may live inside your network, so network latency concerns are a bit different than the average website.
• Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)
• We don't have search. Federated search is going to be a different beast than centralized search. (Think Napster/Kazaa/Limewire/Bittorrent, not Google.) It might not be a thing that individual Entropic instances support!
• We want i18n support. My experience with rich client apps has been that you end up having to generate multiple different asset bundles for each target locale (or else do some kind of code-splitting.)
• We want to have good a11y.
• The server component of the website will have privileged access to an internal backing storage API that frontend code will not.
• We currently only supply the following functionality:
  • receive CLI login session
  • Redirect to GitHub OAuth
  • Receive GitHub OAuth response
  • Sign up for new user
  • Display token list
  • Create new token
  • Homepage (currently served by nginx as flat html)
• We wish to supply, roughly:
  • A paginated list of all packages on the instance
  • Package detail pages with:
    • A readme
    • links to dependencies,
    • links to (local) dependent packages.
    • No download counts (Content addressable store precludes this.)
    • No external dependent packages (we don't know who depends on this package elsewhere!)
  • List of namespaces of which you are a member.
  • List of outstanding namespace membership invites for your user.
  • For each namespace of which you are a member, a settings page:
    • List of extant maintainership invites.
    • (Long term) Ability to invite a member to a namespace.
    • List of current members.
    • List of packages maintained by this namespace.
      • (Long term) ability to invite a maintainer to a package.
      • (Long term) ability to remove a maintainer from a package
      • (Long term) ability to "yank" a package or package version
    • List of packages originally published by this namespace.
      • (Long term) ability to remove a maintainer from a package
  • (Long term) Create a new namespace

[superhawk610]

Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)

Client side rendering is cheaper than both ;)

[wardpeet]

I don't think it needs to be a debate of ssr vs csr. Rather think about maintenance and community. When I said gatsby, I still think it's a good approach as it can handle CSR and prebuild ssr. We also focus on a11y and speed. The downside is that people need to "learn" react & graphql.

you'll have to think about:

The hierarchy of values for the website are: we want the server to be easy to set up and maintain, easy to use, and easy to develop, in that order.

By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors. Entropic instances may live inside your network, so network latency concerns are a bit different than the average website.

[nilsnh]

Thank you for enumerating the requirements @chrisdickinson. :)

The downside is that people need to "learn" react & graphql.

I agree on your main points @wardpeet. :) I just want to note that even React and GraphQL will provide some barriers to entry, especially if we add their conventions of usage on top of that. React or GraphQL could definitely be the right tools here, but people will have to learn them, not "learn".

So, should we try to arrive at some common consensus, or create some minimal Proof of Concept stacks and argue for them? 🤷‍♀

[trygve-lie]

While they do rely on JS, SPAs also make quickly implementing and iterating on things like i18n and a11y much easier.

Thats not a given. That all boils down to how a project is structured and I sure have encountered SPAs which is close to impossible to develop on due to its complexity and poor structure.

SSR templating => easier maintainability, worse user experience SPA => tougher maintainability, better user experience

These are not given facts either. There are tons of SPAs out there with horrible UX.

I would start out with a very basic SSR template based site and enrich with client side scripting for enhanced features and go from there. I would also add a service worker to cache all html which are fairly static and assets to reudce amount of hits to the server for these.

On a bit longer term I would honestly like to see usage of web components for the different features. It would be very nice if one where able to make ex search a reusable web component one used on the web site here but which also others could use to plug into their Vue, React, Svelte, etc app. From a project like this I would very much like to see reusable components which can be plugged into different frontend frameworks rather than going all in on Vue or React component.

[lacaulac]

I'm quite a fan of SPA so I'd naturally lean towards making one, but I really see two issues with what @chrisdickinson described if we were to use one: The maintainability for people setting up their own instance and the "private" API part that only the server code should be allowed to access, even though I don't think the latter should be too much of a problem. But again, the big pro of SPA is minimising the amount of work the server has to do by almost limiting it to API calls with the consequence of potentially breaking non-Javascript browsers. I think that starting to plan how we want the website to work now is a good approach, since being offered the choice of either tough maintenance or almost completely rewriting the website isn't that great.

[alexgarces]

@lacaulac I'm also a big fan of SPA's! The developer experience when working with React and similar technologies is awesome. But being part myself of a failed SPA project some months ago made me much more critical about the SPA hype. Since then, I try to be more practical and minimalistic. And the only use cases I find for SPA's are dashboards and high interactive apps.

@chrisdickinson thanks for listing the requirements!

By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors.

This is a key reason to avoid using a big framework for now.

Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)

I'm going to contradict myself but, after reading this requirement, I would also consider Next.js. I've worked with Next.js and it is great for this particular use case. You can switch between SSR and CSR, and export it as a static site. The downside about using it, again: you are locked to the complexity of React from the beginning.

@superhawk610

SPA and SSR are not mutually exclusive. The only functionality exclusively available to an SSR is page-specific tags, and this can still be very easily accomplished by spinning up a very minimal SSR renderer (either using an existing solution like Gatsby/Next or even a simple Express server) to inject the relevant SEO info into the index.html before serving - this approach has the advantage of minimizing server-side work while still allowing for the richer UX of an SPA.

You are totally right, SPA and SSR are not mutually exlusive. But why adding the complexity of SPA before you need it?

SEO is not only about meta tags. Content is the most important thing for SEO. And client side rendered content via JS is very very far from having a decent indexing.

@trygve-lie

These are not given facts either. There are tons of SPAs out there with horrible UX. I would start out with a very basic SSR template based site and enrich with client side scripting for enhanced features and go from there. I would also add a service worker to cache all html which are fairly static and assets to reudce amount of hits to the server for these.

Totally agree with this approach. It is so easy to quickly end up with a bloated front-end when working with SPAs. Also, it is much more easier to go from simple SSR templates to something more complex than viceversa.

[superhawk610]

SEO is not only about meta tags. Content is the most important thing for SEO.

Huh, TIL. If this is the case then you're right, client-side markdown rendering probably isn't an option.

I do think there's a stronger argument to be made for a more barebones server-rendered approach, but I will say that I don't think going from templated SSR -> SPA is any easier than the alternative. I think it's a difficult transition both ways since any client-side stuff you've written in a non-React/Vue/Angular environment isn't really transferable to an environment with a framework and vice versa.

I've seen multiple people mention that adding a frontend framework like React will increase complexity, but I would argue it doesn't raise the bar for entry, it just requires a different skillset. Client-side UI interactions without a frontend framework like React introduce their own set of problems that are inherently solved when using a UI framework.

I guess I'm moving my vote from a dedicated SPA to a server-rendered SPA, ala Next or Gatsby. Best of both worlds :smile:

[alexgarces]

@superhawk610

I think it's a difficult transition both ways since any client-side stuff you've written in a non-React/Vue/Angular environment isn't really transferable to an environment with a framework and vice versa.

Yes, you are right! It wouldn't be an easy task at all either. I was imagining this transition in a relatively early state (and in case of needing it).

I guess I'm moving my vote from a dedicated SPA to a server-rendered SPA, ala Next or Gatsby. Best of both worlds

Looks to me as a very feasible option as well! :)

[lacaulac]

@superhawk610 I don't know a lot about Server rendered SPAs but that seems to be a good solution.

[jlengstorf]

Hi! Very definitely biased toward Gatsby here, but I wanted to put a few benefits on the table that I haven't seen called out:

1. The UX of Gatsby sites is set up to be stellar out of the box. New sites start with perfect scores on Lighthouse, etc., plus we do things like prefetching routes on JS-enabled pages (we skip this when data-saver is on or a slow connection is detected). This should cut down on yak shaving for deployment/QA since most of the current perf best practices are baked in, and new improvements will be handled upstream.
2. There's very little boilerplate to get started, which should help with getting contributors into working on the site vs. figuring out how to set up a dev environment.
3. If you choose MDX, the content editing experience doesn't need to require React or GraphQL knowledge for a significant number of common workflows, which should significantly decrease the barrier to entry for e.g. docs contributions.

If you're interested in exploring how Gatsby could work, I'm happy to help out!

[maxnordlund]

premature optimization is the root of all evil Donald Knuth

Starting simple and grow slowly in complexity is the way to go for maintainability. The more frameworks/libraries you add the higher the bar of entry, as in order to contribute you need to learn more and more things.

I've done my fair share of frontend and backend, but it has been mostly backend the last couple of years. So I don't feel comfortable in React (or Vue), and if this project would use that I wouldn't want to contribute. On the other hand, I've written a lot of Postgres stuff over the years, which may come useful.

So, my two cents is to keep it as simple as possible, use the least amount of dependencies and for those that are needed chose the ones that are the easiest to learn. Nunjucks is decently simple to pick up, as long as you stay away from macros/call (and custom filter/functions).

[maxnordlund]

Someone mentioned the need for regexp validation. You can actually do that in pure HTML nonwdays

Only allow lower case alpanumerical characters:
<input type="text" pattern="[a-z0-9]+" />

You can get a lot done with a sprinkle of JS:

<div>
<button type="button" class="copy">Copy to clipboard</button>
<input type="text" value="This will be compied" />
</div>

Array.prototype.forEach.call(document.querySelectorAll("button.copy"), function setupClickHandler(button) {
  button.addEventListener("click", copyToClipboard)
})

function copyToClipboard(event) {
  event.preventDefault()

  const input = event.target.nextElementSibling
  input.focus()
  input.setSelectionRange(0, input.value.length)
  document.execCommand("copy")
}

[superhawk610]

@jlengstorf from a bit of tinkering around, my initial concern is disk space. If I understand it correctly, you'll essentially end up with 3 copies of the entire dataset on disk at any given time:

• database set used by other entropic services required when generating routes at build time
• dataset spread between HTML files in prerendered build output
• dataset for data fetching in static directory in build output

We could store the actual package tarballs in a single location, but whatever package meta info (name, version, HTML rendering of description) would always exist in triplicate, right?

[nschonni]

/cc @marcysutton since some keywords that might interest you came up 😄 (a11y, gatsbyjs)

[varjmes]

I haven't seen Next.js (https://github.com/zeit/next.js/) mentioned yet. I've spent the last 13 months working with it in anger to power Vogue Fashion Shows for multiple countries. Looking at @chrisdickinson' feature set, it can do all these things. I've worked personally on SEO, Internationalisation, accessibility and analytics with it. It has been painful at times, but I have a much better understanding of how it works now.

Saying that, I am very much on team "do a bit of vanilla css with an HTML and see how it goes" with @trodrigues.

Really really keen to help out here.

[huijing]

Would love to help out with the website as well. I am too on team "basic HTML and CSS" like @varjmes and @trodrigues, but wouldn't mind having a Gatsby implementation either (@jlengstorf u pretty much sold me on it 😛). Either way, I would never pass up a chance to write some CSS no matter what form it takes ¯\_(ツ)_/¯

[berraknil]

I'm team vanilla with @varjmes and @trodrigues , but whatever you choose to go with @chrisdickinson , feel free to tag me for some frontend issues (I'm one of those people who loves to work with CSS). Looking forward to hopefully contributing to this great thing!

[wardpeet]

If you go the vanilla route you probably need some kind of template engine which will be a learning curve as well. You probably would look for something like https://www.11ty.io/ or go the SSR way with express & ejs but I wouldn't recommend it at all.

The benefit of the approach is that everyone can start without learning a framework and such but you'll have to create lots guidelines and rules to keep code consistent so I'm all for going into a framework route. Can be gatsby, nextjs, nuxtjs, angular, polymer, ...

[lacaulac]

I think @wardpeet is right about coding guidelines, having to create them looks like a pain since you got to be going for something that isn't hard to comprehend, since having the project's "development community" understanding it alone contradicts the idea of "Any user should be able to try and fix it". The advantage of using a framework is that there's generally a solid documentation already and also the people using Entropic might already know those frameworks; Therefore, making the thing from scratch wouldn't reduce the learning curve of people getting to know the project imho.

[maxnordlund]

the people using Entropic might already know those frameworks

This is why I would like to avoid frameworks and stay as vanilla as possible. In order to cast the widest net you need to have the lowest bar of entry, right?

It's true that you need to learn a templating language, and choosing the easiest one is hard. After briefly searching for JS templating languages I think ejs is the simplest, followed by underscore.

_If you want to be super vanilla, you could use with template literals._

But that has its own problems. ```js export default function render({ title, body }) { return ` ${ text } ` } ```

[superhawk610]

Without using an existing framework like React, you have to define your own project-unique set of practices for any frontend code like how to use classnames, how to split files, how each script will be loaded, how code will be shared between routes, etc. That's arguably as much of a learning curve as picking up the basics of React, but it's non-transferable outside of the scope of this project, while knowledge of React is something that many devs already have and will continue to be relevant outside of the scope of this project. Going barebones isn't really lowering the bar of entry, it's just requiring project-specific onboarding instead of framework-specific, which is a net loss IMO.

[workingjubilee]

Speaking as a total newbie outsider who literally got into coding within the past year, has no authority over this project, merely interest, and who does happen to have React experience (as do a lot of new devs -- face it, it's the latest trend which means it's everywhere), that template literal basically just looks like React code I've written and seen written, and writing with it wouldn't make me any more comfortable. I'd just look confusedly at you and ask why I am writing React without writing React.

It matters little but I agree with superhawk610. To manage a project that will become of considerable size, like this one, you will have a framework. You are merely choosing whether you are writing it yourself or using someone else's. The need to have a repeatable, reliable, battle-tested code on every line of a large site is what makes frameworks happen, it's what made Bootstrap happen, it's what made NPM themselves choose Hapi and then React. Entropic is not going to be a small project if it becomes even slightly successful. I may be new, but I can see patterns. If it becomes successful, even if it is decentralized, there will be x100 iterations of its success, and that is not small.

If you want to provide options for people, then just make sure the API is easy to query from multiple kinds of sites (which might mean no GraphQL) and let there be multiple possible sites constructed using that API. That way, anyone can use their preferred templating framework to write their own entropic registry frontend.

[varjmes]

If we're using React, then Next.js (SSR React) or Preact (React, but smol) would be better choices then rocking up with a create-react-app everything and the kitchen sink.

[lacaulac]

@workingjubilee @superhawk610 Thank you, that was exactly what I was trying to convey. If we're to use a framework, we might as well use a well-known one that's gonna be mostly-failproof.

[venikx]

@workingjubilee I disagree with the first point, since I think view libraries like React and Vue are for more expressive than template languages (or the template literal example above), which is (imho) one of the reasons why SPA's became so popular.

I agree with the fact we should use existing frameworks and use those the fullest potential, because they are battle tested. It doesn't matter of it's a robust server- or client-side framework/tool. The most important thing to look out for (again imho) is how much effort it would take to move to something else if it's not working out. It's important we are setting up for success during these many (small or not) iterations. So I think we really should go for something battle tested (in stead of rolling our own everything), but we should be wary of locking ourselves into it too much.

@varjmes Agree 100%, if we decide to go for solution X (with X being something battle tested), then I agree we shouldn't go for a complete kitchen sink approach.

[venikx]

What are you people's thoughts on the Entropic ui being a SPA?

Personally I think it's a nay, purely due to SEO and I don't think the registry feels like Single Page App. I'm fine using a SPA if I'm going to spend a while in it and if it has powerful features. Think of Gmail.

I don't think the registry fills that purpose. If I think about what I do on a site like npm is usually:

1. Search package on npm
2. Read the docs
3. Check how alive it is
4. Close tab and npm install

It's more about content discovery rather than having an application which does an amazing thing.

I'd see a better use-case for an SPA when entropic would include a complicated dashboard for "managing things" purposes. But not for the site that should "replace" npm.

[varjmes]

You can do SEO with SPA, as long as you're SSRing. Add to that, Googlebots are about to or have already become ES6 capable and evergreen.

[maxnordlund]

(I apparently wasn't very clear. I do not want to use the JS templating literal approach. I wanted to show one extreme of the spectrum. That wasn't the greatest of ides, sorry for the confusion)

[venikx]

@varjmes I didn't realize you can also use the SPA term outside of client-side rendering, my bad. Nevertheless, what I'm saying is that I think SSR is key and a purely client-side rendered SPA doesn't seem to fit entropic's usecase.

[superhawk610]

@venikx I agree that the SPA model is a bit at odds with the intended use of a package registry, but I still think the pros of a client-side JS framework (as far as design implementation/iteration) still justify it.

I may be biased from hanging around React Twitter too much but I think you're more approachable to devs when using a frontend framework than a templating language since it's much more declarative/abstracted.

[zacanger]

It kinda sounds like most people are saying the same thing in different words.

• We like out-of-the-box solutions, building your own framework would be a disaster
• SSR seems like a good idea, but we want the slick experience of working in a SPA
• We shouldn't use some giant kitchen-sink solution

Which to me sounds like Gatsby or Next.

[lacaulac]

So, now that we're settled on using a known framework, the question is choosing between doing SSR or CSR and whether or not to use an SPA.

[varjmes]

There is no reason I could fathom why we would choose CSR over SSR, though always happy to be corrected 😊