Closed jaydenseric closed 2 years ago
conartist6
commented
3 years ago As soon as possible! And indeed I will make it 1.0.0 when I'm ready to publish. I just need to set up code coverage and write some more tests, then hopefully I could get it published today or tomorrow.
conartist6
commented
3 years ago We're getting there...
86 tests passed
-------------------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-------------------------|---------|----------|---------|---------|-------------------
All files | 95.1 | 79.69 | 88.03 | 95.1 |
src | 94.73 | 75 | 100 | 94.73 |
error.js | 96.36 | 68.75 | 100 | 96.36 | 28-29
errors.js | 92.72 | 80 | 100 | 92.72 | 14-15,36-37
index.js | 100 | 100 | 100 | 100 |
src/internal | 100 | 100 | 100 | 100 |
header.js | 100 | 100 | 100 | 100 |
src/v8 | 93.93 | 86.02 | 95.23 | 93.93 |
error.js | 100 | 88.88 | 100 | 100 | 17,31,55
errors.js | 79.06 | 76.47 | 80 | 79.06 | 36-42,64-68,74-79
frame.js | 98.65 | 87.5 | 100 | 98.65 | 36-37
index.js | 100 | 100 | 100 | 100 |
src/v8/internal | 98.3 | 95.45 | 100 | 98.3 |
frame-shared.js | 100 | 100 | 100 | 100 |
frame-strict.js | 100 | 100 | 100 | 100 |
header.js | 96.07 | 91.66 | 100 | 96.07 | 41-42
src/v8/internal/nearley | 95.19 | 71.17 | 82.43 | 95.19 |
error.js | 93.87 | 78.94 | 78.57 | 93.87 | 4,47-48
frame-strict.js | 97.14 | 69.76 | 93.1 | 97.14 | 68-69
frame.js | 97.01 | 62.85 | 70.37 | 97.01 | 65-66
util.js | 90.69 | 85.71 | 100 | 90.69 | 27-28,37-38
-------------------------|---------|----------|---------|---------|-------------------
Done in 3.76s. 91 tests passed
-------------------------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
-------------------------|---------|----------|---------|---------|-------------------
All files | 98.6 | 82.67 | 88.98 | 98.6 |
src | 100 | 82.85 | 100 | 100 |
error.js | 100 | 76.47 | 100 | 100 | 17,23,41,48
errors.js | 100 | 88.23 | 100 | 100 | 33,48
index.js | 100 | 100 | 100 | 100 |
src/internal | 100 | 100 | 100 | 100 |
header.js | 100 | 100 | 100 | 100 |
src/v8 | 100 | 89.79 | 100 | 100 |
error.js | 100 | 89.28 | 100 | 100 | 17,31,55
errors.js | 100 | 90.47 | 100 | 100 | 22,54
frame.js | 100 | 89.58 | 100 | 100 | 44-45,50-51,131
index.js | 100 | 100 | 100 | 100 |
src/v8/internal | 100 | 100 | 100 | 100 |
frame-shared.js | 100 | 100 | 100 | 100 |
frame-strict.js | 100 | 100 | 100 | 100 |
header.js | 100 | 100 | 100 | 100 |
src/v8/internal/nearley | 95.19 | 71.68 | 82.43 | 95.19 |
error.js | 93.87 | 78.94 | 78.57 | 93.87 | 4,47-48
frame-strict.js | 97.14 | 69.76 | 93.1 | 97.14 | 68-69
frame.js | 97.01 | 62.85 | 70.37 | 97.01 | 65-66
util.js | 90.69 | 87.5 | 100 | 90.69 | 27-28,37-38
-------------------------|---------|----------|---------|---------|-------------------I'm also going to need to think a little about what environments I intend to support. I'm not too worried about releasing with some of those kind of bugs left in though because they'll always be fixable in patch releases.
jaydenseric
commented
3 years ago Please support Node.js ^12.20.0 || ^14.13.1 || >= 16.0.0, and declare this in the package engines field. This matches the minimum requirements for a lot of tools now like ESLint, etc. and it allows you to work with Node.js ESM. The package engines applies to both dev and prod installs of your package. Try npx npm-check-engines within the package project to verify correctness.
Please support Node.js ESM, and maybe even don't publish CJS at all to avoid the dual package hazard. I write and deploy most of my backend, all dev tool, and some (soon to be most) front end packages in pure ESM. I'm also starting to use some npm packages in Deno via native import maps and https://ga.jspm.io , and the more standard the modules are the better.
Please consider not publishing index files with multiple exports in them at all. Force people to deep import each individual thing, so only the code that is necessary is loaded and used in your app without any sketchy "tree shaking" build tool hacks. Or at least make it possible for people who want to be efficient to do so, if you want to keep indexes; don't internally import things from indexes. I already support deep imports for everything I publish, and plan in the near future to remove the index modules.
conartist6
commented
3 years ago The dual package hazard is minor in stateless code so my inclination is to offer commonjs, particularly because it's quite possible that a lot of code will be requiring from commonjs modules, even in engines which support esm. In a cjs module that means using await import(), which is an unacceptable limitation in that environment.
I've played around a good bit with how much importing I can allow inside the library. My current intent is for the possible imports to be:
stack-toolsstack-tools/errorstack-tools/errorsstack-tools/v8stack-tools/v8/framestack-tools/v8/errorstack-tools/v8/errorsstack-tools/node/framestack-tools/node/errorstack-tools/node/errorsMost important to me is that the v8 stuff (which involves loading a whole parser) isn't included if you don't import inside from /v8. I've made sure is the case. A lot of the other files tend to reference each other so there won't be much savings from importing /v8/error instead of /v8/errors or even /v8. That's because v8/frame is by far the biggest part of it, and all the others depend on it.
conartist6
commented
3 years ago Fortunately parsing errors with their stack traces is the kind of thing that is only likely to be done in developer environments which will tend to have more memory, bandwidth, and processing power. If you're in a production environment you shouldn't be meddling with stack traces and it should be very cheap to import from stack-tools, which contains in its import tree only ~110 LOC in two files.
conartist6
commented
3 years ago I'm still considering eliminating the possibility of importing anything other than stack-tools, stack-tools/v8 and stack-tools/node. I don't see that I really gain anything by it though.
jaydenseric
commented
3 years ago None of those import paths you listed (e.g. stack-tools/node/frame) will actually work, since they don't include file extensions.
Maybe you imagine it working like this:
import { isInternalFrame } from 'stack-tools/node/frame';But looking at the code you have currently, you will actually you have to do this:
import StackToolsNodeFrame from 'stack-tools/node/frame.js';
const { isInternalFrame } = StackToolsNodeFrame;Imagine you just want to import the isInternalFrame function as an implementation detail of another tool you want to publish:
import StackToolsNodeFrame from 'stack-tools/node/frame.js';
const { isInternalFrame } = StackToolsNodeFrame;Instead of ideally only parsing and loading into memory this code (24 lines total):
You are parsing and loading all this extra bullshit as well:
Which also pulls in the entire contents of these modules:
https://unpkg.com/moo@0.5.1/moo.js https://unpkg.com/nearley@2.20.1/lib/nearley.js
Fucking thousands of lines of undesired code. Do you get the point?
is the kind of thing that is only likely to be done in developer environments which will tend to have more memory, bandwidth, and processing power
You're basically saying you don't care about wasting everyone's system resourses. I'm about to lose my cool; this is common sense yet I have to politely explain it again and again year after year and the npm ecosystem never gets any less fucked. Import something from one package, and it has a few thousand lines of extra bullshit pulled in, and it's dependences each pull in a few thousand lines of bullshit, and then third or fourth dependency layers do the same, and before you know it it takes 0.5 GB of JS to colorize a word or do a deep clone, or some other mundane thing that should be < 100 lines total. Do 1 export per module, and everything efficiently works itself out 100% of the time. Sorry to take it out on you, but I'm reaching a breaking point.
You have an opportunity to do it right in a brand new package. But, do whatever you want man.
conartist6
commented
3 years ago Looooooool.
I don't think you get the point though. isInternalModule is a function whose input is a parsed stack frame. You're complaining that I loaded all the code that parses stack frames. /parse and /print may be a better split for allowing you to avoid loading undesired code, but I'm nervous about the idea that parsed stack frames would ever be persisted or would travel over the wire. After all, there is no specification for a parsed stack frame. I just made this one up, and it's very possible that I will need to make breaking changes to it. For this reason I expressly state in the documentation that the only supported input to methods expecting parsed errors is output from a parser imported from the same package. I may end up being willing to relax this restriction, but for the moment it seems better to wait and let people who are using the tool in ways I haven't really imagined yet come to me.
conartist6
commented
3 years ago You're absolutely right though that you should not be pulling thousands of lines of parser code into a production environment to be run by all your users.
conartist6
commented
3 years ago I can put that admonition in the readme if you'd like!
conartist6
commented
3 years ago The only code that should be in a production environment is imports from stack-tools (not stack-tools/v8 ever). Again just 210 loc, and gives you basic abilities like printing causal stacks of errors and slicing error-handling frames out of stacks when you know a fixed number of frames needing to be sliced. It works in any environment.
Also as for the imports inside the library they aren't done. First I'm testing the implementation, then I'll design and test the packaging and importing
conartist6
commented
3 years ago I get that the javascript ecosystem makes everyone a little crazy. I live here too, and I've shoveled my share of shit that should never have been allowed to exist. I've sworn and complained just as loudly to the people around me. But that said,
Fucking thousands of lines of undesired code. Do you get the point?
is not appropriate here.
conartist6
commented
2 years ago I now have 100% coverage outside the parser, and have resolved several issues that I discovered during testing.
jaydenseric
commented
2 years ago I apologize for the aggressive language yesterday.
My patience and resolve is being tested having been working for nearly a year now, unpaid, on some very challenging technical problems that when published I hope will reduce a lot of the complexity in JS projects.
I wish success for you, both in this project and your pursuit of happiness, so don't let it dampen your motivation 💪
conartist6
commented
2 years ago That's cool! What is it that you're working on?
I've been working on a project I could describe the same way, macrome. The idea is to help users pull all the magic out of their environment -- magic like JSX, CSS module imports, macros, and all kinds of other things. I think this has all kinds of cool implications like allowing React components to be packaged in a universally accessible way, allowing git branches to be used as packages, making it easier for new devs to understand how things really work, making it easier to understand reported errors, and even just eliminating needs for certain kinds of configuration e.g. in your test runner.
jaydenseric
commented
2 years ago macrome seems quite novel; looks like you've put in a fair bit of elbow-grease!
I've worked on a lot of open source this year, but the big one that's been taking months is a from-scratch total rethink of a universal JS SSR web application framework. It is based on Deno, which is very web standards aligned and supports HTTP import statements and import maps in a similar way to browsers, opening up all sorts of exciting technical opportunities.
With this framework there is zero installation, builds, bundling or transpilation, and almost no configuration. There is complete type safety for both for the framework source and your project code via JSDoc comments and Deno's out of the box ability to check types. View-source shows exactly the source you're working on in the editor, no source-maps needed!
The framework itself is almost entirely runtime code, and the dev tooling is almost all out of the box Deno powered. No more megabytes of JS across thousands of insecure dev dependencies to work on an app!
If you navigate a typical web app based on bundled code, e.g. via Next.js, with the Chrome dev tools coverage tab open you will see that around 50% (sometimes drastically more) of the code loaded is unused. Often the same code is downloaded multiple times across split bundles. My framework only loads on demand 100% used code, using regular imports. The runtime is absolutely tiny yet allows for much more advanced routing than Next.js/react-router projects by using a very simple convention of regex matching and dynamic imports.
Out of the box, you can isomorphically load data in components anywhere in your app (not just at the page level like in Next.js) and have it hydrated on the client after SSR. There are hooks for loading GraphQL easy-peasy, although you can make custom loaders to get any kind of data.
Generally the API design is, you import something to use that feature, so if you are not importing and using features you don't have a lot of runtime code in production.
The head tags are managed via proper Preact rendering in the head (without interfering with head tags not managed via the head tag manager), so there is no weird buggy diffing and custom rendering going on in the head like with Next.js or React helmet.
It's based around Preact for technical elegance and performance, but since dependencies are managed via project import maps you can theoretically swap it out for React.
One of the final pieces of the puzzle is the system for making sure a route's CSS files are loaded before completing a client-size route navigation. It's 90% there but it turns out even simple aspects like finding out if CSS is loaded or not are quite challenging.
Deno is still evolving so there are a few bugs, particularly around type checking .js files with JSDoc, that hopefully are resolved around the time I'm ready to publish.
conartist6
commented
2 years ago That certainly sounds great to me. I've always thought sourcemaps were a godawful technology, mostly because they cause you to read one set of code while interacting with the runtime environment for another. And in hiding the complexity of code, you reduce the pressure on yourself to maybe, like, not ship godawful ugly code?
So yeah your thing sounds lovely and very well thought out. It is my dream to someday be able to do this job without it making me feel like an insane person.
conartist6
commented
2 years ago @jaydenseric I think you'll be interested in the now-resolved #4. I think the things I was trying to explain to you about how things are broken down should now be more clear to everyone.
conartist6
commented
2 years ago @jaydenseric I am also finally looking at how to offer esm support inside this package. I know you advocated for files with only a single export, but I think that's a bit unrealistic. I could do it yes, but it's not feasible for the ecosystem. Instead I think I'm going to follow node's own modules documentation to the letter, for example having:
import exports from './error';
export const parseError = exports.parseError;
export const printErrorHeader = exports.printErrorHeader;
export const printFrames = exports.printFrames;
export const printError = exports.printError;While it's a little opaque to static analysis, I'm going to cross my fingers and hope that the fact that node explicitly offers this as the example of how to make a dual package is good enough to buy support in most major tools doing static analysis.
conartist6
commented
2 years ago 
I haven't done a complete survey, but VSCode understands what's going on here just using the js language service -- note the green text indicating that it understands these are functions.
conartist6
commented
2 years ago And one final note -- as of now trunk should be a valid git package. I'll publish soon, but anyone can try it out that way.
jaydenseric
commented
2 years ago I know you advocated for files with only a single export, but I think that's a bit unrealistic. I could do it yes, but it's not feasible for the ecosystem.
I'm not understanding why you would say that. My packages, some simple, and some incredibly complex, get installed hundreds of millions of times per year and they are all composed of single export modules. If you provide index modules as well as supporting deep imports, everybody's a winner and people that like to just import everything from a bare module specifier via the main index module can't tell the difference in how it's implemented.
Regarding https://github.com/stack-tools-js/stack-tools/issues/1#issuecomment-964380744, don't make the ESM index module import from the CJS index module. Both index modules should directly deep import the things they export, e.g:
This way you can avoid an extra module layer when importing via the ESM index that would slightly increase the bundle size and add and extra HTTP round trip to a HTTP import in browsers or Deno.
conartist6
commented
2 years ago I'm saying it because the import spec allows both named and default exports. To say the whole ecosystem should standardize on implementations in single-export files to adopt the new module system is absurd in my mind, because part of the point of the newer system is a level of flexibility which would be completely defeated. It's just awful devx for anything vaguely complicated to be juggling files to keep to the one-export rule. Yes it works OK if you know exactly what you need to build, but very few things are ever built that way. I'd much rather do the best I can in a true mixed ecosystem, then be able offer to offer a better user experience in return for being able to drop some of the outdated compatibility concerns.
And indeed I do have only one index layer -- my index imports directly from the files which define exported methods, though those files contain more than one definition apiece. I don't see why I would worry about round trips. In my mind anyone who really cares about optimal perf should bundle. That makes a lot more sense to me than saying that directory structure needs to be just-so because otherwise perf will suffer.
conartist6
commented
2 years ago Also I'm willing to put some of the burden of understanding these slightly more complicated syntaxes on tool maintainers in part because I am someone who is contributing to the maintenance of tools.
conartist6
commented
2 years ago I just don't see why you would fight to make things worse than they are for the only people who can make things better than they are (package authors). I mean in theory I appreciate it that you're going for a perfect, pure setup, but in practice I don't* think that prioritizing mild gains in environments making network requests per import will lead to superior outcomes when compared with encouraging engineers to value their own time and energy over some amount of waste in computer resources. I don't mean we should just stop giving a damn and code things with the totally wrong algorithmic complexity or anything, just that a bit of waste is sometimes the best way to navigate a critical juncture, beyond which lies a better future for all involved.
*edited, originally said the opposite
conartist6
commented
2 years ago Idunno man, I guess there's not much point in arguing about it though. We're each building things, and time will tell. There's room for everyone in this ecosystem.
conartist6
commented
2 years ago The packages are now published to NPM as v0.1.0. I need them published to put up a formal PR for integration with Jest, which I have now completed primary development on. I will bump the version to 1.0.0 prior to merging that PR since usage by Jest clearly constitutes a stable API.
It seems this package is not published to npm yet:
https://www.npmjs.com/package/stack-tools
When will it be up?
BTW, I see you plan to start at v0.1.0:
https://github.com/conartist6/stack-tools/blob/194ae3675a35e9e79746580c357cff5a1c0231a9/package.json#L3
Please consider starting at v1.0.0; updates are so much nicer for consumers to manage when the dependency version has a semver major. There is literally no downside to starting at v1.0.0, you can always publish v2, v3, etc. within a short period of time if you need to publish breaking changes.