Integrate to a monorepo

[nicolas377]

The building process is pretty confusing, since we're having to build across multiple repos and from release branches. It would probably be a good idea to keep all the code for the scripts in one monorepo, since that would make the build process a lot simpler.

List:

1. Prettier doesn't do nested ignore files, so infra/builds/prettier_build.js builds nested ones into one global .prettierignore file.
2. We should build scripts first, so here's what needs to happen for each script:
   1. Change the working directory to the scripts main dir (not the source folder).
   2. Run tsc in that dir (using a child process).
   3. Run r.js optimizer programmatically. (the optimizer will save to the extension/source/scripts dir)
3. We'll need to build the extension after that, here's how we'll do that:
   1. Change the working directory to the scripts main dir (not the source folder).
   2. Copy the source folder to the build folder.
   3. Run tsc in the build dir (again, using a child process).
   4. Remove the comments from all the .js files. (use the removeComments option on tsc).
   5. Delete all the .ts files (glob search, then run through the array of files and delete them).

[Guy-Adler]

How are you planning on doing this? Saving the source code of the extensions with the source code of the chrome extension (and writing adding the r.js optimizer to the build process) or having the source code of the extensions outside of the source of the chrome extension, and copying the distribution file of each extension to the source of the chrome extension? I think converting to a monirepo is a good idea, we just need to figure out how we want to structure it.

[Guy-Adler]

By the way, what do you think of adding the build folder to .gitignore? It doesn't include anything you can't get in another way...

[nicolas377]

I'm thinking of separating the extension, ap++, fmc, spoiler arming, and keyboard mapping into their own folders. They'd have their own infrastructure for building. The scripts would instead of building into their dist folder, build directly into the source of the chrome extension. I'm trying to make a visualization of that the repo file system would look like. Also, I do like the idea of ignoring the build folders, since they don't seem to do anything.

[nicolas377]

Here's something I whipped together in a few minutes

[nicolas377]

[Guy-Adler]

what do we need the built code of the extension for?

[nicolas377]

we can .gitignore it, but that's what chrome will use for the extension

[nicolas377]

The scripts can have a build folder that deletes itself after they build. The could also have r.js build directly to the chrome extension source folder.

[Guy-Adler]

I don't think there is a reason to delete the build folders, just ignore them in .gitignore. r.js should probably just build to source/scripts

[nicolas377]

So we'd just .gitignore **/build, and r.js would build the scripts to extension/source/scripts.

[Guy-Adler]

sounds good

[nicolas377]

Most of the friction of switching to a monorepo will be building the infrastructure. This will be something I'll be working on for a while. I'll definitely be getting some decent work on this done this weekend, since I won't have any soccer games. Hopefully this can get done in about a week. I'll write out some ideas for the building infrastructure and post the ideas that I like here.

[nicolas377]

I'll work on this here this weekend. I've got a few school projects I have to do, and a few commitments on Friday, but the first commits will land on that branch on Saturday.

[nicolas377]

Turns out that prettier only uses the .prettierignore file at the project root, and doesn't respect nested .prettierignore files. We could build the root .prettierignore file from nested ones, so I'm working on that now.

[nicolas377]

@types/node collides with @types/requirejs, so tsc will get insanely mad at us (almost 80 errors) while compiling the .js files. The files will still compile, but the errors will fail CI. There's no solution for this, and neither typescript or definitelytyped have accepted solutions coming anytime soon. I'm still trying to figure out a way to keep @types/node from leaking into places where it's not needed, but this is super annoying, and will make the repo very unstable.

[Guy-Adler]

While on the topic of types, geofs.d.ts needs to update to one of the other repos' file, since the global object is changed.

[nicolas377]

For later reading: https://stackoverflow.com/questions/56690600/resolve-conflicting-definitions-for-node-ts4090-error-in-vs-2017

[Guy-Adler]

Can you please explain how to build the project? As of right now, I get an error upon running npm run build (Error: Cannot find module './builds/prettier'). Are some files ignored by git?

[nicolas377]

That error is because infra/builds/prettier.js got renamed to infra/builds/prettier_build.js. We just need to change that import in infra/build.js. Here's what I'm thinking right now:

1. Prettier doesn't do nested ignore files, so infra/builds/prettier_build.js builds nested ones into one global .prettierignore file.
2. We should build scripts first, so here's what needs to happen for each script:
   1. Change the working directory to the scripts main dir (not the source folder).
   2. Run tsc in that dir (probably using a child process that also needs to change working dirs).
   3. Run r.js optimizer programmatically.
   4. In the r.js optimizer callback, save the file (we can append the global object here) to the source of the extension.
3. We'll need to build the extension after that, here's how we'll do that:
   1. Change the working directory to the scripts main dir (not the source folder).
   2. Copy the source folder to the build folder.
   3. Run tsc in the build dir (again, probably using a child process that also needs to change working dirs).
   4. (?) Remove the comments from all the .js files. (would be a glob search, then running through the files and removing comments on each one).
   5. Delete all the .ts files (glob search, then run through the array of files and delete them).

I haven't finished building all the infrastructure yet. This afternoon, I'll burn down all the infrastructure (don't arrest me for arson plz), and rebuild it all. I get out of school 25 minutes after I post this, so commits should start landing pretty soon.

[Guy-Adler]

ii. Copy the source folder to the build folder. iii. Run tsc in the build dir (again, probably using a child process that also needs to change working dirs).

Can't we just set tsc to compile to the build folder using the outDir compiler option?

iv. (?) Remove the comments from all the .js files. (would be a glob search, then running through the files and removing comments on each one).

We can add "removeComments": true to compilerOptions in tsconfig.json to remove the comments automatically when running tsc.

[nicolas377]

The chrome extension includes other files that aren't .ts files, so they also need to be copied over. The best solution then turns to copying to the build dir, and compiling there. For the scripts, we can just run tsc on them. For removing comments, I'll check that out when I get to the extension infrastructure.

[nicolas377]

The solution to the conflicting types problem was just to manually enter types into the tsconfig.json file. Here's how I did that:

1. Added this to tsconfig,json:

   "compilerOptions": {
   "types": []
   }

2. Ran tsc, and saw what type errors popped up.
3. Whatever types were needed got added to the types array. (i also had to make some small changes to global types)
4. No more type errors!

I got that done and committed it last night, just forgot to push it. I'll push it when I get home (probably about 5:30 EDT). I've gotten keyboard mapping and fmc consistently building. (they're both building to fmc's dist dir though, i'll look into that) Eslint is still insanely pissed off at us, but I'll deal with that after I get all the scripts building to the extension and the extension building.

[nicolas377]

the r.js optimizer is pulling modules from different scripts. Instead of fmc building from only its build directory, and requiring from ap++ at runtime, r.js pulls from the ap++ directory at build time and includes the ap++ modules that would normally get required at runtime. I'm trying to figure out how to keep r.js from searching above the current directory tree, and only build from the build folder.

[nicolas377]

I've fixed that now on the integration branch. The scripts and their map files aren't building to the extension yet, but I can fix that pretty quickly today.

[nicolas377]

Right now, we disable the no-undef rule while configuring requirejs. We could add a global to .eslintrc like this.

[Guy-Adler]

Can we move build.json to the infra folder? It is currently ignored and there is no need to create 4 duplicates of it. Also, can we move to import / export rather than require and module.exports? Just to make the code cleaner and in style with the scripts' code. Edit: maybe putting the object directly in the build script will be better, since we will be able to programmatically change it (for example, running npm run debug will run the optimizer without uglyfing to help debugging scripts).

[nicolas377]

I'm working on different scripts for building each script. It'll be pretty easy to make sure r.js doesn't uglify with some extra code that reads flags. Also, @Guy-Adler, could you upgrade the @typescript-eslint family to its rc-v5 tag on main?

[nicolas377]

Actually, could you just upgrade typescript-eslint? They've published v5 to their main.