jkcfg / jk

Configuration as Code with ECMAScript
https://jkcfg.github.io
Apache License 2.0
406 stars 30 forks source link

__filename / __dirname equivalent & pattern for relying on files from typescript modules? #317

Closed shimmerjs closed 4 years ago

shimmerjs commented 4 years ago

I'm writing some configurations in TypeScript that rely on reading a file. I transpile the TypeScript first, and then run jk generate against the output javascript to build my configuration files.

This all works swimmingly, other than one of my jkcfg modules relies on a file, manifests.yaml. This file is stored next to the source, but I could move it. I would like to leverage __filename or __dirname to build relative paths to the external file -- is there a recommended pattern I can use as an alternative?

Additionally, even when I hardcode the path to manifests.yaml so that it would be correct from the javascript file, I get errors regarding reading outside of base path. Everything (source, YAML files, generated output, compiled javascript) lives under kubernetes/. The file exercised by jk generate is kubernetes/dist/clusters, which contains an index.js. I have tried setting the input directory to kubernetes/ so that jk generate can "see" kubernetes/models/cert-manager/manifests.yaml, but that isn't producing the anticipated result.

Am I misunderstanding these flags or totally missing on the correct pattern to do this?

I clean kubernetes/dist/ and kubernetes/generated/ at the start of the generation script so that I always have the most up to date results, so I would like to avoid something like cp kubernetes/models/cert-manager/manifests.yaml kubernetes/dist/models/cert-manager after transpiling the javascript.

squaremo commented 4 years ago

The module @jkcfg/std/resource gives you the same interface as @jkcfg/std/fs, but with paths resolved relative to the importing module. An example of using it is in https://github.com/jkcfg/jk/blob/master/examples/kubernetes/chart-templated/chart/index.js

I think this is what you want -- see if it works for you ..

shimmerjs commented 4 years ago

Thanks for the quick response. I don't see @jkcfg/std/resource in any of the published modules in npm or in the source code at std/ in this repository. Is it generated by building this repo?

errordeveloper commented 4 years ago

@booninite yes, I believe that all of @jkcfg/std packages are built-in and aren't published as npm modules.

shimmerjs commented 4 years ago

Do you know how to make TypeScript code that leverages @jkcfg/std compile correctly? I have been using the published npm packages based on the TypeScript example in the jkcfg/kubernetes repository here: https://github.com/jkcfg/kubernetes/blob/master/examples/guestbook-ts/package.json

errordeveloper commented 4 years ago

Did you also copy 'tsconfig.json' from the same directory? I recall having to adjust my 'tscomfig.json' to match that one and actually I had to use one 'tsconfig.json' for jk part of my project, and then a different on for another part (GitHub actions in my case). If it would help, I maybe able to share more details via Slack DM later.

On Thu, 6 Feb 2020, 11:38 pm alex weidner, notifications@github.com wrote:

Do you know how to make TypeScript code that leverages @jkcfg/std compile correctly? I have been using the published npm packages based on the TypeScript example in the jkcfg/kubernetes repository here: https://github.com/jkcfg/kubernetes/blob/master/examples/guestbook-ts/package.json

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jkcfg/jk/issues/317?email_source=notifications&email_token=AAB5MS46RRFYNQ2NAHRAF7LRBSNNVA5CNFSM4KM4VZF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELBGFWA#issuecomment-583164632, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB5MS4RYDEDB6LIIZAHBGLRBSNNVANCNFSM4KM4VZFQ .

squaremo commented 4 years ago

I filed #319 for documenting how to set TypeScript to generate files you can use with jk. There's also the question of the types -- I don't think we do publish types for @jkcfg/std/resource, so that's problematic for TypeScripters. I'll look into that too.

shimmerjs commented 4 years ago

Finally circling back to this.

Regarding actually finding the file on the filesystem when my code is running via jk, it was a combination of things

I ultimately ended up writing a paths.ts constants file which made the appropriate paths available to my consuming code as constants. This is probably not the greatest idea, as it means I am now coupled to the value of the -i flag, but our repository is structured such that it makes sense to treat the root of our repository as the input directory and rely on that assumption.

Now that I understand the above, I can see how using @jkcfg/std/resource would solve my problem in a more portable way, which takes me to part 2 of this potentially (probably?) useless summary.

My confusion in https://github.com/jkcfg/jk/issues/317#issuecomment-582237545 and https://github.com/jkcfg/jk/issues/317#issuecomment-583164632 stemmed from not understanding the relationship between the @jkcfg/std npm module and the jk tool itself (namely that some of the functionality is provided via the runtime and the npm module is just providing a contract for that functionality so I can write valid code outside of the jk runtime). So when I didn't see @jkcfg/std/resource in the node module, I was confused -- now I understand. My TypeScript setup that I took from the guestbook-ts example was working correctly the entire time.

I believe the only actual problem on this issue was that @jkcfg/std/resource typings needed to be included in the published module, which can be tracked in another issue. I'm surprised they aren't included in the latest release, since the commit that added the types (https://github.com/jkcfg/jk/commit/3f910f2faf81abce46f09013eb334ce2bea08755) came before the v0.4.0 tag. I double checked the contents of the module on npm:

npm pack @jkcfg/std
npm notice
npm notice 📦  @jkcfg/std@0.4.0
npm notice === Tarball Contents ===
npm notice 1.3kB  package.json
npm notice 201B   debug.d.ts
npm notice 382B   debug.js
npm notice 10.3kB flatbuffers.d.ts
npm notice 1.6kB  fs.d.ts
npm notice 3.1kB  fs.js
npm notice 189B   index.d.ts
npm notice 189B   index.js
npm notice 11.5kB LICENSE
npm notice 70B    log.d.ts
npm notice 274B   log.js
npm notice 6.8kB  merge.js
npm notice 591B   param.d.ts
npm notice 2.6kB  param.js
npm notice 243B   parse.d.ts
npm notice 2.8kB  parse.js
npm notice 484B   read.d.ts
npm notice 2.7kB  read.js
npm notice 4.8kB  README.md
npm notice 1.1kB  schema.d.ts
npm notice 1.6kB  schema.js
npm notice 778B   validation.d.ts
npm notice 1.9kB  validation.js
npm notice 641B   write.d.ts
npm notice 3.1kB  write.js
npm notice 256B   cmd/generate-module.js
npm notice 842B   cmd/generate.d.ts
npm notice 8.4kB  cmd/generate.js
npm notice 153B   cmd/transform-exec.js
npm notice 193B   cmd/transform-module.js
npm notice 132B   cmd/transform.d.ts
npm notice 1.5kB  cmd/transform.js
npm notice 148B   cmd/validate-exec.js
npm notice 190B   cmd/validate-module.js
npm notice 199B   cmd/validate.d.ts
npm notice 1.7kB  cmd/validate.js
npm notice 42.0kB internal/__std_generated.d.ts
npm notice 60.4kB internal/__std_generated.js
npm notice 381B   internal/data.d.ts
npm notice 1.4kB  internal/data.js
npm notice 320B   internal/deferred.d.ts
npm notice 4.4kB  internal/deferred.js
npm notice 33.4kB internal/flatbuffers.js
npm notice 191B   internal/rpc.d.ts
npm notice 2.8kB  internal/rpc.js
npm notice === Tarball Details ===
npm notice name:          @jkcfg/std
npm notice version:       0.4.0
npm notice filename:      jkcfg-std-0.4.0.tgz
npm notice package size:  39.0 kB
npm notice unpacked size: 218.1 kB
npm notice shasum:        d2ab6c6820c13b7af2da05f747a4f6273574cc7f
npm notice integrity:     sha512-KrGxODI0hdQlR[...]OT7WVwtmV1jNA==
npm notice total files:   45
npm notice
jkcfg-std-0.4.0.tgz

Thanks for the help and the great project : )

squaremo commented 4 years ago

It is really useful to hear how people reason their way through using a piece of software, and this is a fine example of the form :-) Thank you @booninite!