Project organization

[bkniffler]

Hey, I really like nano-sql! Maybe it would make sense to make nano-sql a mono-repository, this way all packages (sqlite etc) could be maintained in a single place and changes to core, plugins, adapters could always be tested agains the whole repository. This could be coupled with scoped packages (@nano-sql/sqlite).

Also Currently the whole feature is hard coded with english language into the core. Wouldn't it make more sense to make it all a plugin that gets initiated with the tokenizer? This way we could add more languages.

[only-cliches]

Hello Benjamin,

Bringing all the adapters into the same git repo is a good idea, I didn't know about .npmignore until recently so it should be very doable!

I haven't documented the search feature yet, custom tokenizers can be easily passed into nanoSQL to be used instead of the built in english tokenizer. The english tokenizer is hard coded because english speaking countries are by far the majority on GitHub/NPM and it contributes minimally to nanoSQL's bundle size.

You can easily pass in your own tokenizer with the config object:

nSLQ().config({
    tokenizer: (table, column, args, value) => {
        /**
         * Arguments:
         * table: the table that the row is in being tokenized
         * column: the current column being tokenized
         * args: array of arguments passed into the data model.  For example with props: ["search(english, 3)"] you would get ["english", "3"].
         * value: the string to tokenize, will be the entire unmodified column value.
         *
         * Expects:
         * Array of objects, each object contains three properties:
         * o: original word
         * w: tokenized word
         * i: zero based index of this word's location in the provided string
         *
         * You can also optionally return false to let the default tokenizer take over.
         */
        return value.split(" ").map((w, i) => ({w: w, o: w, i: i}));
   }
}).connect()..

[bkniffler]

Ah, thanks for the info. I'd still take them out (+ the whole fuzzysearch feature as plugin) since bundle size matters to many people a lot, but its just a recommendation!

Something like:

import nano, { DatabaseEvent, NanoSQLInstance } from "@nano/sql";
import fuzzy, { en as tokenizer } from "@nano/plugin-fuzzy"; // taking advantage of tree shaking and esmodules
import adapter from "@nano/adapter-sqlite";

nano().config({
    tokenizer,
    adapter
}).connect()

About the structure, I thought about a mono repo (using https://github.com/lerna/lerna) approach like https://github.com/gatsbyjs/gatsby, https://github.com/babel/babel, https://github.com/facebook/react etc (look into their respective 'packages' folders). I'd recommend putting all related projects in there, not only adapters. Its root would house the tsconfig, linting, tooling into the root so it can be shared by all projects and to reduce duplication.

So:

.gitignore
package.json
tsconfig.json
tslint.json
examples
- react
- ...
packages
- adapter-sqlite
  - lib
  - src
  - .npmignore
  - package.json
- sql or core
- react
- vue
- plugin-fuzzysearch
- plugin-sync
- ....

Here is a simple example and explanation of the advantages of lerna: https://github.com/reggi/lerna-tutorial

Also you could consider using yarn and their workspaces feature https://yarnpkg.com/blog/2017/08/02/introducing-workspaces/

[only-cliches]

1.7.0 has centralized all the "stem" projects into a single git repo (this one). I'll delete the other repos and publish new packages to NPM with updated details soon.

[heri16]

Monorepos are great

[only-cliches]

The transition is almost complete. New versions of all the sub projects have been published to NPM with updated github links, I've archived all the old git repos and copied over any open issues.

I'll probably wait another week then delete the git repos, we can close this issue once that happens.

[only-cliches]

All of the old git repos have been removed, closing this issue now since we've fully migrated to the new monorepo setup.

[codebling]

Can someone give some hints for building? lerna complains:

lerna ERR! ENOPKG `package.json` does not exist, have you run `lerna init`?

I'm able to get lerna somewhat working using these steps:

lerna init
lerna bootstrap
npm i -D typescript webpack ts-node

I'm still not able to build (lerna run build) or run tests (lerna run test) successfully. I'm running into issues with

> nano-sql-vue@1.3.0 build /tmp/nano2/packages/Nano-SQL-Vue
> tsc && webpack -p

../../node_modules/@types/node/assert.d.ts(8,68): error TS1144: '{' or ';' expected.
../../node_modules/@types/node/base.d.ts(13,1): error TS1084: Invalid 'reference' directive syntax.
../../node_modules/@types/node/events.d.ts(43,56): error TS1005: '=' expected.
../../node_modules/@types/node/ts3.6/base.d.ts(10,1): error TS1084: Invalid 'reference' directive syntax.

Parts of the log have been clipped for brevity. These are all type errors, most of them for the node types, so I suspect there is a Typescript version mismatch, but maybe I'm doing something else wrong.

Any suggestions would be appreciated